Specialized Face Perception Mechanisms Extract Both Part and Spacing Information: Evidence from Developmental Prosopagnosia

Face naming and recollection represent key memory deficits in developmental prosopagnosia

Previous studies have found that face perception deficits do not fully account for the severity of face recognition deficits in developmental prosopagnosia (DP). Researchers have begun identifying deficient memory mechanisms such as impaired face recollection, but these findings require replication, and further characterization of additional memory deficits is necessary. Our goals were to replicate prior findings of face recollection impairment in DP and extend these findings to assess different types of face associative memory. We had 69 DPs and 99 controls perform a face perception battery as well as three face memory tasks: 1) Old/New task with confidence ratings to calculate recollection and familiarity using ROC analysis, 2) Face/Scene task to examine remember-know judgments and contextual memory for faces, and 3) Face-Name/Occupation task to assess the ability to learn semantic associations with faces. Compared to controls, DPs showed poorer recollection and familiarity across both Old/New and Face/Scene tasks as well as reduced scene accuracy for correct faces. Of these differences, only Old/New recollection remained significant after controlling for group differences in face perception abilities. In the Face-Name/Occupation task, after controlling for face perception, DPs showed poorer recall of names than controls but performed similarly in recalling occupations. Finally, we found that DPs with major, mild, and no face perception deficits showed consistent impairments in Old/New recollection and face-naming, and larger perceptual deficits were associated with larger memory deficits. Together, these results provide several mechanistic insights into the nature of memory deficits in DPs and have diagnostic and treatment implications.

Memristive patch attention neural network for facial expression recognition and edge computing

Facial expression recognition has made a significant progress as a result of the advent of more and more convolutional neural networks (CNN). However, with the improvement of CNN, the models continues to get deeper and larger so as to a greater focus on the high-level features of the image and the low-level features tend to be lost. Because of the reason above, the dependence of low-level features between different areas of the face often cannot be summarized. In response to this problem, we propose a novel network based on the CNN model. To extract long-range dependencies of low-level features, multiple attention mechanisms has been introduced into the network. In this paper, the patch attention mechanism is designed to obtain the dependence between low-level features of facial expressions firstly. After fusion, the feature maps are input to the backbone network incorporating convolutional block attention module (CBAM) to enhance the feature extraction ability and improve the accuracy of facial expression recognition, and achieve competitive results on three datasets CK+ (98.10%), JAFFE (95.12%) and FER2013 (73.50%). Further, according to the PA Net designed in this paper, a hardware friendly implementation scheme is designed based on memristor crossbars, which is expected to provide a software and hardware co-design scheme for edge computing of personal and wearable electronic products.

Perceptual heterogeneity in developmental prosopagnosia is continuous, not categorical

Developmental prosopagnosia (DP) is associated with considerable perceptual heterogeneity, though the nature of this heterogeneity and whether there are discrete subgroups versus continuous deficits remains unclear. Bennetts et al. (2022) recently found that holistic versus featural processing deficits distinguished discrete DP subgroups, but their sample was relatively small (N = 37), and subgroups were defined using a single task. To characterize perceptual heterogeneity in DPs more comprehensively, we administered a broad face perception battery to a large sample of 109 DPs and 134 controls, including validated measures of face matching (Cambridge Face Perception Test - CFPT, Computerized Benton Facial Recognition Test, Same/Different Face Matching Task), holistic processing (Part-Whole Task), and feature processing (Georges Task and Part-Whole part trials). When examining face matching measures, DPs exhibited a similar distribution of performance as controls, though shifted towards impairment by an average of 1.4 SD. We next applied Bennetts (2022) hierarchical clustering approach and k-means clustering to the CFPT upright, inverted, and inversion index measures, similarly finding one group of DPs with poorer inverted face performance and another with a decreased face inversion effect (holistic processing). However, these subgroup differences failed to generalize to other measures of feature and holistic processing beyond the CFPT. We finally ran hierarchical and k-means cluster analyses on our larger battery of face matching, feature, and holistic processing measures. Results clearly showed subgroups with generally better versus worse performance across all measures, with the distinction between groups being somewhat arbitrary. Together, these findings support a continuous account of DP perceptual heterogeneity, with performance differing primarily across all aspects of face perception.

Disentangling developmental prosopagnosia: A scoping review of terms, tools and topics

The goal of this preregistered scoping review is to create an overview of the research on developmental prosopagnosia (DP). Through analysis of all empirical studies of DP in adults, we investigate 1) how DP is conceptualized and defined, 2) how individuals are classified with DP and 3) which aspects of DP are investigated in the literature. We reviewed 224 peer-reviewed studies of DP. Our analysis of the literature reveals that while DP is predominantly defined as a lifelong face recognition impairment in the absence of acquired brain injury and intellectual/cognitive problems, there is far from consensus on the specifics of the definition with some studies emphasizing e.g., deficits in face perception, discrimination and/or matching as core characteristics of DP. These differences in DP definitions is further reflected in the vast heterogeneity in classification procedures. Only about half of the included studies explicitly state how they classify individuals with DP, and these studies adopt 40 different assessment tools. The two most frequently studied aspects of DP are the role of holistic processing and the specificity of face processing, and alongside a substantial body of neuroimaging studies of DP, this paints a picture of a research field whose scientific interests and aims are rooted in cognitive neuropsychology and neuroscience. We argue that these roots - alongside the heterogeneity in DP definition and classification - may have limited the scope and interest of DP research unnecessarily, and we point to new avenues of research for the field.

The heterogeneity of holistic processing profiles in developmental prosopagnosia: holistic processing is impaired but not absent

Although it is generally assumed that face recognition relies on holistic processing, whether face recognition deficits observed in Developmental Prosopagnosics (DPs) can be explained by impaired holistic processing is currently under debate. The mixed findings from past studies could be the consequence of DP's heterogeneous deficit nature and the use of different measures of holistic processing-the inversion, part-whole, and composite tasks-which showed a poor association among each other. The present study aimed to gain further insight into the role of holistic processing in DPs. Groups of DPs and neurotypicals completed three tests measuring holistic face processing and non-face objects (i.e., Navon task). At a group level, DPs showed (1) diminished, but not absent, inversion and part-whole effects, (2) comparable magnitudes of the composite face effect and (3) global precedence effect in the Navon task. However, single-case analyses showed that these holistic processing deficits in DPs are heterogeneous.

Familial Transmission of Developmental Prosopagnosia: New Case Reports from an Extended Family and Identical Twins

Existing evidence suggests that developmental prosopagnosia (DP) is a surprisingly prevalent condition, with some individuals describing lifelong difficulties with facial identity recognition. Together with case reports of multiple family members with the condition, this evidence suggests that DP is inherited in at least some instances. Here, we offer some novel case series that further support the heritability of the condition. First, we describe five adult siblings who presented to our lab with symptoms of DP. Second, for the first known time in the literature, we describe a pair of adult identical twins who contacted us in the belief that they both experience DP. The condition was confirmed in three of the five siblings (with minor symptoms observed in the remaining two) and in both twins. Supplementary assessments suggested that all individuals also experienced some degree of difficulty with facial identity perception, but that object recognition was preserved. These findings bolster the evidence supporting the heritability of DP and suggest that it can be a specific impairment in some cases.

Using representational similarity analysis to reveal category and process specificity in visual object recognition

Cross-condition comparisons on neurodevelopmental conditions are central in neurodiversity research. In the realm of visual perception, the performance of participants with different category-specific disorders such as developmental prosopagnosia (problems with faces) and dyslexia (problems with words) have contributed to understanding of perceptual processes involved in word and face recognition. Alterations in face and word recognition are present in several neurodiverse populations, and improved knowledge about their relationship may increase our understanding of this variability of impairment. The present study investigates organizing principles of visual object processing and their implications for developmental disorders of recognition. Some accounts suggest that distinct mechanisms are responsible for recognizing objects of different categories, while others propose that categories share or even compete for cortical resources. We took an individual differences approach to estimate the relationship between abilities in recognition. Neurotypical participants (N = 97 after outlier exclusion) performed a match-to-sample task with faces, houses, and pseudowords. Either individual features or feature configurations were manipulated. To estimate the separability of visual recognition mechanisms, we used representational similarity analysis (RSA) where correlational matrices for accuracy were compared to predicted data patterns. Recognition abilities separated into face recognition on one hand and house/pseudoword recognition on the other, indicating that face recognition may rely on relatively selective mechanisms in neurotypicals. We also found evidence for a general visual object recognition mechanism, while some combinations of category (faces, houses, words) and processing type (featural, configural) likely rely on additional mechanisms. Developmental conditions may therefore reflect combinations of impaired and intact aspects of specific and general visual object recognition mechanisms, where featural and configural processes for one object category separate from the featural or configural processing of another. More generally, RSA is a promising approach for advancing understanding of neurodiversity, including shared aspects and distinctions between neurodevelopmental conditions of visual recognition.

Deep learning models challenge the prevailing assumption that face-like effects for objects of expertise support domain-general mechanisms

The question of whether task performance is best achieved by domain-specific, or domain-general processing mechanisms is fundemental for both artificial and biological systems. This question has generated a fierce debate in the study of expert object recognition. Because humans are experts in face recognition, face-like neural and cognitive effects for objects of expertise were considered support for domain-general mechanisms. However, effects of domain, experience and level of categorization, are confounded in human studies, which may lead to erroneous inferences. To overcome these limitations, we trained deep learning algorithms on different domains (objects, faces, birds) and levels of categorization (basic, sub-ordinate, individual), matched for amount of experience. Like humans, the models generated a larger inversion effect for faces than for objects. Importantly, a face-like inversion effect was found for individual-based categorization of non-faces (birds) but only in a network specialized for that domain. Thus, contrary to prevalent assumptions, face-like effects for objects of expertise do not support domain-general mechanisms but may originate from domain-specific mechanisms. More generally, we show how deep learning algorithms can be used to dissociate factors that are inherently confounded in the natural environment of biological organisms to test hypotheses about their isolated contributions to cognition and behaviour.

Looking at the upper facial half enlarges the range of holistic face processing

Previous studies suggested that upper and lower facial halves might be involved in the human holistic face processing differently. In this study, we replicated and extended the finding above. In Experiment 1, we used the standard composite-face task to measure holistic face processing when participants made judgements on the upper and lower facial halves separately. Results showed that the composite-face effect was stronger for the upper facial half compared to the lower half. In Experiment 2, we investigated how facial information was integrated when participants focused on different features, using the perceptual field paradigm. Results showed that: (1) more "peripheral faces" were chosen when participants fixated at the eyes than when they fixated at the mouth; (2) less "peripheral faces" were chosen for inverted faces regardless of the fixated features. Findings from both experiments together indicate that more peripheral facial information were integrated when participants focused on the upper facial half, highlighting the significance of focusing on the upper facial half in face processing.

PRA-Net: Part-and-Relation Attention Network for depression recognition from facial expression

Artificial intelligence methods are widely applied to depression recognition and provide an objective solution. Many effective automated methods for detecting depression use facial expressions, which are strong indicators to reflect psychiatric disorders. However, these methods suffer from insufficient representations of depression. To this end, we propose a novel Part-and-Relation Attention Network (PRA-Net), which can enhance depression representations by accurately focusing on features that are highly correlated with depression. Specifically, we first perform partition on the feature map instead of the original image, in order to obtain part features rich in semantic information. Afterwards, self-attention is used to calculate the weight of each part feature. Following, the relationship between the part feature and the global content representation is explored by relation attention to refine the weight. Finally, all features are aggregated into a more compact and depression-informative representation via both weights for depression score prediction. Extensive experiments demonstrate the superiority of our method. Compared to other end-to-end methods, our method achieves state-of-the-art performance on AVEC2013 and AVEC2014.

Contributions of low- and high-level contextual mechanisms to human face perception

Contextual modulations at primary stages of visual processing depend on the strength of local input. Contextual modulations at high-level stages of (face) processing show a similar dependence to local input strength. Namely, the discriminability of a facial feature determines the amount of influence of the face context on that feature. How high-level contextual modulations emerge from primary mechanisms is unclear due to the scarcity of empirical research systematically addressing the functional link between the two. We tested (62) young adults' ability to process local input independent of the context using contrast detection and (upright and inverted) morphed facial feature matching tasks. We first investigated contextual modulation magnitudes across tasks to address their shared variance. A second analysis focused on the profile of performance across contextual conditions. In upright eye matching and contrast detection tasks, contextual modulations only correlated at the level of their profile (averaged Fisher-Z transformed r = 1.18, BF10 > 100), but not magnitude (r = .15, BF10 = .61), suggesting the functional independence but similar working principles of the mechanisms involved. Both the profile (averaged Fisher-Z transformed r = .32, BF10 = 9.7) and magnitude (r = .28, BF10 = 4.58) of the contextual modulations correlated between inverted eye matching and contrast detection tasks. Our results suggest that non-face-specialized high-level contextual mechanisms (inverted faces) work in connection to primary contextual mechanisms, but that the engagement of face-specialized mechanisms for upright faces obscures this connection. Such combined study of low- and high-level contextual modulations sheds new light on the functional relationship between different levels of the visual processing hierarchy, and thus on its functional organization.

Contrasting domain-general and domain-specific accounts in cognitive neuropsychology: An outline of a new approach with developmental prosopagnosia as a case

The backbone of cognitive neuropsychology is the observation of (double) dissociations in performance between patients, suggesting some degree of independence between cognitive processes (domain specificity). In comparison, observations of associations between disorders/deficits have been deemed less evidential in neuropsychological theorizing about cognitive architecture. The reason is that associations can reflect damage to independent cognitive processes that happen to be mediated by structures commonly affected by the same brain disorder rather than damage to a shared (domain-general) mechanism. Here we demonstrate that it is in principle possible to discriminate between these alternatives by means of a procedure involving large unbiased samples. We exemplify the procedure in the context of developmental prosopagnosia (DP), but the procedure is in principle applicable to all neuropsychological deficits/disorders. A simulation of the procedure on a dataset yields estimates of dissociations/associations that are well in line with existing DP-studies, and also suggests that seemingly selective disorders can reflect damage to both domain-general and domain-specific cognitive processes. However, the simulation also highlights some limitations that should be considered if the procedure is to be applied prospectively. The main advantage of the procedure is that allows for examination of both associations and dissociations in the same sample. Hence, it may help even the balance in the use of associations and dissociations as grounds for neuropsychological theorizing.

Facial Expression Recognition Using Dual Path Feature Fusion and Stacked Attention

Facial Expression Recognition (FER) can achieve an understanding of the emotional changes of a specific target group. The relatively small dataset related to facial expression recognition and the lack of a high accuracy of expression recognition are both a challenge for researchers. In recent years, with the rapid development of computer technology, especially the great progress of deep learning, more and more convolutional neural networks have been developed for FER research. Most of the convolutional neural performances are not good enough when dealing with the problems of overfitting from too-small datasets and noise, due to expression-independent intra-class differences. In this paper, we propose a Dual Path Stacked Attention Network (DPSAN) to better cope with the above challenges. Firstly, the features of key regions in faces are extracted using segmentation, and irrelevant regions are ignored, which effectively suppresses intra-class differences. Secondly, by providing the global image and segmented local image regions as training data for the integrated dual path model, the overfitting problem of the deep network due to a lack of data can be effectively mitigated. Finally, this paper also designs a stacked attention module to weight the fused feature maps according to the importance of each part for expression recognition. For the cropping scheme, this paper chooses to adopt a cropping method based on the fixed four regions of the face image, to segment out the key image regions and to ignore the irrelevant regions, so as to improve the efficiency of the algorithm computation. The experimental results on the public datasets, CK+ and FERPLUS, demonstrate the effectiveness of DPSAN, and its accuracy reaches the level of current state-of-the-art methods on both CK+ and FERPLUS, with 93.2% and 87.63% accuracy on the CK+ dataset and FERPLUS dataset, respectively.

Face specific inversion effects provide evidence for two subtypes of developmental prosopagnosia

Many studies have attempted to identify the perceptual underpinnings of developmental prosopagnosia (DP). The majority have focused on whether holistic and configural processing mechanisms are impaired in DP. However, previous work suggests that there is substantial heterogeneity in holistic and configural processing within the DP population; further, there is disagreement as to whether any deficits are face-specific or reflect a broader perceptual deficit. This study used a data-driven approach to examine whether there are systematic patterns of variability in DP that reflect different underpinning perceptual deficits. A group of individuals with DP (N = 37) completed a cognitive battery measuring holistic/configural and featural processing in faces and non-face objects. A two-stage cluster analysis on data from the Cambridge Face Perception Test identified two subgroups of DPs. Across several tasks, the first subgroup (N = 21) showed typical patterns of holistic/configural processing (measured via inversion effects); the second (N = 16) was characterised by reduced or abolished inversion effects compared to age-matched control participants (N = 91). The subgroups did not differ on tasks measuring upright face matching, object matching, non-face holistic processing, or composite effects. These findings indicate two separable pathways to face recognition impairment, one characterised by impaired configural processing and the other potentially by impaired featural processing. Comparisons to control participants provide some preliminary evidence that the deficit in featural processing may extend to some non-face stimuli. Our results demonstrate the utility of examining both the variability between and consistency across individuals with DP as a means of illuminating our understanding of face recognition in typical and atypical populations.

FERGCN: facial expression recognition based on graph convolution network

Due to the problems of occlusion, pose change, illumination change, and image blur in the wild facial expression dataset, it is a challenging computer vision problem to recognize facial expressions in a complex environment. To solve this problem, this paper proposes a deep neural network called facial expression recognition based on graph convolution network (FERGCN), which can effectively extract expression information from the face in a complex environment. The proposed FERGCN includes three essential parts. First, a feature extraction module is designed to obtain the global feature vectors from convolutional neural networks branch with triplet attention and the local feature vectors from key point-guided attention branch. Then, the proposed graph convolutional network uses the correlation between global features and local features to enhance the expression information of the non-occluded part, based on the topology graph of key points. Furthermore, the graph-matching module uses the similarity between images to enhance the network's ability to distinguish different expressions. Results on public datasets show that our FERGCN can effectively recognize facial expressions in real environment, with RAF-DB of 88.23%, SFEW of 56.15% and AffectNet of 62.03%.

The emotional adaptation aftereffect discriminates between individuals with high and low levels of depressive symptoms

The adaptation aftereffect plays a critical role in human development and survival. Existing studies have found that, compared with general individuals, individuals with learning disability, autism and dyslexia show a smaller amount of non-affective-based cognitive adaptation aftereffect. Nevertheless, it is unclear whether individuals with depression or depression tendency show similar phenomenon in the adaptation aftereffect, and whether such depression tendency occurs in the non-affective-based cognitive or emotional adaptation aftereffect. To address this question, the present study conducted two experiments. Experiments 1A and 1B used the emotional facial expression adaptation paradigm to examine whether Chinese participants showed the emotional adaptation aftereffect and whether the emotional adaptation aftereffect was influenced by physical features of faces, respectively. Experiment 2 recruited two groups of participants, with high and low depression, respectively, to examine whether they showed differences in the emotional or cognitive adaptation aftereffect. Results showed that Chinese participants showed the typical emotional adaptation aftereffect, which was not influenced by physical features of faces. More importantly, compared to the low-depression group, the high-depression group showed a smaller emotional adaptation aftereffect, but the two groups showed a similar cognitive adaptation aftereffect. These results suggest that level of depressive symptoms is associated with the emotional adaptation aftereffect.

Learning Deep Global Multi-Scale and Local Attention Features for Facial Expression Recognition in the Wild

Facial expression recognition (FER) in the wild received broad concerns in which occlusion and pose variation are two key issues. This paper proposed a global multi-scale and local attention network (MA-Net) for FER in the wild. Specifically, the proposed network consists of three main components: a feature pre-extractor, a multi-scale module, and a local attention module. The feature pre-extractor is utilized to pre-extract middle-level features, the multi-scale module to fuse features with different receptive fields, which reduces the susceptibility of deeper convolution towards occlusion and variant pose, while the local attention module can guide the network to focus on local salient features, which releases the interference of occlusion and non-frontal pose problems on FER in the wild. Extensive experiments demonstrate that the proposed MA-Net achieves the state-of-the-art results on several in-the-wild FER benchmarks: CAER-S, AffectNet-7, AffectNet-8, RAFDB, and SFEW with accuracies of 88.42%, 64.53%, 60.29%, 88.40%, and 59.40% respectively. The codes and training logs are publicly available at https://github.com/zengqunzhao/MA-Net.

Developmental prosopagnosics and super recognizers rely on the same facial features used by individuals with normal face recognition abilities for face identification

Face recognition depends on the ability of the face processing system to extract facial features that define the identity of a face. In a recent study we discovered that altering a subset of facial features changed the identity of the face, indicating that they are critical for face identification. Changing another set of features did not change the identity of a face, indicating that they are not critical for face identification. In the current study, we assessed whether developmental prosopagnosics (DPs) and super recognizers (SRs) also rely more heavily on these critical features than non-critical features for face identification. To that end, we presented to DPs and SRs faces in which either the critical or the non-critical features were manipulated. In Study 1, we presented SRs with a famous face recognition task. We found that overall SRs recognized famous faces that differ in either critical or non-critical features better than controls. Similar to controls, changes in critical features had a larger effect on SRs' face recognition than changes in non-critical features. In Study 2, we presented an identity matching task to DPs and SRs. Similar to controls, DPs and SRs perceived faces that differed in critical features as more different than faces that differed in non-critical features. Taken together, our results indicate that SRs and DPs use the same critical features for face identification as normal individuals. These findings emphasize the fundamental role of this subset of features for face identification.

The effect of facial features on facial anthropomorphic trustworthiness in social robots

As the nature of human-robot relationships have become increasingly bound to shift from supervisor-machine to friend-companion, people have exhibited an increasing interest in making social judgments toward such anthropomorphic objects, such as trustworthiness. However, the facial features of social robots and their potential effect on anthropomorphic trustworthiness are seldom analyzed and discussed comprehensively. This study examined whether the trustworthiness perception toward a social robot shared similarity with baby schema features on the human face. It also explored the effects of different combinations of baby schema facial features, especially the positions and sizes of the eyes and mouth, on facial anthropomorphic trustworthiness. A 5-way mixed experiment (N = 270) was conducted accordingly. The results indicated that people would experience a high level of facial anthropomorphic trustworthiness toward robots with baby schema features (i.e., large eyes, with medium vertical and horizontal positions of the eyes and mouth). This paper contributes to the literature on facial anthropomorphic trustworthiness in human-robot interaction and provides suggestions for social robot design.

FFA and OFA Encode Distinct Types of Face Identity Information

Faces of different people elicit distinct fMRI patterns in several face-selective regions of the human brain. Here we used representational similarity analysis to investigate what type of identity-distinguishing information is encoded in three face-selective regions: fusiform face area (FFA), occipital face area (OFA), and posterior superior temporal sulcus (pSTS). In a sample of 30 human participants (22 females, 8 males), we used fMRI to measure brain activity patterns elicited by naturalistic videos of famous face identities, and compared their representational distances in each region with models of the differences between identities. We built diverse candidate models, ranging from low-level image-computable properties (pixel-wise, GIST, and Gabor-Jet dissimilarities), through higher-level image-computable descriptions (OpenFace deep neural network, trained to cluster faces by identity), to complex human-rated properties (perceived similarity, social traits, and gender). We found marked differences in the information represented by the FFA and OFA. Dissimilarities between face identities in FFA were accounted for by differences in perceived similarity, Social Traits, Gender, and by the OpenFace network. In contrast, representational distances in OFA were mainly driven by differences in low-level image-based properties (pixel-wise and Gabor-Jet dissimilarities). Our results suggest that, although FFA and OFA can both discriminate between identities, the FFA representation is further removed from the image, encoding higher-level perceptual and social face information.SIGNIFICANCE STATEMENT Recent studies using fMRI have shown that several face-responsive brain regions can distinguish between different face identities. It is however unclear whether these different face-responsive regions distinguish between identities in similar or different ways. We used representational similarity analysis to investigate the computations within three brain regions in response to naturalistically varying videos of face identities. Our results revealed that two regions, the fusiform face area and the occipital face area, encode distinct identity information about faces. Although identity can be decoded from both regions, identity representations in fusiform face area primarily contained information about social traits, gender, and high-level visual features, whereas occipital face area primarily represented lower-level image features.

Birthweight predicts individual differences in adult face recognition ability

It has long been known that premature birth and/or low birthweight can lead to general difficulties in cognitive and emotional functioning throughout childhood. However, the influence of these factors on more specific processes has seldom been addressed, despite their potential to account for wide individual differences in performance that often appear innate. Here, we examined the influence of gestation and birthweight on adults' face perception and face memory skills. Performance on both sub-processes was predicted by birthweight and birthweight-for-gestation, but not gestation alone. Evidence was also found for the domain-specificity of these effects: No perinatal measure correlated with performance on object perception or memory tasks, but they were related to the size of the face inversion effect on the perceptual test. This evidence indicates a novel, very early influence on individual differences in face recognition ability, which persists into adulthood, influences face-processing strategy itself, and may be domain-specific.

Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition

Occlusion and pose variations, which can change facial appearance significantly, are two major obstacles for automatic Facial Expression Recognition (FER). Though automatic FER has made substantial progresses in the past few decades, occlusion-robust and pose-invariant issues of FER have received relatively less attention, especially in real-world scenarios. This paper addresses the real-world pose and occlusion robust FER problem in the following aspects. First, to stimulate the research of FER under real-world occlusions and variant poses, we annotate several in-the-wild FER datasets with pose and occlusion attributes for the community. Second, we propose a novel Region Attention Network (RAN), to adaptively capture the importance of facial regions for occlusion and pose variant FER. The RAN aggregates and embeds varied number of region features produced by a backbone convolutional neural network into a compact fixed-length representation. Last, inspired by the fact that facial expressions are mainly defined by facial action units, we propose a region biased loss to encourage high attention weights for the most important regions. We validate our RAN and region biased loss on both our built test datasets and four popular datasets: FERPlus, AffectNet, RAF-DB, and SFEW. Extensive experiments show that our RAN and region biased loss largely improve the performance of FER with occlusion and variant pose. Our method also achieves state-of-the-art results on FERPlus, AffectNet, RAF-DB, and SFEW. Code and the collected test data will be publicly available.

The domain-specificity of face matching impairments in 40 cases of developmental prosopagnosia

A prevailing debate in the psychological literature concerns the domain-specificity of the face recognition system, where evidence from typical and neurological participants has been interpreted as evidence that faces are "special". Although several studies have investigated the same question in cases of developmental prosopagnosia, the vast majority of this evidence has recently been discounted due to methodological concerns. This leaves an uncomfortable void in the literature, restricting our understanding of the typical and atypical development of the face recognition system. The current study addressed this issue in 40 individuals with developmental prosopagnosia, completing a sequential same/different face and biological (hands) and non-biological (houses) object matching task, with upright and inverted conditions. Findings support domain-specific accounts of face-processing for both hands and houses: while significant correlations emerged between all the object categories, no condition correlated with performance in the upright faces condition. Further, a categorical analysis demonstrated that, when face matching was impaired, object matching skills were classically dissociated in six out of 15 individuals (four for both categories). These findings provide evidence about domain-specificity in developmental disorders of face recognition, and present a theoretically-driven means of partitioning developmental prosopagnosia.

Tests of whole upright face processing in prosopagnosia: A literature review

Prosopagnosia refers to an acquired or developmental deficit in face recognition. This neuropsychological impairment has received increasing attention over the last decade, in particular because of an increased scientific interest in developmental prosopagnosia. Studies investigating prosopagnosia have used a variety of different clinical and experimental tests to assess face processing abilities. With such a large variety of assessment methods available, test selection can be challenging. Some previous works have aimed to provide an overview of tests used to diagnose prosopagnosia. However, no overview that is based on a structured review of the literature is available. We review the literature to identify tests that have been used to assess the processing of whole upright faces in acquired and developmental prosopagnosia over the last five years (2013-2017). We not only review tests that have been used for diagnostic purposes, but also tests that have been used for experimental purposes. Tests are categorised according to i) their experimental designs and, ii) the stage of face processing that they assess. On this basis, we discuss considerations regarding test designs for future studies. A visual illustration providing a structured overview of paradigms available for testing the processing of whole upright faces is provided. This visual illustration can be used to inform test selection when designing a study and to apply a structured approach to interpreting findings from the literature. The different approaches to assessment of face processing in prosopagnosia have been necessary and fruitful in generating data and hypotheses about the cause of face processing deficits. However, impairments at different levels of face processing have often been interpreted as reflecting a deficit in the recognition stage of face processing. Based on the data now available on prosopagnosia, we advocate for a more structured approach to assessment, which may facilitate a better understanding of the key deficits in prosopagnosia and of the level(s) of face processing that are impaired.

The Fusiform and Occipital Face Areas Can Process a Nonface Category Equivalently to Faces

The fusiform and occipital face areas (FFA and OFA) are functionally defined brain regions in human ventral occipitotemporal cortex associated with face perception. There is an ongoing debate, however, whether these regions are face-specific or whether they also facilitate the perception of nonface object categories. Here, we present evidence that, under certain conditions, bilateral FFA and OFA respond to a nonface category equivalently to faces. In two fMRI sessions, participants performed same–different judgments on two object categories (faces and chairs). In one session, participants differentiated between distinct exemplars of each category, and in the other session, participants differentiated between exemplars that differed only in the shape or spatial configuration of their features (featural/configural differences). During the latter session, the within-category similarity was comparable for both object categories. When differentiating between distinct exemplars of each category, bilateral FFA and OFA responded more strongly to faces than to chairs. In contrast, during featural/configural difference judgments, bilateral FFA and OFA responded equivalently to both object categories. Importantly, during featural/configural difference judgments, the magnitude of activity within FFA and OFA evoked by the chair task predicted the participants' behavioral performance. In contrast, when participants differentiated between distinct chair exemplars, activity within these face regions did not predict the behavioral performance of the chair task. We conclude that, when the within-category similarity of a face and a nonface category is comparable and when the same cognitive strategies used to process a face are applied to a nonface category, the FFA and OFA respond equivalently to that nonface category and faces.

Spatial Mechanisms within the Dorsal Visual Pathway Contribute to the Configural Processing of Faces

Human face recognition is often attributed to configural processing; namely, processing the spatial relationships among the features of a face. If configural processing depends on fine-grained spatial information, do visuospatial mechanisms within the dorsal visual pathway contribute to this process? We explored this question in human adults using functional magnetic resonance imaging and transcranial magnetic stimulation (TMS) in a same-different face detection task. Within localized, spatial-processing regions of the posterior parietal cortex, configural face differences led to significantly stronger activation compared to featural face differences, and the magnitude of this activation correlated with behavioral performance. In addition, detection of configural relative to featural face differences led to significantly stronger functional connectivity between the right FFA and the spatial processing regions of the dorsal stream, whereas detection of featural relative to configural face differences led to stronger functional connectivity between the right FFA and left FFA. Critically, TMS centered on these parietal regions impaired performance on configural but not featural face difference detections. We conclude that spatial mechanisms within the dorsal visual pathway contribute to the configural processing of facial features and, more broadly, that the dorsal stream may contribute to the veridical perception of faces.

The neural network for face recognition: Insights from an fMRI study on developmental prosopagnosia

Face recognition is supported by collaborative work of multiple face-responsive regions in the brain. Based on findings from individuals with normal face recognition ability, a neural model has been proposed with the occipital face area (OFA), fusiform face area (FFA), and face-selective posterior superior temporal sulcus (pSTS) as the core face network (CFN) and the rest of the face-responsive regions as the extended face network (EFN). However, little is known about how these regions work collaboratively for face recognition in our daily life. Here we focused on individuals suffering developmental prosopagnosia (DP), a neurodevelopmental disorder specifically impairing face recognition, to shed light on the infrastructure of the neural model of face recognition. Specifically, we used a variant of global brain connectivity method to comprehensively explore resting-state functional connectivity (FC) among face-responsive regions in a large sample of DPs (N = 64). We found that both the FCs within the CFN and those between the CFN and EFN were largely reduced in DP. Importantly, the right OFA and FFA served as the dysconnectivity hubs within the CFN, i.e., FCs concerning these two regions within the CFN were largely disrupted. In addition, DPs' right FFA also showed reduced FCs with the EFN. Moreover, these disrupted FCs were related to DP's behavioral deficit in face recognition, with the FCs from the FFA to the anterior temporal lobe (ATL) and pSTS the most predictive. Based on these findings, we proposed a revised neural model of face recognition demonstrating the relatedness of interactions among face-responsive regions to face recognition.

Congenital prosopagnosia without object agnosia? A literature review

A longstanding controversy concerns the functional organization of high-level vision, and the extent to which the recognition of different classes of visual stimuli engages a single system or multiple independent systems. We examine this in the context of congenital prosopagnosia (CP), a neurodevelopmental disorder in which individuals, without a history of brain damage, are impaired at face recognition. This paper reviews all CP cases from 1976 to 2016, and explores the evidence for the association or dissociation of face and object recognition. Of the 238 CP cases with data permitting a satisfactory evaluation, 80.3% evinced an association between impaired face and object recognition whereas 19.7% evinced a dissociation. We evaluate the strength of the evidence and correlate the face and object recognition behaviour. We consider the implications for theories of functional organization of the visual system, and offer suggestions for further adjudication of the relationship between face and object recognition.

Individual recognition and the 'face inversion effect' in medaka fish (Oryzias latipes)

Individual recognition (IR) is essential for maintaining various social interactions in a group, and face recognition is one of the most specialised cognitive abilities in IR. We used both a mating preference system and an electric shock conditioning experiment to test IR ability in medaka, and found that signals near the face are important. Medaka required more time to discriminate vertically inverted faces, but not horizontally shifted faces or inverted non-face objects. The ability may be comparable to the classic 'face inversion effect' in humans and some other mammals. Extra patterns added to the face also did not influence the IR. These findings suggest the possibility that the process of face recognition may differ from that used for other objects. The complex form of recognition may promote specific processing adaptations, although the mechanisms and neurological bases might differ in mammals and medaka. The ability to recognise other individuals is important for shaping animal societies.

The DalHouses: 100 new photographs of houses with ratings of typicality, familiarity, and degree of similarity to faces

Houses have often been used as comparison stimuli in face-processing studies because of the many attributes they share with faces (e.g., distinct members of a basic category, consistent internal features, mono-orientation, and relative familiarity). Despite this, no large, well-controlled databases of photographs of houses that have been developed for research use currently exist. To address this gap, we photographed 100 houses and carefully edited these images. We then asked 41 undergraduate students (18 to 31 years of age) to rate each house on three dimensions: typicality, likeability, and face-likeness. The ratings had a high degree of face validity, and analyses revealed a significant positive correlation between typicality and likeability. We anticipate that this stimulus set (i.e., the DalHouses) and the associated ratings will prove useful to face-processing researchers by minimizing the effort required to acquire stimuli and allowing for easier replication and extension of studies. The photographs of all 100 houses and their ratings data can be obtained at http://dx.doi.org/10.6084/m9.figshare.1279430.

A new approach to the diagnosis of deficits in processing faces: Potential application in autism research

Deficits in social communication are one of the behavioral signatures of autism spectrum disorder (ASD). Because faces are arguably the most important social stimuli that we encounter in everyday life, investigating the ability of individuals with ASD to process faces is thought to be important for understanding the nature of ASD. However, although a considerable body of evidence suggests that ASD individuals show specific impairments in face processing, a significant number of studies argue otherwise. Through a literature review, we found that this controversy is largely attributable to the different face tests used across different studies. Therefore, a more reliable and valid face test is needed. To this end, we performed a meta-analysis on data gleaned from a variety of face tests conducted on individuals with developmental prosopagnosia (DP) who suffer a selective deficit in face processing. Based on this meta-analysis, we selected an old/new face recognition test that relies on face memory as a standard diagnostic test for measuring specific face processing deficits. This test not only reliably reflects DP individuals' subjective experiences with faces in their daily lives, but also effectively differentiates deficits in face processing from deficits caused by other general problems. In addition, DP individuals' performance in this test predicts their performance in a variety of face tests that examine specific components of face processing (e.g., holistic processing of faces). Finally, this test can be easily administrated and is not overly sensitive to prior knowledge. In summary, this test can be used to evaluate face-processing ability, and it helped to resolve the controversy whether individuals with ASD exhibit face-processing deficits.

Altered spontaneous neural activity in the occipital face area reflects behavioral deficits in developmental prosopagnosia

Individuals with developmental prosopagnosia (DP) exhibit severe difficulties in recognizing faces and to a lesser extent, also exhibit difficulties in recognizing non-face objects. We used fMRI to investigate whether these behavioral deficits could be accounted for by altered spontaneous neural activity. Two aspects of spontaneous neural activity were measured: the intensity of neural activity in a voxel indexed by the fractional amplitude of spontaneous low-frequency fluctuations (fALFF), and the connectivity of a voxel to neighboring voxels indexed by regional homogeneity (ReHo). Compared with normal adults, both the fALFF and ReHo values within the right occipital face area (rOFA) were significantly reduced in DP subjects. Follow-up studies on the normal adults revealed that these two measures indicated further functional division of labor within the rOFA. The fALFF in the rOFA was positively correlated with behavioral performance in recognition of non-face objects, whereas ReHo in the rOFA was positively correlated with processing of faces. When considered together, the altered fALFF and ReHo within the same region (rOFA) may account for the comorbid deficits in both face and object recognition in DPs, whereas the functional division of labor in these two measures helps to explain the relative independency of deficits in face recognition and object recognition in DP.

The problem of being bad at faces

Developmental prosopagnosia has received increased attention in recent years, but as yet has no confirmed genetic or structural markers. It is not certain whether this condition reflects simply the low-end of the spectrum of normal face recognition, an 'under-development', or a pathologic failure to develop such mechanisms, a 'mal-development'. This difference in views creates challenges for the diagnosis of developmental prosopagnosia by behavioural criteria alone, which also vary substantially between studies, with secondary effects on issues such as determining its prevalence. After review of the literature and the problems inherent to diagnoses based solely on behavioural data, we propose as a starting discussion point a set of two primary and four secondary criteria for the diagnosis of developmental prosopagnosia.

Seeing the eyes in acquired prosopagnosia

Case reports have suggested that perception of the eye region may be impaired more than that of other facial regions in acquired prosopagnosia. However, it is unclear how frequently this occurs, whether such impairments are specific to a certain anatomic subtype of prosopagnosia, and whether these impairments are related to changes in the scanning of faces. We studied a large cohort of 11 subjects with this rare disorder, who had a variety of occipitotemporal or anterior temporal lesions, both unilateral and bilateral. Lesions were characterized by functional and structural imaging. Subjects performed a perceptual discrimination test in which they had to discriminate changes in feature position, shape, or external contour. Test conditions were manipulated to stress focused or divided attention across the whole face. In a second experiment we recorded eye movements while subjects performed a face memory task. We found that greater impairment for eye processing was more typical of subjects with occipitotemporal lesions than those with anterior temporal lesions. This eye selectivity was evident for both eye position and shape, with no evidence of an upper/lower difference for external contour. A greater impairment for eye processing was more apparent under attentionally more demanding conditions. Despite these perceptual deficits, most subjects showed a normal tendency to scan the eyes more than the mouth. We conclude that occipitotemporal lesions are associated with a partially selective processing loss for eye information and that this deficit may be linked to loss of the right fusiform face area, which has been shown to have activity patterns that emphasize the eye region.

Perceptual and memorial contributions to developmental prosopagnosia

Developmental prosopagnosia (DP) is commonly associated with the failure to properly perceive individuating facial properties, notably those conveying configural or holistic content. While this may indicate that the primary impairment is perceptual, it is conceivable that some cases of DP are instead caused by a memory impairment, with any perceptual complaint merely allied rather than causal. To investigate this possibility, we administered a battery of face perception tasks to 11 individuals who reported that their face recognition difficulties disrupt daily activity and who also performed poorly on two formal tests of face recognition. Group statistics identified, relative to age- and gender-matched controls, difficulties in apprehending global-local relations and the holistic properties of faces, and in matching across viewpoints, but these were mild in nature and were not consistently evident at the level of individual participants. Six of the 11 individuals failed to show any evidence of perceptual impairment. In the remaining five individuals, no single perceptual deficit, or combination of deficits, was necessary or sufficient for poor recognition performance. These data suggest that some cases of DP are better explained by a memorial rather than perceptual deficit, and highlight the relevance of the apperceptive/associative distinction more commonly applied to the allied syndrome of acquired prosopagnosia.

Face Perception and Test Reliabilities in Congenital Prosopagnosia in Seven Tests

Congenital prosopagnosia, the innate impairment in recognizing faces, is a very heterogeneous disorder with different phenotypical manifestations. To investigate the nature of prosopagnosia in more detail, we tested 16 prosopagnosics and 21 controls with an extended test battery addressing various aspects of face recognition. Our results show that prosopagnosics exhibited significant impairments in several face recognition tasks: impaired holistic processing (they were tested amongst others with the Cambridge Face Memory Test (CFMT)) as well as reduced processing of configural information of faces. This test battery also revealed some new findings. While controls recognized moving faces better than static faces, prosopagnosics did not exhibit this effect. Furthermore, prosopagnosics had significantly impaired gender recognition—which is shown on a groupwise level for the first time in our study. There was no difference between groups in the automatic extraction of face identity information or in object recognition as tested with the Cambridge Car Memory Test. In addition, a methodological analysis of the tests revealed reduced reliability for holistic face processing tests in prosopagnosics. To our knowledge, this is the first study to show that prosopagnosics showed a significantly reduced reliability coefficient (Cronbach’s alpha) in the CFMT compared to the controls. We suggest that compensatory strategies employed by the prosopagnosics might be the cause for the vast variety of response patterns revealed by the reduced test reliability. This finding raises the question whether classical face tests measure the same perceptual processes in controls and prosopagnosics.

Guidelines for studying developmental prosopagnosia in adults and children

Developmental prosopagnosia (DP) is a neurodevelopmental condition characterized by severe face identity recognition problems that results from a failure to develop the mechanisms necessary for adequate face processing (Duchaine BC, Nakayama K. Developmental prosopagnosia: a window to content-specific face processing. Curr Opin Neurobiol 2006, 16:166-173.). It occurs in children and adults with normal visual acuity, and without intellectual impairments or known brain injuries. Given the importance of face recognition in daily life, and the detrimental effects of impaired face recognition, DP is an important area of study. Yet conventions for classifying individuals as DP for research purposes are poorly defined. In this focus paper, we discuss: (1) criteria for an operational definition of DP; 2) tests of face recognition and conventions for classifying individuals as DP; and 3) important considerations regarding common associations and dissociations, and cognitive heterogeneity in DP. We also highlight issues unique to studying DP in children, a relatively new endeavor that is proving to be an important complement to the work with adults. Ultimately, we hope to identify challenges researchers face when studying DP, and offer guidelines for others to consider when embarking on their own research pursuits on the topic. For further resources related to this article, please visit the WIREs website.

Processing deficits for familiar and novel faces in patients with left posterior fusiform lesions

Pure alexia (PA) arises from damage to the left posterior fusiform gyrus (pFG) and the striking reading disorder that defines this condition has meant that such patients are often cited as evidence for the specialisation of this region to processing of written words. There is, however, an alternative view that suggests this region is devoted to processing of high acuity foveal input, which is particularly salient for complex visual stimuli like letter strings. Previous reports have highlighted disrupted processing of non-linguistic visual stimuli after damage to the left pFG, both for familiar and unfamiliar objects and also for novel faces. This study explored the nature of face processing deficits in patients with left pFG damage. Identification of famous faces was found to be compromised in both expressive and receptive tasks. Discrimination of novel faces was also impaired, particularly for those that varied in terms of second-order spacing information, and this deficit was most apparent for the patients with the more severe reading deficits. Interestingly, discrimination of faces that varied in terms of feature identity was considerably better in these patients and it was performance in this condition that was related to the size of the length effects shown in reading. This finding complements functional imaging studies showing left pFG activation for faces varying only in spacing and frontal activation for faces varying only on features. These results suggest that the sequential part-based processing strategy that promotes the length effect in the reading of these patients also allows them to discriminate between faces on the basis of feature identity, but processing of second-order configural information is most compromised due to their left pFG lesion. This study supports a view in which the left pFG is specialised for processing of high acuity foveal visual information that supports processing of both words and faces.

The man who mistook his neuropsychologist for a popstar: when configural processing fails in acquired prosopagnosia

We report the case of an individual with acquired prosopagnosia who experiences extreme difficulties in recognizing familiar faces in everyday life despite excellent object recognition skills. Formal testing indicates that he is also severely impaired at remembering pre-experimentally unfamiliar faces and that he takes an extremely long time to identify famous faces and to match unfamiliar faces. Nevertheless, he performs as accurately and quickly as controls at identifying inverted familiar and unfamiliar faces and can recognize famous faces from their external features. He also performs as accurately as controls at recognizing famous faces when fracturing conceals the configural information in the face. He shows evidence of impaired global processing but normal local processing of Navon figures. This case appears to reflect the clearest example yet of an acquired prosopagnosic patient whose familiar face recognition deficit is caused by a severe configural processing deficit in the absence of any problems in featural processing. These preserved featural skills together with apparently intact visual imagery for faces allow him to identify a surprisingly large number of famous faces when unlimited time is available. The theoretical implications of this pattern of performance for understanding the nature of acquired prosopagnosia are discussed.

Neural decoding reveals impaired face configural processing in the right fusiform face area of individuals with developmental prosopagnosia

Most of human daily social interactions rely on the ability to successfully recognize faces. Yet ∼2% of the human population suffers from face blindness without any acquired brain damage [this is also known as developmental prosopagnosia (DP) or congenital prosopagnosia]). Despite the presence of severe behavioral face recognition deficits, surprisingly, a majority of DP individuals exhibit normal face selectivity in the right fusiform face area (FFA), a key brain region involved in face configural processing. This finding, together with evidence showing impairments downstream from the right FFA in DP individuals, has led some to argue that perhaps the right FFA is largely intact in DP individuals. Using fMRI multivoxel pattern analysis, here we report the discovery of a neural impairment in the right FFA of DP individuals that may play a critical role in mediating their face-processing deficits. In seven individuals with DP, we discovered that, despite the right FFA's preference for faces and it showing decoding for the different face parts, it exhibited impaired face configural decoding and did not contain distinct neural response patterns for the intact and the scrambled face configurations. This abnormality was not present throughout the ventral visual cortex, as normal neural decoding was found in an adjacent object-processing region. To our knowledge, this is the first direct neural evidence showing impaired face configural processing in the right FFA in individuals with DP. The discovery of this neural impairment provides a new clue to our understanding of the neural basis of DP.