Familiarity effects on categorization levels of faces and objects

Common Sequential Organization of Face Processing in the Human Brain and Convolutional Neural Networks

Face processing includes two crucial processing levels - face detection and face recognition. However, it remains unclear how human brains organize the two processing levels sequentially. While some studies found that faces are recognized as fast as they are detected, others have reported that faces are detected first, followed by recognition. We discriminated the two processing levels on a fine time scale by combining human intracranial EEG (two females, three males, and three subjects without reported sex information) and representation similarity analysis. Our results demonstrate that the human brain exhibits a "detection-first, recognition-later" pattern during face processing. In addition, we used convolutional neural networks to test the hypothesis that the sequential organization of the two face processing levels in the brain reflects computational optimization. Our findings showed that the networks trained on face recognition also exhibited the "detection-first, recognition-later" pattern. Moreover, this sequential organization mechanism developed gradually during the training of the networks and was observed only for correctly predicted images. These findings collectively support the computational account as to why the brain organizes them in this way.

Decoding face recognition abilities in the human brain

Why are some individuals better at recognizing faces? Uncovering the neural mechanisms supporting face recognition ability has proven elusive. To tackle this challenge, we used a multimodal data-driven approach combining neuroimaging, computational modeling, and behavioral tests. We recorded the high-density electroencephalographic brain activity of individuals with extraordinary face recognition abilities-super-recognizers-and typical recognizers in response to diverse visual stimuli. Using multivariate pattern analyses, we decoded face recognition abilities from 1 s of brain activity with up to 80% accuracy. To better understand the mechanisms subtending this decoding, we compared representations in the brains of our participants with those in artificial neural network models of vision and semantics, as well as with those involved in human judgments of shape and meaning similarity. Compared to typical recognizers, we found stronger associations between early brain representations of super-recognizers and midlevel representations of vision models as well as shape similarity judgments. Moreover, we found stronger associations between late brain representations of super-recognizers and representations of the artificial semantic model as well as meaning similarity judgments. Overall, these results indicate that important individual variations in brain processing, including neural computations extending beyond purely visual processes, support differences in face recognition abilities. They provide the first empirical evidence for an association between semantic computations and face recognition abilities. We believe that such multimodal data-driven approaches will likely play a critical role in further revealing the complex nature of idiosyncratic face recognition in the human brain.

Category systems for real-world scenes

Categorization performance is a popular metric of scene recognition and understanding in behavioral and computational research. However, categorical constructs and their labels can be somewhat arbitrary. Derived from exhaustive vocabularies of place names (e.g., Deng et al., 2009), or the judgements of small groups of researchers (e.g., Fei-Fei, Iyer, Koch, & Perona, 2007), these categories may not correspond with human-preferred taxonomies. Here, we propose clustering by increasing the rand index via coordinate ascent (CIRCA): an unsupervised, data-driven clustering method for deriving ground-truth scene categories. In Experiment 1, human participants organized 80 stereoscopic images of outdoor scenes from the Southampton-York Natural Scenes (SYNS) dataset (Adams et al., 2016) into discrete categories. In separate tasks, images were grouped according to i) semantic content, ii) three-dimensional spatial structure, or iii) two-dimensional image appearance. Participants provided text labels for each group. Using the CIRCA method, we determined the most representative category structure and then derived category labels for each task/dimension. In Experiment 2, we found that these categories generalized well to a larger set of SYNS images, and new observers. In Experiment 3, we tested the relationship between our category systems and the spatial envelope model (Oliva & Torralba, 2001). Finally, in Experiment 4, we validated CIRCA on a larger, independent dataset of same-different category judgements. The derived category systems outperformed the SUN taxonomy (Xiao, Hays, Ehinger, Oliva, & Torralba, 2010) and an alternative clustering method (Greene, 2019). In summary, we believe this novel categorization method can be applied to a wide range of datasets to derive optimal categorical groupings and labels from psychophysical judgements of stimulus similarity.

Am I the same person across my life span? An event-related brain potentials study of the temporal perspective in self-identity

While self-identity recognition has been largely explored, less is known on how self-identity changes as a function of time. The present work aims to explore the influence of the temporal perspective on self-identity by studying event-related brain potentials (ERP) associated with face processing. To this purpose, participants had to perform a recognition task in two blocks with different task demands: (i) identity recognition (self, close-friend, unknown), and (ii) life stage recognition (adulthood -current-, adolescence, and childhood). The results showed that the N170 component was sensitive to changes in the global face configuration when comparing adulthood with other life stages. The N250 was the earliest neural marker discriminating self from other identities and may be related to a preferential deployment of attentional resources to recognize own face. The P3 was a robust index of self-specificity, reflecting stimulus categorization and presumably adding an emotional value. The results of interest emerged for the subsequent late positive complex (LPC). The larger amplitude for the LPC to the self-face was probably associated with further personal significance. The LPC, therefore, was able to distinguish the continuity of the self over time (i.e., between current self and past selves). Likewise, this component also could discriminate, at each life stage, the self-identity from other identities (e.g., between past self and past close-friend). This would confirm a remarkable role of the LPC reflecting higher self-relevance processes. Taken together, the neural representation of oneself (i.e., "I am myself") seems to be stable and also updated across time.

The Mnemonic of Intuitive Ontology Violation is not the Distinctiveness Effect: Evidence from a Broad Age Spectrum of Persons in the uk and China during a Free-Recall Task

The typical formulation of Pascal Boyer’s counterintuitiveness theory asserts that concepts violating intuitive ontological-category structures are more memorable. However, Boyer’s (2001) original claim centred on the transmission advantages of counter-ontological representations that were cultural. Nevertheless, subsequent studies focused on the recall of novel counterintuitive representations, and an “alternative account” of the memorability of counterintuitive concepts has emerged resembling the distinctiveness effect. Yet, experimental evidence shows that familiar concepts have memorability advantages over novel ones. This investigation of these pan-cultural transmission biases used a large age-representative sample (13–86 years; N = 365) in the uk and China. Results were analysed by hlm, with familiarity, counterintuitiveness, and delay as 2-level fixed factors, and age as a covariate. No support was revealed for the typical formulation of the hypothesis — however, a significant age effect and interaction of familiarity × counterintuitiveness were found.

Repetition probability effects depend on prior experiences

The magnitude of repetition suppression (RS) in the Fusiform Face Area is influenced by the probability of repetitions of faces (Summerfield et al., 2008), implying that perceptual expectations affect repetition-related processes. Surprisingly, however, macaque single-cell (Kaliukhovich and Vogels, 2011) and human fMRI (Kovács et al., 2013) studies have failed to find repetition probability [P(rep)] modulations of RS with nonface stimuli in the occipitotemporal cortex, suggesting that the effect is face specific. One possible explanation of this category selectivity is that the extensive experience humans have with faces affects the neural mechanisms of RS specifically, creating P(rep) modulatory effects. To address this question, we used fMRI to test the P(rep) effects for another well trained stimulus category, upright letters of the roman alphabet as well as for unfamiliar false fonts. We observed significant RS for both stimulus sets in the Letter Form Area as well as in the caudodorsal part of the lateral occipital complex. Interestingly, the influence of P(rep) on RS was dependent on the stimulus: while we observed P(rep) modulations for the roman letters, no such effects were found for the unfamiliar false fonts in either area. Our findings suggest that P(rep) effects on RS are manifest for nonface stimuli as well, but that they depend on the experience of the subjects with the stimulus category. This shows, for the first time, that prior experience affects the influence of contextual predictive information on RS in the human occipitotemporal cortex.

Stimulus familiarity modulates functional connectivity of the perirhinal cortex and anterior hippocampus during visual discrimination of faces and objects

Recent research suggests that the medial temporal lobe (MTL) is involved in perception as well as in declarative memory. Amnesic patients with focal MTL lesions and semantic dementia patients showed perceptual deficits when discriminating faces and objects. Interestingly, these two patient groups showed different profiles of impairment for familiar and unfamiliar stimuli. For MTL amnesics, the use of familiar relative to unfamiliar stimuli improved discrimination performance. By contrast, patients with semantic dementia—a neurodegenerative condition associated with anterolateral temporal lobe damage—showed no such facilitation from familiar stimuli. Given that the two patient groups had highly overlapping patterns of damage to the perirhinal cortex, hippocampus, and temporal pole, the neuroanatomical substrates underlying their performance discrepancy were unclear. Here, we addressed this question with a multivariate reanalysis of the data presented by Barense et al. (2011), using functional connectivity to examine how stimulus familiarity affected the broader networks with which the perirhinal cortex, hippocampus, and temporal poles interact. In this study, healthy participants were scanned while they performed an odd-one-out perceptual task involving familiar and novel faces or objects. Seed-based analyses revealed that functional connectivity of the right perirhinal cortex and right anterior hippocampus was modulated by the degree of stimulus familiarity. For familiar relative to unfamiliar faces and objects, both right perirhinal cortex and right anterior hippocampus showed enhanced functional correlations with anterior/lateral temporal cortex, temporal pole, and medial/lateral parietal cortex. These findings suggest that in order to benefit from stimulus familiarity, it is necessary to engage not only the perirhinal cortex and hippocampus, but also a network of regions known to represent semantic information.

Spatial demonstratives and perceptual space: describing and remembering object location

Spatial demonstratives - terms including this and that - are among the most common words across all languages. Yet, there are considerable differences between languages in how demonstratives carve up space and the object characteristics they can refer to, challenging the idea that the mapping between spatial demonstratives and the vision and action systems is universal. In seven experiments we show direct parallels between spatial demonstrative usage in English and (non-linguistic) memory for object location, indicating close connections between the language of space and non-linguistic spatial representation. Spatial demonstrative choice in English and immediate memory for object location are affected by a range of parameters - distance, ownership, visibility and familiarity - that are lexicalized in the demonstrative systems of some other languages. The results support a common set of constraints on language used to talk about space and on (non-linguistic) spatial representation itself. Differences in demonstrative systems across languages may emerge from basic distinctions in the representation and memory for object location. In turn, these distinctions offer a building block from which non-spatial uses of demonstratives can develop.

A new standardized stimulus set for studying need-of-help recognition (NeoHelp)

This article presents the NeoHelp visual stimulus set created to facilitate investigation of need-of-help recognition with clinical and normative populations of different ages, including children. Need-of-help recognition is one aspect of socioemotional development and a necessary precondition for active helping. The NeoHelp consists of picture pairs showing everyday situations: The first item in a pair depicts a child needing help to achieve a goal; the second one shows the child achieving the goal. Pictures of birds in analogue situations are also included. These control stimuli enable implementation of a human-animal categorization task which serves to separate behavioral correlates specific to need-of-help recognition from general differentiation processes. It is a concern in experimental research to ensure that results do not relate to systematic perceptual differences when comparing responses to categories of different content. Therefore, we not only derived the NeoHelp-pictures within a pair from one another by altering as little as possible, but also assessed their perceptual similarity empirically. We show that NeoHelp-picture pairs are very similar regarding low-level perceptual properties across content categories. We obtained data from 60 children in a broad age range (4 to 13 years) for three different paradigms, in order to assess whether the intended categorization and differentiation could be observed reliably in a normative population. Our results demonstrate that children can differentiate the pictures' content regarding both need-of-help category as well as species as intended in spite of the high perceptual similarities. We provide standard response characteristics (hit rates and response times) that are useful for future selection of stimuli and comparison of results across studies. We show that task requirements coherently determine which aspects of the pictures influence response characteristics. Thus, we present NeoHelp, the first open-access standardized visual stimuli set for investigation of need-of-help recognition and invite researchers to use and extend it.

The fidelity of visual memory for faces and non-face objects

The fidelity of visual working memory was assessed for faces and non-face objects. In two experiments, four levels of memory load (1, 2, 3, or 4 items) were combined with four perceptual distances between probe and study items, with maximum item confusability occurring for the minimum memory load. Under these conditions, recognition memory for multiple faces exceeded that of a single face. This result was primarily due to the higher false alarm rates for faces than non-face objects, even though the two classes of stimuli had been matched for perceptual discriminability. Control experiments revealed that this counterintuitive result emerged only for old-new recognition choices based on near-threshold image differences. For non-face objects, instead, recognition performance decreased with increasing memory load. It is speculated that the low memorial discriminability of the transient properties of a face may serve the purpose of enhancing recognition at the individual-exemplar level.

Basic-level categories: A review

This paper analyses selected literature on basic-level categories, explores related theories and discusses theoretical explanations of the phenomenon of basic-level categories. A substantial body of research has proposed that basic-level categories are the first categories formed during perception of the environment, the first learned by children and those most used in language. Experimental studies suggest that high-level (or superordinate) categories lack informativeness because they are represented by only a few attributes and low-level (or subordinate) categories lack cognitive economy because they are represented by too many attributes. Studies in library and information science have demonstrated the prevalence of basic-level categories in knowledge organization and representation systems such as thesauri and in image indexing and retrieval; and it has been suggested that the universality of basic-level categories could be used for building crosswalks between classificatory systems and user-centred indexing. However, while there is evidence of the pervasiveness of basic-level categories, they may actually be unstable across individuals, domains or cultures and thus unable to support broad generalizations. This paper discusses application of Heidegger’s notion of handiness as a framework for understanding the relational nature of basic-level categories.

Fast and Famous: Looking for the Fastest Speed at Which a Face Can be Recognized

Face recognition is supposed to be fast. However, the actual speed at which faces can be recognized remains unknown. To address this issue, we report two experiments run with speed constraints. In both experiments, famous faces had to be recognized among unknown ones using a large set of stimuli to prevent pre-activation of features which would speed up recognition. In the first experiment (31 participants), recognition of famous faces was investigated using a rapid go/no-go task. In the second experiment, 101 participants performed a highly time constrained recognition task using the Speed and Accuracy Boosting procedure. Results indicate that the fastest speed at which a face can be recognized is around 360–390 ms. Such latencies are about 100 ms longer than the latencies recorded in similar tasks in which subjects have to detect faces among other stimuli. We discuss which model of activation of the visual ventral stream could account for such latencies. These latencies are not consistent with a purely feed-forward pass of activity throughout the visual ventral stream. An alternative is that face recognition relies on the core network underlying face processing identified in fMRI studies (OFA, FFA, and pSTS) and reentrant loops to refine face representation. However, the model of activation favored is that of an activation of the whole visual ventral stream up to anterior areas, such as the perirhinal cortex, combined with parallel and feed-back processes. Further studies are needed to assess which of these three models of activation can best account for face recognition.

The early development of face processing--what makes faces special?

In the present article we review behavioral and neurophysiological studies on face processing in adults and in early development. From the existing empirical and theoretical literature we derive three aspects that distinguish face processing from the processing of other visual object categories. Each of these aspects is discussed from a developmental perspective. First, faces are recognized and represented at the individual level rather than at the basic level. Second, humans typically acquire extensive expertise in individuating faces from early on in development. And third, more than other objects, faces are processed holistically. There is a quantitative difference in the amount of visual experience for faces and other object categories in that the amount of expertise typically acquired for faces is greater than that for other object categories. In addition, we discuss possible qualitative differences in experience for faces and objects. For instance, there is evidence for a sensitive period in infancy for building up a holistic face representation and for perceptual narrowing for faces of one's own species and race. We conclude our literature review with questions for future research, for instance, regarding the exact relationship between behavioral and neuronal markers of face processing across development.

How Fast is Famous Face Recognition?

The rapid recognition of familiar faces is crucial for social interactions. However the actual speed with which recognition can be achieved remains largely unknown as most studies have been carried out without any speed constraints. Different paradigms have been used, leading to conflicting results, and although many authors suggest that face recognition is fast, the speed of face recognition has not been directly compared to “fast” visual tasks. In this study, we sought to overcome these limitations. Subjects performed three tasks, a familiarity categorization task (famous faces among unknown faces), a superordinate categorization task (human faces among animal ones), and a gender categorization task. All tasks were performed under speed constraints. The results show that, despite the use of speed constraints, subjects were slow when they had to categorize famous faces: minimum reaction time was 467 ms, which is 180 ms more than during superordinate categorization and 160 ms more than in the gender condition. Our results are compatible with a hierarchy of face processing from the superordinate level to the familiarity level. The processes taking place between detection and recognition need to be investigated in detail.

The role of features and configural processing in face-race classification

We explored perceptual factors that might account for the other-race classification advantage (ORCA) in classifying faces by race. Testing Chinese participants in China and Israeli participants in Israel we show that: (a) The distinction between Chinese and Israeli faces is highly accurate even on the basis of isolated eyes or faces with eyes concealed, but full faces are categorized faster. (b) The ORCA is similarly robust for full faces and for face parts. (c) The ORCA was larger when the configuration of the inner-face components was distorted, reflecting delayed categorization of own-race distorted faces relative to own-race normally configured faces but no conspicuous distortion effect on other-race faces. These data demonstrate that perceptual factors can account for the ORCA independently of social bias. We suggest that one source of the ORCA in race categorization is the configural analysis applied by default while processing own-race but not other-race faces.

Face-sensitive processes one hundred milliseconds after picture onset

The human face is the most studied object category in visual neuroscience. In a quest for markers of face processing, event-related potential (ERP) studies have debated whether two peaks of activity – P1 and N170 – are category-selective. Whilst most studies have used photographs of unaltered images of faces, others have used cropped faces in an attempt to reduce the influence of features surrounding the “face–object” sensu stricto. However, results from studies comparing cropped faces with unaltered objects from other categories are inconsistent with results from studies comparing whole faces and objects. Here, we recorded ERPs elicited by full front views of faces and cars, either unaltered or cropped. We found that cropping artificially enhanced the N170 whereas it did not significantly modulate P1. In a second experiment, we compared faces and butterflies, either unaltered or cropped, matched for size and luminance across conditions, and within a narrow contrast bracket. Results of Experiment 2 replicated the main findings of Experiment 1. We then used face–car morphs in a third experiment to manipulate the perceived face-likeness of stimuli (100% face, 70% face and 30% car, 30% face and 70% car, or 100% car) and the N170 failed to differentiate between faces and cars. Critically, in all three experiments, P1 amplitude was modulated in a face-sensitive fashion independent of cropping or morphing. Therefore, P1 is a reliable event sensitive to face processing as early as 100 ms after picture onset.

Role of temporal processing stages by inferior temporal neurons in facial recognition

In this review, we focus on the role of temporal stages of encoded facial information in the visual system, which might enable the efficient determination of species, identity, and expression. Facial recognition is an important function of our brain and is known to be processed in the ventral visual pathway, where visual signals are processed through areas V1, V2, V4, and the inferior temporal (IT) cortex. In the IT cortex, neurons show selective responses to complex visual images such as faces, and at each stage along the pathway the stimulus selectivity of the neural responses becomes sharper, particularly in the later portion of the responses. In the IT cortex of the monkey, facial information is represented by different temporal stages of neural responses, as shown in our previous study: the initial transient response of face-responsive neurons represents information about global categories, i.e., human vs. monkey vs. simple shapes, whilst the later portion of these responses represents information about detailed facial categories, i.e., expression and/or identity. This suggests that the temporal stages of the neuronal firing pattern play an important role in the coding of visual stimuli, including faces. This type of coding may be a plausible mechanism underlying the temporal dynamics of recognition, including the process of detection/categorization followed by the identification of objects. Recent single-unit studies in monkeys have also provided evidence consistent with the important role of the temporal stages of encoded facial information. For example, view-invariant facial identity information is represented in the response at a later period within a region of face-selective neurons. Consistent with these findings, temporally modulated neural activity has also been observed in human studies. These results suggest a close correlation between the temporal processing stages of facial information by IT neurons and the temporal dynamics of face recognition.