Neural microgenesis of personally familiar face recognition

The anterior fusiform gyrus: The ghost in the cortical face machine

Face-selective regions in the human ventral occipito-temporal cortex (VOTC) have been defined for decades mainly with functional magnetic resonance imaging. This face-selective VOTC network is traditionally divided in a posterior 'core' system thought to subtend face perception, and regions of the anterior temporal lobe as a semantic memory component of an extended general system. In between these two putative systems lies the anterior fusiform gyrus and surrounding sulci, affected by magnetic susceptibility artifacts. Here we suggest that this methodological gap overlaps with and contributes to a conceptual gap between (visual) perception and semantic memory for faces. Filling this gap with intracerebral recordings and direct electrical stimulation reveals robust face-selectivity in the anterior fusiform gyrus and a crucial role of this region, especially in the right hemisphere, in identity recognition for both familiar and unfamiliar faces. Based on these observations, we propose an integrated theoretical framework for human face (identity) recognition according to which face-selective regions in the anterior fusiform gyrus join the dots between posterior and anterior cortical face memories.

Neural mechanisms of face familiarity and learning in the human amygdala and hippocampus

Recognizing familiar faces and learning new faces play an important role in social cognition. However, the underlying neural computational mechanisms remain unclear. Here, we record from single neurons in the human amygdala and hippocampus and find a greater neuronal representational distance between pairs of familiar faces than unfamiliar faces, suggesting that neural representations for familiar faces are more distinct. Representational distance increases with exposures to the same identity, suggesting that neural face representations are sharpened with learning and familiarization. Furthermore, representational distance is positively correlated with visual dissimilarity between faces, and exposure to visually similar faces increases representational distance, thus sharpening neural representations. Finally, we construct a computational model that demonstrates an increase in the representational distance of artificial units with training. Together, our results suggest that the neuronal population geometry, quantified by the representational distance, encodes face familiarity, similarity, and learning, forming the basis of face recognition and memory.

Intracerebral electrical stimulation of the face-selective right lateral fusiform gyrus transiently impairs face identity recognition

Neuroimaging and intracranial electrophysiological studies have consistently shown the largest and most consistent face-selective neural activity in the middle portion of the human right lateral fusiform gyrus ('fusiform face area(s)', FFA). Yet, direct evidence for the critical role of this region in face identity recognition (FIR) is still lacking. Here we report the first evidence of transient behavioral impairment of FIR during focal electrical stimulation of the right FFA. Upon stimulation of an electrode contact within this region, subject CJ, who shows typical FIR ability outside of stimulation, was transiently unable to point to pictures of famous faces among strangers and to match pictures of famous or unfamiliar faces presented simultaneously for their identity. Her performance at comparable tasks with other visual materials (written names, pictures of buildings) remained unaffected by stimulation at the same location. During right FFA stimulation, CJ consistently reported that simultaneously presented faces appeared as being the same identity, with little or no distortion of the spatial face configuration. Independent electrophysiological recordings showed the largest neural face-selective and face identity activity at the critical electrode contacts. Altogether, this extensive multimodal case report supports the causal role of the right FFA in FIR.

Decreased frontal lobe complexity in left-behind children during joint attention: a fNIRS study with multivariable and multiscale sample entropy analysis

Human brain development is shaped by experiences, especially during preschool, the critical period for cognitive and socioemotional development. This study employed the functional Near-Infrared Spectroscopy technique to explore the neural differences between left-behind children (LBC) and non-left-behind children (NLBC) on joint attention. Through collecting brain image data of 50 children (26 boys, aged 65.08 ± 6.28 months) and conducting multivariable and multiscale sample entropy (MMSE) analysis, the present study found that: (i) LBC showed lower brain complexity than NLBC in right prefrontal cortex; (ii) all participants demonstrated higher brain complexity in responding to joint attention conditions, compared to initiating joint attention ones; (iii) their brain complexity during joint attention was negatively associated with their emotional abilities. The findings advance our understanding of early brain development in LBC by providing evidence for the neural process characteristics of joint attention. Implications for early intervention to promote their brain development are also addressed.

Single neuron responses underlying face recognition in the human midfusiform face-selective cortex

Faces are critical for social interactions and their recognition constitutes one of the most important and challenging functions of the human brain. While neurons responding selectively to faces have been recorded for decades in the monkey brain, face-selective neural activations have been reported with neuroimaging primarily in the human midfusiform gyrus. Yet, the cellular mechanisms producing selective responses to faces in this hominoid neuroanatomical structure remain unknown. Here we report single neuron recordings performed in 5 human subjects (1 male, 4 females) implanted with intracerebral microelectrodes in the face-selective midfusiform gyrus, while they viewed pictures of familiar and unknown faces and places. We observed similar responses to faces and places at the single cell level, but a significantly higher number of neurons responding to faces, thus offering a mechanistic account for the face-selective activations observed in this region. Although individual neurons did not respond preferentially to familiar faces, a population level analysis could consistently determine whether or not the faces (but not the places) were familiar, only about 50 ms after the initial recognition of the stimuli as faces. These results provide insights into the neural mechanisms of face processing in the human brain.

Grasping with a Twist: Dissociating Action Goals from Motor Actions in Human Frontoparietal Circuits

In daily life, prehension is typically not the end goal of hand-object interactions but a precursor for manipulation. Nevertheless, functional MRI (fMRI) studies investigating manual manipulation have primarily relied on prehension as the end goal of an action. Here, we used slow event-related fMRI to investigate differences in neural activation patterns between prehension in isolation and prehension for object manipulation. Sixteen (seven males and nine females) participants were instructed either to simply grasp the handle of a rotatable dial (isolated prehension) or to grasp and turn it (prehension for object manipulation). We used representational similarity analysis (RSA) to investigate whether the experimental conditions could be discriminated from each other based on differences in task-related brain activation patterns. We also used temporal multivoxel pattern analysis (tMVPA) to examine the evolution of regional activation patterns over time. Importantly, we were able to differentiate isolated prehension and prehension for manipulation from activation patterns in the early visual cortex, the caudal intraparietal sulcus (cIPS), and the superior parietal lobule (SPL). Our findings indicate that object manipulation extends beyond the putative cortical grasping network (anterior intraparietal sulcus, premotor and motor cortices) to include the superior parietal lobule and early visual cortex.SIGNIFICANCE STATEMENT A simple act such as turning an oven dial requires not only that the CNS encode the initial state (starting dial orientation) of the object but also the appropriate posture to grasp it to achieve the desired end state (final dial orientation) and the motor commands to achieve that state. Using advanced temporal neuroimaging analysis techniques, we reveal how such actions unfold over time and how they differ between object manipulation (turning a dial) versus grasping alone. We find that a combination of brain areas implicated in visual processing and sensorimotor integration can distinguish between the complex and simple tasks during planning, with neural patterns that approximate those during the actual execution of the action.

Brainprint Recognition Based on the Stable SSVEP Space-Frequency Energy Distribution

Brainprint recognition has received increasing attention in information security. Electroencephalography (EEG) signals measured under task-related or task-free conditions have been exploited as brain biometrics. However, what components make the uniqueness of one's brain signals remains unclear. In this study, we proposed an interpretable biomarker based on steady-state visual evoked potentials (SSVEP) signals for EEG biometric identification. Firstly, we recovered pure SSVEP components from EEG by a point-position equivalent reconstruction (PPER) method. Then, we calculated the distribution properties of SSVEP components in space and frequency. By using the uniform manifold approximation and projection, we reduced the distribution features to 2-dimensions, which shows the separability of the subjects. Lastly, we built a long short-term memory (LSTM) network to perform brainprint recognition on the SSVEP benchmark dataset. The average recognition accuracy can reach up to 98.33%. Our results demonstrate that the space-frequency energy feature of SSVEP is an effective and interpretable biomarker for brainprint recognition. This study provides a further understanding of the uniqueness of individual EEG signal, and facilitates its potential application for personal identification.

Prosopagnosia does not abolish other-race effects

Healthy observers recognize more accurately same-than other-race faces (i.e., the Same-Race Recognition Advantage - SRRA) but categorize them by race more slowly than other-race faces (i.e., the Other-Race Categorization Advantage - ORCA). Several fMRI studies reported discrepant bilateral activations in the Fusiform Face Area (FFA) and Occipital Face Area (OFA) correlating with both effects. However, due to the very nature and limits of fMRI results, whether these face-sensitive regions play an unequivocal causal role in those other-race effects remains to be clarified. To this aim, we tested PS, a well-studied pure case of acquired prosopagnosia with lesions encompassing the left FFA and the right OFA. PS, healthy age-matched and young adults performed two recognition and three categorization by race tasks, respectively using Western Caucasian and East Asian faces normalized for their low-level properties with and without-external features, as well as in naturalistic settings. As expected, PS was slower and less accurate than the controls. Crucially, however, the magnitudes of her SRRA and ORCA were comparable to the controls in all the tasks. Our data show that prosopagnosia does not abolish other-race effects, as an intact face system, the left FFA and/or right OFA are not critical for eliciting the SRRA and ORCA. Race is a strong visual and social signal that is encoded in a large neural face-sensitive network, robustly tuned for processing same-race faces.

Top-down modulation and cortical-AMG/HPC interaction in familiar face processing

Humans can accurately recognize familiar faces in only a few hundred milliseconds, but the underlying neural mechanism remains unclear. Here, we recorded intracranial electrophysiological signals from ventral temporal cortex (VTC), superior/middle temporal cortex (STC/MTC), medial parietal cortex (MPC), and amygdala/hippocampus (AMG/HPC) in 20 epilepsy patients while they viewed faces of famous people and strangers as well as common objects. In posterior VTC and MPC, familiarity-sensitive responses emerged significantly later than initial face-selective responses, suggesting that familiarity enhances face representations after they are first being extracted. Moreover, viewing famous faces increased the coupling between cortical areas and AMG/HPC in multiple frequency bands. These findings advance our understanding of the neural basis of familiar face perception by identifying the top-down modulation in local face-selective response and interactions between cortical face areas and AMG/HPC.

Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part II: Neural basis

Patient PS sustained her dramatic brain injury in 1992, the same year as the first report of a neuroimaging study of human face recognition. The present paper complements the review on the functional nature of PS's prosopagnosia (part I), illustrating how her case study directly, i.e., through neuroimaging investigations of her brain structure and activity, but also indirectly, through neural studies performed on other clinical cases and neurotypical individuals, inspired and constrained neural models of human face recognition. In the dominant right hemisphere for face recognition in humans, PS's main lesion concerns (inputs to) the inferior occipital gyrus (IOG), in a region where face-selective activity is typically found in normal individuals ('Occipital Face Area', OFA). Her case study initially supported the criticality of this region for face identity recognition (FIR) and provided the impetus for transcranial magnetic stimulation (TMS), intracerebral electrical stimulation, and cortical surgery studies that have generally supported this view. Despite PS's right IOG lesion, typical face-selectivity is found anteriorly in the middle portion of the fusiform gyrus, a hominoid structure (termed the right 'Fusiform Face Area', FFA) that is widely considered to be the most important region for human face recognition. This finding led to the original proposal of direct anatomico-functional connections from early visual cortices to the FFA, bypassing the IOG/OFA (lulu), a hypothesis supported by further neuroimaging studies of PS, other neurological cases and neuro-typical individuals with original visual stimulation paradigms, data recordings and analyses. The proposal of a lack of sensitivity to face identity in PS's right FFA due to defective reentrant inputs from the IOG/FFA has also been supported by other cases, functional connectivity and cortical surgery studies. Overall, neural studies of, and based on, the case of prosopagnosia PS strongly question the hierarchical organization of the human neural face recognition system, supporting a more flexible and dynamic view of this key social brain function.

Face learning via brief real-world social interactions includes changes in face-selective brain areas and hippocampus

Making new acquaintances requires learning to recognise previously unfamiliar faces. In the current study, we investigated this process by staging real-world social interactions between actors and the participants. Participants completed a face-matching behavioural task in which they matched photographs of the actors (whom they had yet to meet), or faces similar to the actors (henceforth called foils). Participants were then scanned using functional magnetic resonance imaging (fMRI) while viewing photographs of actors and foils. Immediately after exiting the scanner, participants met the actors for the first time and interacted with them for 10 min. On subsequent days, participants completed a second behavioural experiment and then a second fMRI scan. Prior to each session, actors again interacted with the participants for 10 min. Behavioural results showed that social interactions improved performance accuracy when matching actor photographs, but not foil photographs. The fMRI analysis revealed a difference in the neural response to actor photographs and foil photographs across all regions of interest (ROIs) only after social interactions had occurred. Our results demonstrate that short social interactions were sufficient to learn and discriminate previously unfamiliar individuals. Moreover, these learning effects were present in brain areas involved in face processing and memory.

The sizable difficulty in matching unfamiliar faces differing only moderately in orientation in depth is a function of image dissimilarity

Attempting to match unfamiliar, highly similar faces at moderate differences in orientation in depth is surprisingly difficult. No neurocomputational account of these costs that addressed the representation of faces by which a face-similarity metric can be derived has been offered. A metric specifying the similarity of the to-be-distinguished faces is required as the rotation costs will be a function of the difficulty in distinguishing the faces. Consequently, rotation costs have typically been described in terms of angle of disparity, rather than the dissimilarity of the faces produced by the rotation. We assessed the effects of orientation disparity in a match-to-sample paradigm of a simultaneous presentation of a triangular display of three faces. Two lower test faces, a matching face and a foil, were always at the same orientation and differed by 0° to 20° from the sample on top. The similarity of the images was scaled by a model based on simple cell tuning, modeled as Gabor wavelets, that correlates almost perfectly with psychophysical similarity. Two measures of face similarity, with approximately additive effects on reaction times, accounted for matching performance: a) the decrease in similarity between the images of the matching and sample faces produced by increases in their orientation disparity, and b) the similarity between the matching face and the selection of a particular foil. The 20° orientation disparity was sufficient to yield a sizeable 301 msec increase in reaction time. An implication of the results is that the activity in V1 produced by viewing a face is fed forward to areas responsible for the individuation of that face.

Spatially Adjacent Regions in Posterior Cingulate Cortex Represent Familiar Faces at Different Levels of Complexity

Extensive research has shown that perceptual information of faces is processed in a network of hierarchically-organized areas within ventral temporal cortex. For familiar and famous faces, perceptual processing of faces is normally accompanied by extraction of semantic knowledge about the social status of persons. Semantic processing of familiar faces could entail progressive stages of information abstraction. However, the cortical mechanisms supporting multistage processing of familiar faces have not been characterized. Here, using an event-related fMRI experiment, familiar faces from four celebrity groups (actors, singers, politicians, and football players) and unfamiliar faces were presented to the human subjects (both males and females) while they were engaged in a face categorization task. We systematically explored the cortical representations for faces, familiar faces, subcategories of familiar faces, and familiar face identities using whole-brain univariate analysis, searchlight-based multivariate pattern analysis (MVPA), and functional connectivity analysis. Convergent evidence from all these analyses revealed a set of overlapping regions within posterior cingulate cortex (PCC) that contained decodable fMRI responses for representing different levels of semantic knowledge about familiar faces. Our results suggest a multistage pathway in PCC for processing semantic information of faces, analogous to the multistage pathway in ventral temporal cortex for processing perceptual information of faces.SIGNIFICANCE STATEMENT Recognizing familiar faces is an important component of social communications. Previous research has shown that a distributed network of brain areas is involved in processing the semantic information of familiar faces. However, it is not clear how different levels of semantic information are represented in the brain. Here, we evaluated the multivariate response patterns across the entire cortex to discover the areas that contain information for familiar faces, subcategories of familiar faces, and identities of familiar faces. The searchlight maps revealed that different levels of semantic information are represented in topographically adjacent areas within posterior cingulate cortex (PCC). The results suggest that semantic processing of faces is mediated through progressive stages of information abstraction in PCC.

Shared neural codes for visual and semantic information about familiar faces in a common representational space

Our brain processes faces of close others differently than faces of visually familiar individuals. While both types of faces activate similar visual areas, faces of close others activate areas involved in processing social and semantic information. Here, we used between-subject linear classifiers trained on hyperaligned brain data to investigate the neural code for visual and semantic information about familiar others. The identity of both visually and personally familiar faces could be decoded across participants from brain activity in visual areas. Instead, only the identity of personally familiar faces could be decoded in areas involved in social cognition. Our results suggest that individually distinctive information associated with familiar faces is embedded in a neural code that is shared across brains.

Processes evoked by seeing a personally familiar face encompass recognition of visual appearance and activation of social and person knowledge. Whereas visual appearance is the same for all viewers, social and person knowledge may be more idiosyncratic. Using between-subject multivariate decoding of hyperaligned functional magnetic resonance imaging data, we investigated whether representations of personally familiar faces in different parts of the distributed neural system for face perception are shared across individuals who know the same people. We found that the identities of both personally familiar and merely visually familiar faces were decoded accurately across brains in the core system for visual processing, but only the identities of personally familiar faces could be decoded across brains in the extended system for processing nonvisual information associated with faces. Our results show that personal interactions with the same individuals lead to shared neural representations of both the seen and unseen features that distinguish their identities.

Evidence for a General Neural Signature of Face Familiarity

We explored the neural signatures of face familiarity using cross-participant and cross-experiment decoding of event-related potentials, evoked by unknown and experimentally familiarized faces from a set of experiments with different participants, stimuli, and familiarization-types. Human participants of both sexes were either familiarized perceptually, via media exposure, or by personal interaction. We observed significant cross-experiment familiarity decoding involving all three experiments, predominantly over posterior and central regions of the right hemisphere in the 270-630 ms time window. This shared face familiarity effect was most prominent across the Media and the Personal, as well as between the Perceptual and Personal experiments. Cross-experiment decodability makes this signal a strong candidate for a general neural indicator of face familiarity, independent of familiarization methods, participants, and stimuli. Furthermore, the sustained pattern of temporal generalization suggests that it reflects a single automatic processing cascade that is maintained over time.

Coarse-to-Fine(r) Automatic Familiar Face Recognition in the Human Brain

At what level of spatial resolution can the human brain recognize a familiar face in a crowd of strangers? Does it depend on whether one approaches or rather moves back from the crowd? To answer these questions, 16 observers viewed different unsegmented images of unfamiliar faces alternating at 6 Hz, with spatial frequency (SF) content progressively increasing (i.e., coarse-to-fine) or decreasing (fine-to-coarse) in different sequences. Variable natural images of celebrity faces every sixth stimulus generated an objective neural index of single-glanced automatic familiar face recognition (FFR) at 1 Hz in participants' electroencephalogram (EEG). For blurry images increasing in spatial resolution, the neural FFR response over occipitotemporal regions emerged abruptly with additional cues at about 6.3-8.7 cycles/head width, immediately reaching amplitude saturation. When the same images progressively decreased in resolution, the FFR response disappeared already below 12 cycles/head width, thus providing no support for a predictive coding hypothesis. Overall, these observations indicate that rapid automatic recognition of heterogenous natural views of familiar faces is achieved from coarser visual inputs than generally thought, and support a coarse-to-fine FFR dynamics in the human brain.

Statistical power or more precise insights into neuro-temporal dynamics? Assessing the benefits of rapid temporal sampling in fMRI

Functional magnetic resonance imaging (fMRI), a non-invasive and widely used human neuroimaging method, is most known for its spatial precision. However, there is a growing interest in its temporal sensitivity. This is despite the temporal blurring of neuronal events by the blood oxygen level dependent (BOLD) signal, the peak of which lags neuronal firing by 4-6 seconds. Given this, the goal of this review is to answer a seemingly simple question - "What are the benefits of increased temporal sampling for fMRI?". To answer this, we have combined fMRI data collected at multiple temporal scales, from 323 to 1000 milliseconds, with a review of both historical and contemporary temporal literature. After a brief discussion of technological developments that have rekindled interest in temporal research, we next consider the potential statistical and methodological benefits. Most importantly, we explore how fast fMRI can uncover previously unobserved neuro-temporal dynamics - effects that are entirely missed when sampling at conventional 1 to 2 second rates. With the intrinsic link between space and time in fMRI, this temporal renaissance also delivers improvements in spatial precision. Far from producing only statistical gains, the array of benefits suggest that the continued temporal work is worth the effort.

Critical information thresholds underlying generic and familiar face categorisation at the same face encounter

Seeing a face in the real world provokes a host of automatic categorisations related to sex, emotion, identity, and more. Such individual facets of human face recognition have been extensively examined using overt categorisation judgements, yet their relative informational dependencies during the same face encounter are comparatively unknown. Here we used EEG to assess how increasing access to sensory input governs two ecologically relevant brain functions elicited by seeing a face: Distinguishing faces and nonfaces, and recognising people we know. Observers viewed a large set of natural images that progressively increased in either image duration (experiment 1) or spatial frequency content (experiment 2). We show that in the absence of an explicit categorisation task, the human brain requires less sensory input to categorise a stimulus as a face than it does to recognise whether that face is familiar. Moreover, where sensory thresholds for distinguishing faces/nonfaces were remarkably consistent across observers, there was high inter-individual variability in the lower informational bound for familiar face recognition, underscoring the neurofunctional distinction between these categorisation functions. By i) indexing a form of face recognition that goes beyond simple low-level differences between categories, and ii) tapping multiple recognition functions elicited by the same face encounters, the information minima we report bear high relevance to real-world face encounters, where the same stimulus is categorised along multiple dimensions at once. Thus, our finding of lower informational requirements for generic vs. familiar face recognition constitutes some of the strongest evidence to date for the intuitive notion that sensory input demands should be lower for recognising face category than face identity.

A fast link between face perception and memory in the temporal pole

The question of how the brain recognizes the faces of familiar individuals has been important throughout the history of neuroscience. Cells linking visual processing to person memory have been proposed but not found. Here, we report the discovery of such cells through recordings from an area in the macaque temporal pole identified with functional magnetic resonance imaging. These cells responded to faces that were personally familiar. They responded nonlinearly to stepwise changes in face visibility and detail and holistically to face parts, reflecting key signatures of familiar face recognition. They discriminated between familiar identities, as fast as a general face identity area. The discovery of these cells establishes a new pathway for the fast recognition of familiar individuals.

Getting to Know You: Emerging Neural Representations during Face Familiarization

The successful recognition of familiar persons is critical for social interactions. Despite extensive research on the neural representations of familiar faces, we know little about how such representations unfold as someone becomes familiar. In three EEG experiments on human participants of both sexes, we elucidated how representations of face familiarity and identity emerge from different qualities of familiarization: brief perceptual exposure (Experiment 1), extensive media familiarization (Experiment 2), and real-life personal familiarization (Experiment 3). Time-resolved representational similarity analysis revealed that familiarization quality has a profound impact on representations of face familiarity: they were strongly visible after personal familiarization, weaker after media familiarization, and absent after perceptual familiarization. Across all experiments, we found no enhancement of face identity representation, suggesting that familiarity and identity representations emerge independently during face familiarization. Our results emphasize the importance of extensive, real-life familiarization for the emergence of robust face familiarity representations, constraining models of face perception and recognition memory.SIGNIFICANCE STATEMENT Despite extensive research on the neural representations of familiar faces, we know little about how such representations unfold as someone becomes familiar. To elucidate how face representations change as we get familiar with someone, we conducted three EEG experiments where we used brief perceptual exposure, extensive media familiarization, or real-life personal familiarization. Using multivariate representational similarity analysis, we demonstrate that the method of familiarization has a profound impact on face representations, and emphasize the importance of real-life familiarization. Additionally, familiarization shapes representations of face familiarity and identity differently: as we get to know someone, familiarity signals seem to appear before the formation of identity representations.

Rapid Neural Representations of Personally Relevant Faces

The faces of those most personally relevant to us are our primary source of social information, making their timely perception a priority. Recent research indicates that gender, age and identity of faces can be decoded from EEG/MEG data within 100 ms. Yet, the time course and neural circuitry involved in representing the personal relevance of faces remain unknown. We applied simultaneous EEG-fMRI to examine neural responses to emotional faces of female participants' romantic partners, friends, and a stranger. Combining EEG and fMRI in cross-modal representational similarity analyses, we provide evidence that representations of personal relevance start prior to structural encoding at 100 ms, with correlated representations in visual cortex, but also in prefrontal and midline regions involved in value representation, and monitoring and recall of self-relevant information. Our results add to an emerging body of research that suggests that models of face perception need to be updated to account for rapid detection of personal relevance in cortical circuitry beyond the core face processing network.

Perceptual difficulty modulates the direction of information flow in familiar face recognition

Humans are fast and accurate when they recognize familiar faces. Previous neurophysiological studies have shown enhanced representations for the dichotomy of familiar vs. unfamiliar faces. As familiarity is a spectrum, however, any neural correlate should reflect graded representations for more vs. less familiar faces along the spectrum. By systematically varying familiarity across stimuli, we show a neural familiarity spectrum using electroencephalography. We then evaluated the spatiotemporal dynamics of familiar face recognition across the brain. Specifically, we developed a novel informational connectivity method to test whether peri-frontal brain areas contribute to familiar face recognition. Results showed that feed-forward flow dominates for the most familiar faces and top-down flow was only dominant when sensory evidence was insufficient to support face recognition. These results demonstrate that perceptual difficulty and the level of familiarity influence the neural representation of familiar faces and the degree to which peri-frontal neural networks contribute to familiar face recognition.

Clarifying the role of higher-level cortices in resolving perceptual ambiguity using ultra high field fMRI

The brain is organized into distinct, flexible networks. Within these networks, cognitive variables such as attention can modulate sensory representations in accordance with moment-to-moment behavioral requirements. These modulations can be studied by varying task demands; however, the tasks employed are often incongruent with the postulated functions of a sensory system, limiting the characterization of the system in relation to natural behaviors. Here we combine domain-specific task manipulations and ultra-high field fMRI to study the nature of top-down modulations. We exploited faces, a visual category underpinned by a complex cortical network, and instructed participants to perform either a stimulus-relevant/domain-specific or a stimulus-irrelevant task in the scanner. We found that 1. perceptual ambiguity (i.e. difficulty of achieving a stable percept) is encoded in top-down modulations from higher-level cortices; 2. the right inferior-temporal lobe is active under challenging conditions and uniquely encodes trial-by-trial variability in face perception.

Getting to Know Someone: Familiarity, Person Recognition, and Identification in the Human Brain

In our everyday life, we continuously get to know people, dominantly through their faces. Several neuroscientific experiments showed that familiarization changes the behavioral processing and underlying neural representation of faces of others. Here, we propose a model of the process of how we actually get to know someone. First, the purely visual familiarization of unfamiliar faces occurs. Second, the accumulation of associated, nonsensory information refines person representation, and finally, one reaches a stage where the effortless identification of very well-known persons occurs. We offer here an overview of neuroimaging studies, first evaluating how and in what ways the processing of unfamiliar and familiar faces differs and, second, by analyzing the fMRI adaptation and multivariate pattern analysis results we estimate where identity-specific representation is found in the brain. The available neuroimaging data suggest that different aspects of the information emerge gradually as one gets more and more familiar with a person within the same network. We propose a novel model of familiarity and identity processing, where the differential activation of long-term memory and emotion processing areas is essential for correct identification.

How We Perceive Others Resembling Us

Eye contact is essential for social cognition, acting as an important tool for social communication. While differences in face scanning patterns concerning familiarity have been thoroughly investigated, the impact of facial similarity on gaze behavior has not been examined yet. We addressed this topic by recording subjects' eye-directed gazing while looking at faces that were individually created systematically varying in terms of similarity to the self-face and familiarity. Subjects' self-faces were morphed into three other faces including a close friend of the same sex. Afterwards, they rated similarity to their self-face of those morphed face stimuli in a separate rating task. Our results show a general preference for the eyes' area as well as differences regarding fixation patterns depending on similarity to the self-face. The lower the similarity to the self-face, the more fixations on the eyes' area. Subjects' ratings followed a linear line, indicating well-pronounced face perception. Nevertheless, other faces were rated faster than the self-face independent of familiarity, while morphed faces got the slowest ratings. Our results mirror the importance of similarity to the self-face as a factor shaping the way we look at the eyes of others explaining variance apart from familiarity.

Individual changes in visual performance in non-demented Parkinson's disease patients: a 1-year follow-up study

Cognitive deficits in Parkinson's disease (PD) are heterogeneous entities, and the cognitive status fluctuates over time. However, individual changes in longitudinal cognitive performance in PD are not fully understood. We evaluated three visual indices (visuoperception, visuoconstruction, and visuospatial ability) and four cognitive domains (attention/working memory, executive function, memory, and language) at baseline (Time1) and at 1-year follow-up (Time2) in 36 patients with PD and 32 healthy controls (HCs). To explore the magnitude and frequency of cognitive changes, we analyzed data using the simple difference method and the standardized regression-based method. We also explored the correlations between changes in test scores and several clinical predictors, using logistic regression analysis. At 1 year, patients with PD showed higher rates of change in scores on several cognitive tests, especially the Incomplete Letters test of visuoperception, compared to HCs. After adjusting for demographic variables, the visuoperceptual change was 61.1% overall, with the largest effect size. The changes in scores of visuoperception correlated with those of memory (r = 0.672, p < 0.001), language (r = 0.389, p < 0.05), and visuospatial ability (r = 0.379, p < 0.05). The severity of olfactory disturbance, the MDS-UPDRS Part I score, and younger PD onset predicted the significant changes observed in the Incomplete Letters test scores. Visuoperception changed more in non-demented PD patients than in HCs at 1-year follow-up. The changes in visuoperception could relate to involvement of the ventral occipitotemporal pathway, the more widespread temporal lobe, and brain reserve in PD.

Typical visual unfamiliar face individuation in left and right mesial temporal epilepsy

Patients with chronic mesial temporal lobe epilepsy have difficulties at identifying familiar faces as well as at explicit old/new face recognition tasks. However, the extent to which these difficulties can be attributed to visual individuation of faces, independently of general explicit learning and semantic memory processes, is unknown. We tested 42 mesial temporal lobe epilepsy patients divided into two groups according to the side of epilepsy (left and right) and 42 matched controls on an extensive series of individuation tasks of unfamiliar faces and control visual stimuli, as well as on face detection, famous face recognition and naming, and face and non-face learning. Overall, both patient groups had difficulties at identifying and naming famous faces, and at explicitly learning face and non-face images. However, there was no group difference in accuracy between patients and controls at the two most widely used neuropsychological tests assessing visual individuation of unfamiliar faces (Benton Facial Recognition Test and Cambridge Face Memory Test). While patients with right mesial temporal lobe epilepsy were slowed down at all tasks, this effect was not specific to faces or even high-level stimuli. Importantly, both groups showed the same profile of response as typical participants across various stimulus manipulations, showing no evidence of qualitative processing impairments. Overall, these results point to largely preserved visual face individuation processes in patients with mesial temporal lobe epilepsy, with semantic and episodic memory difficulties being consistent with the localization of the neural structures involved in their epilepsy (anterior temporal cortex and hippocampus). These observations have implications for the prediction of neuropsychological outcomes in the case of surgery and support the validity of intracranial electroencephalographic recordings performed in this population to understand neural mechanisms of human face individuation, notably through intracranial electrophysiological recordings and stimulations.

Catching the imposter in the brain: The case of Capgras delusion

Here we describe a rare case of Capgras delusion - a misidentification syndrome characterized by the belief that a person has been replaced by an imposter - in a patient without evident neurological or psychiatric symptoms. Intriguingly, delusional belief was selective for both person and modality, as the patient believed that his son - not his daughter or other relatives - was substituted with an imposter only while being in presence of him and looking at his face, but not when merely listening to his voice. A neuroanatomical reconstruction obtained integrating morphological and functional patient's neuroimaging data highlighted two main peculiarities: a compression of the rostral portion of right temporal lobe due to a large arachnoid cyst, and a bilaterally reduced metabolism of frontal areas. Autonomic data obtained from thermal infra-red camera and skin conductance recordings showed that a higher sympathetic activation was evoked by the observation of daughter's face, relative to the observation of the son's face as well as of not-familiar faces; conversely, daughter and son voices elicited a similar sympathetic activation, higher relative to not-familiar voices, indicating a modality-dependent dissociation consistent with the delusional behavior. Our case supports the "two-hit hypothesis" about Capgras delusion etiopathogenesis: here, the first hit is represented by the right-temporal lesion impairing the association between familiar faces and emotional values, the second one is the frontal bilateral hypometabolism favoring delusional behavior. The selective occurrence of "imposter" delusion for a particular subject and for a specific perceptual modality suggests the involvement of modality-specific interactions in the retrieval of affective properties during familiar people recognition.

COMT-Polymorphisms Modulated Functional Profile of the Fusiform Face Area Contributes to Face-Specific Recognition Ability

Previous studies have shown that face-specific recognition ability (FRA) is heritable; however, the neural basis of this heritability is unclear. Candidate gene studies have suggested that the catechol-O-methyltransferase (COMT) rs4680 polymorphism is related to face perception. Here, using a partial least squares (PLS) method, we examined the multivariate association between 12 genotypes of 4 COMT polymorphisms (rs6269-rs4633-rs4818-rs4680) and multimodal MRI phenotypes in the human fusiform face area (FFA), which selectively responds to face stimuli, in 338 Han Chinese adults (mean age 20.45 years; 135 males). The MRI phenotypes included gray matter volume (GMV), resting-state fractional amplitude of low-frequency fluctuations (fALFF), and face-selective blood-oxygen-level-dependent (BOLD) responses (FS). We found that the first COMT-variant component (PLS1) was positively associated with the FS but negatively associated with the fALFF in the FFA. Moreover, participants with the COMT heterozygous-HEA-haplotype showed higher PLS1 FFA-MRI scores, which were positively associated with the FRA in an old/new face recognition task, than those with the COMT homozygous HEA haplotype and HEA non-carriers, suggesting that individuals with an appropriate (intermediate) level of dopamine activity in the FFA might have better FRA. In summary, our study provides empirical evidence for the genetic and neural basis for the heritability of face recognition and informs the formation of neural module functional specificity.

The neural mechanisms of face processing: cells, areas, networks, and models

Since its discovery, the face-processing network in the brain of the macaque monkey has emerged as a model system that allowed for major neural mechanisms of face recognition to be identified - with implications for object recognition at large. Populations of face cells encode faces through broad tuning curves, whose shapes change over time. Face representations differ qualitatively across faces areas, and we not only understand the global organization of these specializations, but also some of the transformations between face areas, both feed-forward and feed-back, and the computational principles behind face representations and transformations. Facial information is combined with physical features and mnemonic features in extensions of the core network, which forms an early part of the primate social brain.

Filtered beauty in Oslo and Tokyo: A spatial frequency analysis of facial attractiveness

Images of European female and male faces were digitally processed to generate spatial frequency (SF) filtered images containing only a narrow band of visual information within the Fourier spectrum. The original unfiltered images and four SF filtered images (low, medium-low, medium-high and high) were then paired in trials that kept constant SF band and face gender and participants made a forced-choice decision about the more attractive among the two faces. In this way, we aimed at identifying those specific SF bands where forced-choice preferences corresponded best to forced-choice judgements made when viewing the natural, broadband, facial images. We found that aesthetic preferences dissociated across SFs and face gender, but similarly for participants from Asia (Japan) and Europe (Norway). Specifically, preferences when viewing SF filtered images were best related to the preference with the broadband face images when viewing the highest filtering band for the female faces (about 48-77 cycles per face). In contrast, for the male faces, the medium-low SF band (about 11-19 cpf) related best to choices made with the natural facial images. Eye tracking provided converging evidence for the above, gender-related, SF dissociations. We suggest greater aesthetic relevance of the mobile and communicative parts for the female face and, conversely, of the rigid, structural, parts for the male face for facial aesthetics.

Anton's syndrome: a rare and unusual form of blindness

Anton syndrome is characterised by visual anosognosia. It results from damage to both occipital lobes, while the anterior visual pathways remain intact. We describe four cases of Anton's syndrome. First case is that of a 73-year-old woman, who presented with two separate events of intraparenchymal brain haemorrhage, 4 years apart. Her first stroke affected the left and second affected the right occipital lobe. Bilateral occipital lobe damage resulted in cortical blindness. Second case is an 88-year-old man, who suffered from two ischaemic strokes, 2 days apart. Each stroke involved one posterior cerebral artery. This resulted in bilateral occipital and temporal lobe infarcts. Third case is a 64-year-old woman with chronic renal failure, who suffered bilateral occipital lobe infarction after haemodialysis, due to posterior reversible encephalopathy syndrome. Last case is that of an 80-year-old woman who suffered a basilar artery stroke, resulting in bilateral thalamic, temporal and occipital lobe infarction.

Modelling face memory reveals task-generalizable representations

Current cognitive theories are cast in terms of information-processing mechanisms that use mental representations^1-4. For example, people use their mental representations to identify familiar faces under various conditions of pose, illumination and ageing, or to draw resemblance between family members. Yet, the actual information contents of these representations are rarely characterized, which hinders knowledge of the mechanisms that use them. Here, we modelled the three-dimensional representational contents of 4 faces that were familiar to 14 participants as work colleagues. The representational contents were created by reverse-correlating identity information generated on each trial with judgements of the face's similarity to the individual participant's memory of this face. In a second study, testing new participants, we demonstrated the validity of the modelled contents using everyday face tasks that generalize identity judgements to new viewpoints, age and sex. Our work highlights that such models of mental representations are critical to understanding generalization behaviour and its underlying information-processing mechanisms.

Unconscious Detection of One's Own Image

A key mechanism behind preferential processing of self-related information might be an early and automatic capture of attention. Therefore, the present study tested a hypothesis that one's own face will attract bottom-up attention even without conscious identification. To test this, we used a dot-probe paradigm with electrophysiological recordings, in which participants ( N = 18) viewed masked and unmasked pairs of faces (other, self) presented laterally. Analysis of the sensitivity measure d ' indicated that faces were not consciously identified in the masked condition. A clear N2 posterior-contralateral (N2pc) component (a neural marker of attention shifts) was found in both the masked and unmasked conditions, revealing that one's own face automatically captures attention when processed unconsciously. Therefore, our study (a) demonstrates that self-related information is boosted at an early (preconscious) stage of processing, (b) identifies further features (beyond simple physical ones) that cause automatic attention capture, and (c) provides further evidence for the dissociative nature of attention and consciousness.

Idiosyncratic, Retinotopic Bias in Face Identification Modulated by Familiarity

The perception of gender and age of unfamiliar faces is reported to vary idiosyncratically across retinal locations such that, for example, the same androgynous face may appear to be male at one location but female at another. Here, we test spatial heterogeneity for the recognition of the identity of personally familiar faces in human participants. We found idiosyncratic biases that were stable within participants and that varied more across locations for low as compared to high familiar faces. These data suggest that like face gender and age, face identity is processed, in part, by independent populations of neurons monitoring restricted spatial regions and that the recognition responses vary for the same face across these different locations. Moreover, repeated and varied social interactions appear to lead to adjustments of these independent face recognition neurons so that the same familiar face is eventually more likely to elicit the same recognition response across widely separated visual field locations. We provide a mechanistic account of this reduced retinotopic bias based on computational simulations.

Temporal multivariate pattern analysis (tMVPA): A single trial approach exploring the temporal dynamics of the BOLD signal

Background:

fMRI provides spatial resolution that is unmatched by non-invasive neuroimaging techniques. Its temporal dynamics however are typically neglected due to the sluggishness of the hemodynamic signal.

New Methods:

We present temporal multivariate pattern analysis (tMVPA), a method for investigating the temporal evolution of neural representations in fMRI data, computed on single-trial BOLD time-courses, leveraging both spatial and temporal components of the fMRI signal. We implemented an expanding sliding window approach that allows identifying the time-window of an effect.

Results:

We demonstrate that tMVPA can successfully detect condition-specific multivariate modulations over time, in the absence of mean BOLD amplitude differences. Using Monte-Carlo simulations and synthetic data, we quantified family-wise error rate (FWER) and statistical power. Both at the group and single-subject levels, FWER was either at or significantly below 5%. We reached the desired power with 18 subjects and 12 trials for the group level, and with 14 trials in the single-subject scenario.

Comparison with existing methods:

We compare the tMVPA statistical evaluation to that of a linear support vector machine (SVM). SVM outperformed tMVPA with large N and trial numbers. Conversely, tMVPA, leveraging on single trials analyses, outperformed SVM in low N and trials and in a single-subject scenario.

Conclusion:

Recent evidence suggesting that the BOLD signal carries finer-grained temporal information than previously thought, advocates the need for analytical tools, such as tMVPA, tailored to investigate BOLD temporal dynamics. The comparable performance between tMVPA and SVM, a powerful and reliable tool for fMRI, supports the validity of our technique.

Neurobiological Aspects of Face Recognition: The Role of Oxytocin

Face recognition is an important index in the formation of social cognition and neurodevelopment in humans. Changes in face perception and memory are connected with altered sociability, which is a symptom of numerous brain conditions including autism spectrum disorder (ASD). Various brain regions and neuropeptides are implicated in face processing. The neuropeptide oxytocin (OT) plays an important role in various social behaviors, including face and emotion recognition. Nasal OT administration is a promising new therapy that can address social cognition deficits in individuals with ASD. New instrumental neurotechnologies enable the assessment of brain region activation during specific social tasks and therapies, and can characterize the involvement of genes and peptides in impaired neurodevelopment. The present review sought to discuss some of the mechanisms of the face distinguishing process, the ability of OT to modulate social cognition, as well as new perspectives and technologies for research and rehabilitation of face recognition.

Distinct neural processes for the perception of familiar versus unfamiliar faces along the visual hierarchy revealed by EEG

Humans recognize faces with ease, despite the complexity of the task and of the visual system which underlies it. Different spatial regions, including both the core and extended face processing networks, and distinct temporal stages of processing have been implicated in face recognition, but there is ongoing controversy regarding the extent to which the mechanisms for recognizing a familiar face differ from those for an unfamiliar face. Here, we used electroencephalogram (EEG) and flicker SSVEP, a high signal-to-noise approach, and searchlight decoding methods to elucidate the mechanisms mediating the processing of familiar and unfamiliar faces in the time domain. Familiar and unfamiliar faces were presented periodically at 15 Hz, 6 Hz and 3.75 Hz either upright or inverted in separate blocks, with the rationale that faster frequencies require shorter processing times per image and tap into fundamentally different levels of visual processing. The 15 Hz trials, likely to reflect early visual processing, exhibited enhanced neural responses for familiar over unfamiliar face trials, but only when the faces were upright. In contrast, decoding methods revealed similar classification accuracies for upright and inverted faces for both familiar and unfamiliar faces. For the 6 Hz frequency, familiar faces had lower amplitude responses than unfamiliar faces, and decoding familiarity was more accurate for upright compared with inverted faces. Finally, the 3.75 Hz frequency revealed no main effects of familiarity, but decoding showed significant correlations with behavioral ratings of face familiarity, suggesting that activity evoked by this slow presentation frequency reflected higher-level, cognitive aspects of familiarity processing. This three-way dissociation between frequencies reveals that fundamentally different stages of the visual hierarchy are modulated by face familiarity. The combination of experimental and analytical approaches used here represent a novel method for elucidating spatio-temporal characteristics within the visual system.

Differential Resting-State Connectivity Patterns of the Right Anterior and Posterior Dorsolateral Prefrontal Cortices (DLPFC) in Schizophrenia

In schizophrenia (SCZ), dysfunction of the dorsolateral prefrontal cortex (DLPFC) has been linked to the deficits in executive functions and attention. It has been suggested that, instead of considering the right DLPFC as a cohesive functional entity, it can be divided into two parts (anterior and posterior) based on its whole-brain connectivity patterns. Given these two subregions' differential association with cognitive processes, we investigated the functional connectivity (FC) profile of both subregions through resting-state data to determine whether they are differentially affected in SCZ. Resting-state magnetic resonance imaging (MRI) scans were obtained from 120 patients and 172 healthy controls (HC) at 6 different MRI sites. The results showed differential FC patterns for the anterior and posterior parts of the right executive control-related DLPFC in SCZ with the parietal, the temporal and the cerebellar regions, along with a convergent reduction of connectivity with the striatum and the occipital cortex. An increased psychopathology level was linked to a higher difference in posterior vs. anterior FC for the left IFG/anterior insula, regions involved in higher-order cognitive processes. In sum, the current analysis demonstrated that even between two neighboring clusters connectivity could be differentially disrupted in SCZ. Lacking the necessary anatomical specificity, such notions may in fact be detrimental to a proper understanding of SCZ pathophysiology.

The neural representation of personally familiar and unfamiliar faces in the distributed system for face perception

Personally familiar faces are processed more robustly and efficiently than unfamiliar faces. The human face processing system comprises a core system that analyzes the visual appearance of faces and an extended system for the retrieval of person-knowledge and other nonvisual information. We applied multivariate pattern analysis to fMRI data to investigate aspects of familiarity that are shared by all familiar identities and information that distinguishes specific face identities from each other. Both identity-independent familiarity information and face identity could be decoded in an overlapping set of areas in the core and extended systems. Representational similarity analysis revealed a clear distinction between the two systems and a subdivision of the core system into ventral, dorsal and anterior components. This study provides evidence that activity in the extended system carries information about both individual identities and personal familiarity, while clarifying and extending the organization of the core system for face perception.

Two areas for familiar face recognition in the primate brain

Familiarity alters face recognition: Familiar faces are recognized more accurately than unfamiliar ones and under difficult viewing conditions when unfamiliar face recognition fails. The neural basis for this fundamental difference remains unknown. Using whole-brain functional magnetic resonance imaging, we found that personally familiar faces engage the macaque face-processing network more than unfamiliar faces. Familiar faces also recruited two hitherto unknown face areas at anatomically conserved locations within the perirhinal cortex and the temporal pole. These two areas, but not the core face-processing network, responded to familiar faces emerging from a blur with a characteristic nonlinear surge, akin to the abruptness of familiar face recognition. In contrast, responses to unfamiliar faces and objects remained linear. Thus, two temporal lobe areas extend the core face-processing network into a familiar face-recognition system.