The independence of expression and identity in face-processing: evidence from neuropsychological case studies

Asymmetries in event-related potentials part 1: A systematic review of face processing studies

The human brain shows distinct lateralized activation patterns for a range of cognitive processes. One such function, which is thought to be lateralized to the right hemisphere (RH), is human face processing. Its importance for social communication and interaction has led to a plethora of studies investigating face processing in health and disease. Temporally highly resolved methods, like event-related potentials (ERPs), allow for a detailed characterization of different processing stages and their specific lateralization patterns. This systematic review aimed at disentangling some of the contradictory findings regarding the RH specialization in face processing focusing on ERP research in healthy participants. Two databases were searched for studies that investigated left and right electrodes while participants viewed (mostly neutral) facial stimuli. The included studies used a variety of different tasks, which ranged from passive viewing to memorizing faces. The final data selection highlights, that strongest lateralization to the RH was found for the N170, especially for right-handed young male participants. Left-handed, female, and older participants showed less consistent lateralization patterns. Other ERP components like the P1, P2, N2, P3, and the N400 were overall less clearly lateralized. The current review highlights that many of the assumed lateralization patterns are less clear than previously thought and that the variety of stimuli, tasks, and EEG setups used, might contribute to the ambiguous findings.

Autistic adults exhibit typical sensitivity to changes in interpersonal distance

The visual processing differences seen in autism often impede individuals' visual perception of the social world. In particular, many autistic people exhibit poor face recognition. Here, we sought to determine whether autistic adults also show impaired perception of dyadic social interactions-a class of stimulus thought to engage face-like visual processing. Our focus was the perception of interpersonal distance. Participants completed distance change detection tasks, in which they had to make perceptual decisions about the distance between two actors. On half of the trials, participants judged whether the actors moved closer together; on the other half, whether they moved further apart. In a nonsocial control task, participants made similar judgments about two grandfather clocks. We also assessed participants' face recognition ability using standardized measures. The autistic and nonautistic observers showed similar levels of perceptual sensitivity to changes in interpersonal distance when viewing social interactions. As expected, however, the autistic observers showed clear signs of impaired face recognition. Despite putative similarities between the visual processing of faces and dyadic social interactions, our results suggest that these two facets of social vision may dissociate.

The role of preSMA and STS in face recognition: A transcranial magnetic stimulation (TMS) study

Current models propose that facial recognition is mediated by two independent yet interacting anatomo-functional systems: one processing facial features mainly mediated by the Fusiform Face Area and the other involved in the extraction of dynamic information from faces, subserved by Superior Temporal Sulcus (STS). Also, the pre-Supplementary Motor Area (pre-SMA) is implicated in facial expression processing as it is involved in its motor mimicry. However, the literature only shows evidence of the implication of STS and preSMA for facial expression recognition, without relating it to face recognition. In addition, the literature shows a facilitatory role of facial motion in the recognition of unfamiliar faces, particularly for poor recognizers. The present study aimed at studying the role of STS and preSMA in unfamiliar face recognition in people with different face recognition skills. 34 healthy participants received repetitive transcranial magnetic stimulation over the right posterior STS, pre-SMA and as sham during a task of matching of faces encoded through: facial expression, rigid head movement or as static (i.e., absence of any facial or head motion). All faces were represented without emotional content. Results indicate that STS has a direct role in recognizing identities through rigid head movement and an indirect role in facial expression processing. This dissociation represents a step forward with respect to current face processing models suggesting that different types of motion involve separate brain and cognitive processes. PreSMA interacts with face recognition skills, increasing the performance of poor recognizers and decreasing that of good recognizers in all presentation conditions. Together, the results suggest the use of at least partially different mechanisms for face recognition in poor and good recognizers and a different role of STS and preSMA in face recognition.

Both identity and non-identity face perception tasks predict developmental prosopagnosia and face recognition ability

Developmental prosopagnosia (DP) is characterised by deficits in face identification. However, there is debate about whether these deficits are primarily perceptual, and whether they extend to other face processing tasks (e.g., identifying emotion, age, and gender; detecting faces in scenes). In this study, 30 participants with DP and 75 controls completed a battery of eight tasks assessing four domains of face perception (identity; emotion; age and gender; face detection). The DP group performed worse than the control group on both identity perception tasks, and one task from each other domain. Both identity perception tests uniquely predicted DP/control group membership, and performance on two measures of face memory. These findings suggest that deficits in DP may arise from issues with face perception. Some non-identity tasks also predicted DP/control group membership and face memory, even when face identity perception was accounted for. Gender perception and speed of face detection consistently predicted unique variance in group membership and face memory; several other tasks were only associated with some measures of face recognition ability. These findings indicate that face perception deficits in DP may extend beyond identity perception. However, the associations between tasks may also reflect subtle aspects of task demands or stimuli.

Positive and negative facial valence perception are modulated differently by eccentricity in the parafovea

Understanding whether people around us are in a good, bad or neutral mood can be critical to our behavior, both when looking directly at them or when they are in our peripheral visual field. However, facial expressions of emotions are often investigated at central visual field or at locations right or left of fixation. Here we assumed that perception of facial emotional valence (the emotion's pleasantness) changes with distance from central visual field (eccentricity) and that different emotions may be influenced differently by eccentricity. Participants (n = 58) judged the valence of emotional faces across the parafovea (≤ 4°, positive (happy), negative (fearful), or neutral)) while their eyes were being tracked. As expected, performance decreased with eccentricity. Positive valence perception was least affected by eccentricity (accuracy reduction of 10-19% at 4°) and negative the most (accuracy reduction of 35-38% at 4°), and this was not a result of speed-accuracy trade-off or response biases. Within-valence (but not across-valence) performance was associated across eccentricities suggesting perception of different valences is supported by different mechanisms. While our results may not generalize to all positive and negative emotions, they indicate that beyond-foveal investigations can reveal additional characteristics of the mechanisms that underlie facial expression processing and perception.

The cultural learning account of first impressions

Humans spontaneously attribute character traits to strangers based on their facial appearance. Although these 'first impressions' typically have no basis in reality, some authors have assumed that they have an innate origin. By contrast, the Trait Inference Mapping (TIM) account proposes that first impressions are products of culturally acquired associative mappings that allow activation to spread from representations of facial appearance to representations of trait profiles. According to TIM, cultural instruments, including propaganda, illustrated storybooks, art and iconography, ritual, film, and TV, expose many individuals within a community to common sources of correlated face-trait experience, yielding first impressions that are shared by many, but typically inaccurate. Here, we review emerging empirical findings, many of which accord with TIM, and argue that future work must distinguish first impressions based on invariant facial features (e.g., shape) from those based on facial behaviours (e.g., expressions).

Face processing in the temporal lobe

Face perception is a socially important but complex process with many stages and many facets. There is substantial evidence from many sources that it involves a large extent of the temporal lobe, from the ventral occipitotemporal cortex and superior temporal sulci to anterior temporal regions. While early human neuroimaging work suggested a core face network consisting of the occipital face area, fusiform face area, and posterior superior temporal sulcus, studies in both humans and monkeys show a system of face patches stretching from posterior to anterior in both the superior temporal sulcus and inferotemporal cortex. Sophisticated techniques such as fMRI adaptation have shown that these face-activated regions show responses that have many of the attributes of human face processing. Lesions of some of these regions in humans lead to variants of prosopagnosia, the inability to recognize the identity of a face. Lesion, imaging, and electrophysiologic data all suggest that there is a segregation between identity and expression processing, though some suggest this may be better characterized as a distinction between static and dynamic facial information.

An update of the Benton Facial Recognition Test

The Benton Facial Recognition Test (BFRT) is a paper-and-pen task that is traditionally used to assess face perception skills in neurological, clinical and psychiatric conditions. Despite criticisms of its stimuli, the task enjoys a simple procedure and is rapid to administer. Further, it has recently been computerised (BFRT-c), allowing reliable measurement of completion times and the need for online testing. Here, in response to calls for repeat screening for the accurate detection of face processing deficits, we present the BFRT-Revised (BFRT-r): a new version of the BFRT-c that maintains the task's basic paradigm, but employs new, higher-quality stimuli that reflect recent theoretical advances in the field. An initial validation study with typical participants indicated that the BFRT-r has good internal reliability and content validity. A second investigation indicated that while younger and older participants had comparable accuracy, completion times were longer in the latter, highlighting the need for age-matched norms. Administration of the BFRT-r and BFRT-c to 32 individuals with developmental prosopagnosia resulted in improved sensitivity in diagnostic screening for the BFRT-r compared to the BFRT-c. These findings are discussed in relation to current diagnostic screening protocols for face perception deficits. The BFRT-r is stored in an open repository and is freely available to other researchers.

Early Capture of Attention by Self-Face: Investigation Using a Temporal Order Judgment Task

Earlier work on self-face processing has reported a bias in the processing of self-face result in faster response to self-face in comparison to other familiar and unfamiliar faces (termed as self-face advantage or SFA). Even though most studies agree that the SFA occurs due to an attentional bias, there is little agreement regarding the stage at which it occurs. While a large number of studies show self-face influencing processing later at disengagement stage, early event-related potential components show differential activity for the self-face suggesting that SFA occurs early. We address this contradiction using a cueless temporal order judgment task that allows us to investigate early perceptual processing, while bias due to top-down expectation is controlled. A greater shift in point of subjective simultaneity for self-face would indicate a greater processing advantage at early perceptual stage. With help of two experiments, we show an early perceptual advantage for self-face, compared to both a friend's face and an unfamiliar face (Experiment 1). This advantage is present even when the effect of criterion shift is minimized (Experiment 2). Interestingly, the magnitude of advantage is similar for self-friend and self-unfamiliar pair. The evidence from the two experiments suggests early capture of attention as a likely reason for the SFA, which is present for the self-face but not for other familiar faces.

What does a "face cell" want?'

In the 1970s Charlie Gross was among the first to identify neurons that respond selectively to faces, in the macaque inferior temporal (IT) cortex. This seminal finding has been followed by numerous studies quantifying the visual features that trigger a response from face cells in order to answer the question; what do face cells want? However, the connection between face-selective activity in IT cortex and visual perception remains only partially understood. Here we present fMRI results in the macaque showing that some face patches respond to illusory facial features in objects. We argue that to fully understand the functional role of face cells, we need to develop approaches that test the extent to which their response explains what we see.

The domain-specificity of face matching impairments in 40 cases of developmental prosopagnosia

A prevailing debate in the psychological literature concerns the domain-specificity of the face recognition system, where evidence from typical and neurological participants has been interpreted as evidence that faces are "special". Although several studies have investigated the same question in cases of developmental prosopagnosia, the vast majority of this evidence has recently been discounted due to methodological concerns. This leaves an uncomfortable void in the literature, restricting our understanding of the typical and atypical development of the face recognition system. The current study addressed this issue in 40 individuals with developmental prosopagnosia, completing a sequential same/different face and biological (hands) and non-biological (houses) object matching task, with upright and inverted conditions. Findings support domain-specific accounts of face-processing for both hands and houses: while significant correlations emerged between all the object categories, no condition correlated with performance in the upright faces condition. Further, a categorical analysis demonstrated that, when face matching was impaired, object matching skills were classically dissociated in six out of 15 individuals (four for both categories). These findings provide evidence about domain-specificity in developmental disorders of face recognition, and present a theoretically-driven means of partitioning developmental prosopagnosia.

Objective Patterns of Face Recognition Deficits in 165 Adults with Self-Reported Developmental Prosopagnosia

In the last 15 years, increasing numbers of individuals have self-referred to research laboratories in the belief that they experience severe everyday difficulties with face recognition. The condition “developmental prosopagnosia” (DP) is typically diagnosed when impairment is identified on at least two objective face-processing tests, usually involving assessments of face perception, unfamiliar face memory, and famous face recognition. While existing evidence suggests that some individuals may have a mnemonic form of prosopagnosia, it is also possible that other subtypes exist. The current study assessed 165 adults who believe they experience DP, and 38% of the sample were impaired on at least two of the tests outlined above. While statistical dissociations between face perception and face memory were only observed in four cases, a further 25% of the sample displayed dissociations between impaired famous face recognition and intact short-term unfamiliar face memory and face perception. We discuss whether this pattern of findings reflects (a) limitations within dominant diagnostic tests and protocols, (b) a less severe form of DP, or (c) a currently unrecognized but prevalent form of the condition that affects long-term face memory, familiar face recognition or semantic processing.

Applied screening tests for the detection of superior face recognition

In recent years there has been growing interest in the identification of people with superior face recognition skills, for both theoretical and applied investigations. These individuals have mostly been identified via their performance on a single attempt at a tightly controlled test of face memory-the long form of the Cambridge Face Memory Test (CFMT+). The consistency of their skills over a range of tests, particularly those replicating more applied policing scenarios, has yet to be examined systematically. The current investigation screened 200 people who believed they have superior face recognition skills, using the CFMT+ and three new, more applied tests (measuring face memory, face matching and composite-face identification in a crowd). Of the sample, 59.5% showed at least some consistency in superior face recognition performance, although only five individuals outperformed controls on overall indices of target-present and target-absent trials. Only one participant outperformed controls on the Crowds test, suggesting that some applied face recognition tasks require very specific skills. In conclusion, future screening protocols need to be suitably thorough to test for consistency in performance, and to allow different types of superior performer to be detected from the outset. Screening for optimal performers may sometimes need to directly replicate the task in question, taking into account target-present and target-absent performance. Self-selection alone is not a reliable means of identifying those at the top end of the face recognition spectrum.

Linking signal detection theory and encoding models to reveal independent neural representations from neuroimaging data

Many research questions in visual perception involve determining whether stimulus properties are represented and processed independently. In visual neuroscience, there is great interest in determining whether important object dimensions are represented independently in the brain. For example, theories of face recognition have proposed either completely or partially independent processing of identity and emotional expression. Unfortunately, most previous research has only vaguely defined what is meant by “independence,” which hinders its precise quantification and testing. This article develops a new quantitative framework that links signal detection theory from psychophysics and encoding models from computational neuroscience, focusing on a special form of independence defined in the psychophysics literature: perceptual separability. The new theory allowed us, for the first time, to precisely define separability of neural representations and to theoretically link behavioral and brain measures of separability. The framework formally specifies the relation between these different levels of perceptual and brain representation, providing the tools for a truly integrative research approach. In particular, the theory identifies exactly what valid inferences can be made about independent encoding of stimulus dimensions from the results of multivariate analyses of neuroimaging data and psychophysical studies. In addition, commonly used operational tests of independence are re-interpreted within this new theoretical framework, providing insights on their correct use and interpretation. Finally, we apply this new framework to the study of separability of brain representations of face identity and emotional expression (neutral/sad) in a human fMRI study with male and female participants.

A common question in vision research is whether certain stimulus properties, like face identity and expression, are represented and processed independently. We develop a theoretical framework that allowed us, for the first time, to link behavioral and brain measures of independence. Unlike previous approaches, our framework formally specifies the relation between these different levels of perceptual and brain representation, providing the tools for a truly integrative research approach in the study of independence. This allows to identify what kind of inferences can be made about brain representations from multivariate analyses of neuroimaging data or psychophysical studies. We apply this framework to the study of independent processing of face identity and expression.

Decisional space determines saccadic reaction times in healthy observers and acquired prosopagnosia

Determining the familiarity and identity of a face have been considered as independent processes. Covert face recognition in cases of acquired prosopagnosia, as well as rapid detection of familiarity have been taken to support this view. We tested P.S. a well-described case of acquired prosopagnosia, and two healthy controls (her sister and daughter) in two saccadic reaction time (SRT) experiments. Stimuli depicted their family members and well-matched unfamiliar distractors in the context of binary gender, or familiarity decisions. Observers' minimum SRTs were estimated with Bayesian approaches. For gender decisions, P.S. and her daughter achieved sufficient performance, but displayed different SRT distributions. For familiarity decisions, her daughter exhibited above chance level performance and minimum SRTs corresponding to those reported previously in healthy observers, while P.S. performed at chance. These findings extend previous observations, indicating that decisional space determines performance in both the intact and impaired face processing system.

Identifying Hallmark Symptoms of Developmental Prosopagnosia for Non-Experts

Developmental prosopagnosia (DP) is characterised by a severe and relatively selective deficit in face recognition, in the absence of neurological injury. Because public and professional awareness of DP is low, many adults and children are not identified for formal testing. This may partly result from the lack of appropriate screening tools that can be used by non-experts in either professional or personal settings. To address this issue, the current study sought to (a) explore when DP can first be detected in oneself and another, and (b) identify a list of the condition’s everyday behavioural manifestations. Questionnaires and interviews were administered to large samples of adult DPs, their unaffected significant others, and parents of children with the condition; and data were analysed using inductive content analysis. It was found that DPs have limited insight into their difficulties, with most only achieving realisation in adulthood. Nevertheless, the DPs’ reflections on their childhood experiences, together with the parental responses, revealed specific indicators that can potentially be used to spot the condition in early childhood. These everyday hallmark symptoms may aid the detection of individuals who would benefit from objective testing, in oneself (in adults) or another person (for both adults and children).

Concurrent development of facial identity and expression discrimination

Facial identity and facial expression processing both appear to follow a protracted developmental trajectory, yet these trajectories have been studied independently and have not been directly compared. Here we investigated whether these processes develop at the same or different rates using matched identity and expression discrimination tasks. The Identity task begins with a target face that is a morph between two identities (Identity A/Identity B). After a brief delay, the target face is replaced by two choice faces: 100% Identity A and 100% Identity B. Children 5-12-years-old were asked to pick the choice face that is most similar to the target identity. The Expression task is matched in format and difficulty to the Identity task, except the targets are morphs between two expressions (Angry/Happy, or Disgust/Surprise). The same children were asked to pick the choice face with the expression that is most similar to the target expression. There were significant effects of age, with performance improving (becoming more accurate and faster) on both tasks with increasing age. Accuracy and reaction times were not significantly different across tasks and there was no significant Age x Task interaction. Thus, facial identity and facial expression discrimination appear to develop at a similar rate, with comparable improvement on both tasks from age five to twelve. Because our tasks are so closely matched in format and difficulty, they may prove useful for testing face identity and face expression processing in special populations, such as autism or prosopagnosia, where one of these abilities might be impaired.