The Speed of Individual Face Categorization

Asymmetries in event-related potentials part 1: A systematic review of face processing studies

The human brain shows distinct lateralized activation patterns for a range of cognitive processes. One such function, which is thought to be lateralized to the right hemisphere (RH), is human face processing. Its importance for social communication and interaction has led to a plethora of studies investigating face processing in health and disease. Temporally highly resolved methods, like event-related potentials (ERPs), allow for a detailed characterization of different processing stages and their specific lateralization patterns. This systematic review aimed at disentangling some of the contradictory findings regarding the RH specialization in face processing focusing on ERP research in healthy participants. Two databases were searched for studies that investigated left and right electrodes while participants viewed (mostly neutral) facial stimuli. The included studies used a variety of different tasks, which ranged from passive viewing to memorizing faces. The final data selection highlights, that strongest lateralization to the RH was found for the N170, especially for right-handed young male participants. Left-handed, female, and older participants showed less consistent lateralization patterns. Other ERP components like the P1, P2, N2, P3, and the N400 were overall less clearly lateralized. The current review highlights that many of the assumed lateralization patterns are less clear than previously thought and that the variety of stimuli, tasks, and EEG setups used, might contribute to the ambiguous findings.

Your place or mine? The neural dynamics of personally familiar scene recognition suggests category independent familiarity encoding

Recognizing a stimulus as familiar is an important capacity in our everyday life. Recent investigation of visual processes has led to important insights into the nature of the neural representations of familiarity for human faces. Still, little is known about how familiarity affects the neural dynamics of non-face stimulus processing. Here we report the results of an EEG study, examining the representational dynamics of personally familiar scenes. Participants viewed highly variable images of their own apartments and unfamiliar ones, as well as personally familiar and unfamiliar faces. Multivariate pattern analyses were used to examine the time course of differential processing of familiar and unfamiliar stimuli. Time-resolved classification revealed that familiarity is decodable from the EEG data similarly for scenes and faces. The temporal dynamics showed delayed onsets and peaks for scenes as compared to faces. Familiarity information, starting at 200 ms, generalized across stimulus categories and led to a robust familiarity effect. In addition, familiarity enhanced category representations in early (250-300 ms) and later (>400 ms) processing stages. Our results extend previous face familiarity results to another stimulus category and suggest that familiarity as a construct can be understood as a general, stimulus-independent processing step during recognition.

Neural evidence of face processing in social anxiety disorder: A systematic review with meta-analysis

Numerous previous studies have used event-related potentials (ERPs) to examine facial processing deficits in individuals with social anxiety disorder (SAD). However, researchers still need to determine whether the deficits are general or specific and what the dominant factors are behind different cognitive stages. Meta-analysis was performed to quantitatively identify face processing deficits in individuals with SAD. Ninety-seven results in 27 publications involving 1032 subjects were calculated using Hedges' g. The results suggest that the face itself elicits enlarged P1 amplitudes, threat-related facial expressions induce larger P2 amplitudes, and negative facial expressions lead to enhanced P3/LPP amplitudes in SAD individuals compared with controls. That is, there is face perception attentional bias in the early phase (P1), threat attentional bias in the mid-term phase (P2), and negative emotion attentional bias in the late phase (P3/LPP), which can be summarized into a three-phase SAD face processing deficit model. These findings provide an essential theoretical basis for cognitive behavioral therapy and have significant application value for the initial screening, intervention, and treatment of social anxiety.

ERP indicators of situational empathy pain

This study aimed to validate a recent conceptualization proposed by Coll and colleagues (2017a) that defines empathic response as a situational, cognitively complex process requiring emotion identification and affective sharing. Sixty right-handed women university students (18-29 years) voluntarily participated in the study. We measured ratings for empathy pain to assess the individual differences in empathy. At the same time, we collected peak amplitudes of the event-related potentials (ERPs) components to empathic stimulations of painful faces or hand stimuli and neutral images. Electrophysiological results proved that the P2, N170, N2, and P3 ERP components were associated with the modulation of empathic responses. Participants with low empathic responses (p < 0.05) disclosed a larger frontal central N2 for the painful hands than for painful faces (p < .05) and a reduced temporoparietal N170 for painful hands compared to neutral ones. Furthermore, our results highlighted higher frontal central P3a and P3b to painful stimuli than controls (p ≤ 0.01). We explained these findings assuming that in identifying the emotional value of a stimulus, the emotional content can modulate the reorientation of attention and the in-memory updating process associated with the empathic response. Results are in line with Coll and colleagues' conceptualization of the empathic response that includes two cognitive processes, the identification of emotions, and affective sharing, related to the recognition of the emotional state of the other in the self.

Contributions of low- and high-level contextual mechanisms to human face perception

Contextual modulations at primary stages of visual processing depend on the strength of local input. Contextual modulations at high-level stages of (face) processing show a similar dependence to local input strength. Namely, the discriminability of a facial feature determines the amount of influence of the face context on that feature. How high-level contextual modulations emerge from primary mechanisms is unclear due to the scarcity of empirical research systematically addressing the functional link between the two. We tested (62) young adults' ability to process local input independent of the context using contrast detection and (upright and inverted) morphed facial feature matching tasks. We first investigated contextual modulation magnitudes across tasks to address their shared variance. A second analysis focused on the profile of performance across contextual conditions. In upright eye matching and contrast detection tasks, contextual modulations only correlated at the level of their profile (averaged Fisher-Z transformed r = 1.18, BF10 > 100), but not magnitude (r = .15, BF10 = .61), suggesting the functional independence but similar working principles of the mechanisms involved. Both the profile (averaged Fisher-Z transformed r = .32, BF10 = 9.7) and magnitude (r = .28, BF10 = 4.58) of the contextual modulations correlated between inverted eye matching and contrast detection tasks. Our results suggest that non-face-specialized high-level contextual mechanisms (inverted faces) work in connection to primary contextual mechanisms, but that the engagement of face-specialized mechanisms for upright faces obscures this connection. Such combined study of low- and high-level contextual modulations sheds new light on the functional relationship between different levels of the visual processing hierarchy, and thus on its functional organization.

The time course of categorical perception of facial expressions

Decoding emotions on others' faces is one of the most important functions of the human brain, which has been widely studied in cognitive neuroscience. However, the precise time course of facial expression categorization in the human brain is still a matter of debate. Here we used an original paradigm to measure categorical perception of facial expression changes during event-related potentials (ERPs) recording, in which a face stimulus dynamically switched either to a different expression (between-category condition) or to the same expression (within-category condition), the physical distance between the two successive faces being equal across conditions. The switch between faces generated a negative differential potential peaking at around 160 ms over occipito-temporal regions, similar in term of latency and topography to the well-known face-selective N170 component. This response was larger in the condition where the switch occurred between faces that were perceived as having different facial expressions compared to the same expression. In addition, happy expressions were categorized around 20 ms faster than fearful expressions (respectively, 135 and 156 ms). These findings provide evidence that changes of facial expressions are categorically perceived as early as 160 ms following stimulus onset over the occipito-temporal cortex.

Frequency-specific gaze modulation of emotional face processing in the human amygdala

Determining the social significance of emotional face expression is of major importance for adaptive behavior, and gaze direction provides critical information in this process. The amygdala is implicated in both emotion and gaze processing, but how and when it integrates expression and gaze cues remains unresolved. We tackled this question using intracranial electroencephalography in epileptic patients to assess both amygdala (n = 12) and orbitofrontal cortex (OFC; n = 11) time-frequency evoked responses to faces with different emotional expressions and different gaze directions. As predicted, self-relevant threat signals (averted fearful and directed angry faces) elicited stronger amygdala activity than self-irrelevant threat (directed fearful and averted angry faces). Fear effects started at early latencies in both amygdala and OFC (~110 and 160 ms, respectively), while gaze direction effects and their interaction with emotion occurred at later latencies. Critically, the amygdala showed differential gamma band increases to fearful averted gaze (starting ~550 ms) and to angry directed gaze (~470 ms). Moreover, when comparing the 2 self-relevant threat conditions among them, we found higher gamma amygdala activity for averted fearful faces and higher beta OFC activity for angry directed faces. Together, these results reveal for the first time frequency-specific effects of emotion and gaze on amygdala and OFC neural activity.

A featural account for own-face processing? Looking for support from face inversion, composite face, and part-whole tasks

It is widely accepted that face perception relies on holistic processing. However, this holistic advantage is not always found in the processing of the own face. Our study aimed to explore the role of holistic and featural processing in the identification of the own face, using three standard, but largely independent measures of holistic face processing: the face inversion task, the composite face task, and the part-whole task. Participants were asked to identify their face, a friend’s face, and an unfamiliar face in three different experimental blocks: (a) inverted versus upright; (b) top and bottom halves of the face aligned versus misaligned; and (c) facial features presented in isolation versus whole foil face context. Inverting a face impaired its identification, regardless of the identity. However, alignment effects were only found when identifying a friend or an unfamiliar face. In addition, a stronger feature advantage (i.e., better recognition for isolated features compared to in a whole-face context) was observed for the own face compared to the friend and unfamiliar faces. Altogether, these findings suggest that the own face is processed in a more featural manner but also relies on holistic processing. This work also highlights the importance of taking into consideration that different holistic processing paradigms could tap different forms of holistic processing.

Observer Gaze Patterns of Patient Photographs Before and After Facial Feminization

BACKGROUND

Human interaction begins with visual evaluation of others, and this often centers on the face. Objective measurement of this evaluation gives clues to social perception.

OBJECTIVES

The objective of this study was to use eye-tracking technology to evaluate if there are scanpath differences when observers view faces of cisgender men, cisgender women, and transgender women before and after facial feminization surgery (FFS) including when assigning tasks assessing femininity, attractiveness, and likability.

METHODS

Undergraduate psychology students were prospectively recruited as observers at a single institution. Their eye movements were recorded by eye-tracking technology when they were presented with frontal photographs of prototypical male, prototypical female, and pre- and post-FFS faces in a random order and then with prompting to assess femininity, attractiveness, and likability.

RESULTS

Twenty-seven observers performed the tasks. Participants focused their attention more on the central triangle of post-FFS and prototypical female images and on the forehead of pre-FFS and prototypical male images. Higher femininity ratings were associated with longer proportional fixations to the central triangle and lower proportional fixations to the forehead.

CONCLUSIONS

This preliminary study implies the scanpath for viewing a post-FFS face is closer to that for viewing a prototypical female than a prototypical male based on differences viewing the forehead and brow vs the central triangle.

What Happens in Your Brain When You Walk Down the Street? Implications of Architectural Proportions, Biophilia, and Fractal Geometry for Urban Science

This article reviews current research in visual urban perception. The temporal sequence of the first few milliseconds of visual stimulus processing sheds light on the historically ambiguous topic of aesthetic experience. Automatic fractal processing triggers initial attraction/avoidance evaluations of an environment’s salubriousness, and its potentially positive or negative impacts upon an individual. As repeated cycles of visual perception occur, the attractiveness of urban form affects the user experience much more than had been previously suspected. These perceptual mechanisms promote walkability and intuitive navigation, and so they support the urban and civic interactions for which we establish communities and cities in the first place. Therefore, the use of multiple fractals needs to reintegrate with biophilic and traditional architecture in urban design for their proven positive effects on health and well-being. Such benefits include striking reductions in observers’ stress and mental fatigue. Due to their costs to individual well-being, urban performance, environmental quality, and climatic adaptation, this paper recommends that nontraditional styles should be hereafter applied judiciously to the built environment.

Contrast Adaptation in Face Perception Revealed Through EEG and Behavior

Exposure to a face can produce biases in the perception of subsequent faces. Typically, these face aftereffects are studied by adapting to an individual face or category (e.g., faces of a given gender) and can result in renormalization of perceptions such that the adapting face appears more neutral. These shifts are analogous to chromatic adaptation, where a renormalization for the average adapting color occurs. However, in color vision, adaptation can also adjust to the variance or range of colors in the distribution. We examined whether this variance or contrast adaptation also occurs for faces, using an objective EEG measure to assess response changes following adaptation. An average female face was contracted or expanded along the horizontal or vertical axis to form four images. Observers viewed a 20 s sequence of the four images presented in a fixed order at a rate of 6 Hz, while responses to the faces were recorded with EEG. A 6 Hz signal was observed over right occipito-temporal channels, indicating symmetric responses to the four images. This test sequence was repeated after 20 s adaptation to alternations between two of the faces (e.g., horizontal contracted and expanded). This adaptation resulted in an additional signal at 3 Hz, consistent with asymmetric responses to adapted and non-adapted test faces. Adapting pairs have the same mean (undistorted) as the test sequence and thus should not bias responses driven only by the mean. Instead, the results are consistent with selective adaptation to the distortion axis. A 3 Hz signal was also observed after adapting to face pairs selected to induce a mean bias (e.g., expanded vertical and expanded horizontal), and this signal was not significantly different from that observed following adaption to a single image that did not form part of the test sequence (e.g., a single image expanded both vertically and horizontally). In a further experiment, we found that this variance adaptation can also be observed behaviorally. Our results suggest that adaptation calibrates face perception not only for the average characteristics of the faces we experience but also for the gamut of faces to which we are exposed.

Who speaks next? Adaptations to speaker identity in processing spoken sentences

When listening to a speaker, we need to adapt to her individual speaking characteristics, such as error proneness, accent, etc. The present study investigated two aspects of adaptation to speaker identity during processing spoken sentences in multi-speaker situations: the effect of speaker sequence across sentences and the effect of learning speaker-specific error probability. Spoken sentences were presented, cued, and accompanied by one of three portraits that were labeled as the speakers' faces. In Block 1 speaker-specific probabilities of syntax errors were 10%, 50%, or 90%; in Block 2 they were uniformly 50%. In both blocks, speech errors elicited P600 effects in the scalp recorded ERP. We found a speaker sequence effect only in Block 1: the P600 to target words was larger after speaker switches than after speaker repetitions, independent of sentence correctness. In Block 1, listeners showed higher accuracy in judging sentence correctness spoken by speakers with lower error proportions. No speaker-specific differences in target word P600 and accuracy were found in Block 2. When speakers differ in error proneness, listeners seem to flexibly adapt their speech processing for the upcoming sentence through attention reorientation and resource reallocation if the speaker is about to change, and through proactive maintenance of neural resources if the speaker remains the same.

Average faces: How does the averaging process change faces physically and perceptually?

Average faces have been used frequently in face recognition studies, either as a theoretical concept (e.g., face norm) or as a tool to manipulate facial attributes (e.g., modifying identity strength). Nonetheless, how the face averaging process- the creation of average faces using an increasing number of faces -changes the resulting averaged faces and our ability to differentiate between them remains to be elucidated. Here we addressed these questions by combining 3D-face averaging, eye-movement tracking, and the computation of image-based face similarity. Participants judged whether two average faces showed the same person while we systematically increased their average level (i.e., number of faces being averaged). Our results showed, with increasing averaging, both a nonlinear increase of the computational similarity between the resulting average faces and a nonlinear decrease of face discrimination performance. Participants' performance dropped from near-ceiling level when two different faces had been averaged together to chance level when 80 faces were mixed. We also found a nonlinear relationship between face similarity and face discrimination performance, which was fitted nicely with an exponential function. Furthermore, when the comparison task became more challenging, participants performed more fixations onto the faces. Nonetheless, the distribution of fixations across facial features (eyes, nose, mouth, and the center area of a face) remained unchanged. These results not only set new constraints on the theoretical characterization of the average face and its role in establishing face norms but also offer practical guidance for creating approximated face norms to manipulate face identity.

Varying Stimulus Duration Reveals Consistent Neural Activity and Behavior for Human Face Individuation

Establishing consistent relationships between neural activity and behavior is a challenge in human cognitive neuroscience research. We addressed this issue using variable time constraints in an oddball frequency-sweep design for visual discrimination of complex images (face exemplars). Sixteen participants viewed sequences of ascending presentation durations, from 25 to 333 ms (40-3 Hz stimulation rate) while their electroencephalogram (EEG) was recorded. Throughout each sequence, the same unfamiliar face picture was repeated with variable size and luminance changes while different unfamiliar facial identities appeared every 1 s (1 Hz). A neural face individuation response, tagged at 1 Hz and its unique harmonics, emerged over the occipito-temporal cortex at 50 ms stimulus duration (25-100 ms across individuals), with an optimal response reached at 170 ms stimulus duration. In a subsequent experiment, identity changes appeared non-periodically within fixed-frequency sequences while the same participants performed an explicit face individuation task. The behavioral face individuation response also emerged at 50 ms presentation time, and behavioral accuracy correlated with individual participants' neural response amplitude in a weighted middle stimulus duration range (50-125 ms). Moreover, the latency of the neural response peaking between 180 and 200 ms correlated strongly with individuals' behavioral accuracy in this middle duration range, as measured independently. These observations point to the minimal (50 ms) and optimal (170 ms) stimulus durations for human face individuation and provide novel evidence that inter-individual differences in the magnitude and latency of early, high-level neural responses are predictive of behavioral differences in performance at this function.

Mapping the Featural and Holistic Face Processing of Bad and Good Face Recognizers

Individual abilities in face recognition (good versus bad recognizers) were explored by means of event-related potentials (ERPs). The adaptation response profile of the N170 component to whole faces, eyes and mouths was used in order to highlight the crucial role of individual abilities in identity repetition processes for unfamiliar faces. The main point of this study is to underline the importance of characterizing the performance (bad or good) of the participants and to show that behaviorally selected groups might reveal neural differences. Good recognizers showed selective right hemisphere N170 repetition effects for whole faces and not for features. On the contrary, bad recognizers showed a general repetition effect not specifically related to faces and more pronounced processing for features. These findings suggest a different contribution of holistic and featural analysis in bad and good performers. In conclusion, we propose that the N170 might be used as a tool to tease apart face encoding processes as a function of individual differences.

Development of infants' representation of female and male faces

We examined development of 5- and 10.5-month-old infants' face representations, focusing on infants' discrimination and categorization of female and male faces. We tested for gender-based preferences and categorization of female and male faces by presenting infants with pairs of faces and then habituating them to a series of majority female or male face ensembles. We then tested for gender preferences with new face pairs (one female and one male; Study 1) or new face ensembles (majority female and majority male; Study 2). We found that both 5- and 10.5-month-old infants discriminated female from male faces in face pairs, and both age groups looked more at female faces during habituation. Neither age group, however, provided evidence of gender-based categorization. We interpret these findings within a theoretical framework that stresses environmental exposure to different social categories, and infants' ability to detect commonalities of features within categories. We conclude that infants' gender-based categorization of faces is constrained by the set of features available in the input.

Self-face perception in 12-month-old infants: A study using the morphing technique

The present study investigated self-face perception in 12-month-old infants using the morphing technique. Twenty-four 12-month-old infants participated in both the main and control experiments. In the main experiment, we used the participant's own face, an unfamiliar infant's face (age- and gender-matched), and a morphed face comprising 50 % each of the self and unfamiliar faces as stimuli. The control experiment followed the same procedure, except that the self-face was replaced with another unfamiliar face. In both experiments, two of these stimuli were presented side by side on a monitor in each trial, and infants' fixation duration was measured. Results showed that shorter fixation durations were found for the morphed face compared with the self-face and the unfamiliar face in the main experiment, but there were no significant preferences for any comparisons in the control experiment. The results suggest that 12-month-old infants could detect subtle differences in facial features between the self-face and the other faces, and infants might show less preference for the self-resembling morphed face due to increased processing costs, which can be interpreted using the uncanny valley hypothesis. Overall, representations of the self-face seem to a certain extent to be formed by the end of the first year of life through daily visual experience.

How familiarity warps representation in the face space

Recognition of familiar as compared to unfamiliar faces is robust and resistant to marked image distortion or degradation. Here we tested the flexibility of familiar face recognition with a morphing paradigm where the appearance of a personally familiar face was mixed with the appearance of a stranger (Experiment 1) and the appearance of one's own face with the appearance of a familiar face and the appearance of a stranger (Experiment 2). The aim of the two experiments was to assess how categorical boundaries for recognition of identity are affected by familiarity. We found a narrower categorical boundary for the identity of personally familiar faces when they were mixed with unfamiliar identities as compared to the control condition, in which the appearance of two unfamiliar faces was mixed. Our results suggest that familiarity warps the representational geometry of face space, amplifying perceptual distances for small changes in the appearance of familiar faces that are inconsistent with the structural features that define their identities.

Multisensory stimulation modulates perceptual and post perceptual face representations: Evidence from event-related potentials

Seeing a face being touched in spatial and temporal synchrony with the own face produces a bias in self-recognition, whereby the other face becomes more likely to be perceived as the self. The present study employed event-related potentials to explore whether this enfacement effect reflects initial face encoding, enhanced distinctiveness of the enfaced face, modified self-identity representations, or even later processing stages that are associated with the emotional processing of faces. Participants were stroked in synchrony or asynchrony with an unfamiliar face they observed on a monitor in front of them, in a situation approximating a mirror image. Subsequently, event-related potentials were recorded during the presentation of (a) a previously synchronously stimulated face, (b) an asynchronously stimulated face, (c) observers' own face, (d) filler faces, and (e) a to-be-detected target face, which required a response. Observers reported a consistent enfacement illusion after synchronous stimulation. Importantly, the synchronously stimulated face elicited more prominent N170 and P200 responses than the asynchronously stimulated face. By contrast, similar N250 and P300 responses were observed in these conditions. These results suggest that enfacement modulates early neural correlates of face encoding and facial prototypicality, rather than identity self-representations and associated emotional processes.

Face categorization and behavioral templates in rats

Rodents have become a popular model in vision science. It is still unclear how vision in rodents relates to primate vision when it comes to complex visual tasks. Here we report on the results of training rats in a face-categorization and generalization task. Additionally, the Bubbles paradigm is used to determine the behavioral templates of the animals. We found that rats are capable of face categorization and can generalize to previously unseen exemplars. Performance is affected-but remains above chance-by stimulus modifications such as upside-down and contrast-inverted stimuli. The behavioral templates of the rats overlap with a pixel-based template, with a bias toward the upper left parts of the stimuli. Together, these findings significantly expand the evidence about the extent to which rats learn complex visual-categorization tasks.

Speed of person perception affects immediate and ongoing aesthetic evaluation

Recent studies have shed light on how aesthetic judgments are formed following presentations lasting less than a second. Meanwhile, dedicated neural mechanisms are understood to enable the rapid detection of human faces, bodies, and actions. On the basis of cognitive studies of: (i) the speed and acuity of person perception, and (ii) preferential attention given to human imagery (e.g., faces and bodies), we hypothesize that the visual detection of humans in portraits increases the magnitude and stability (i.e., similarity to later responses) of aesthetic ratings. Ease of person perception is also expected to elicit longer durations of preferential viewing time, a surplus measure of viewing behavior that should be positively related to subsequent ratings. To test these ideas, we use a set of cubist portraits previously established to be more or less categorizable in terms of the aggregate time required to perceive the depicted person. Using these images, we track aesthetic judgments made following short and unconstrained presentations; in an intervening task, we measure viewing behavior when subjects are able to selectively reveal regions of these images. We find that highly categorizable artworks (those that require less time to identify the figure as human) elicit higher and more predictive aesthetic ratings following 30 ms presentations while also eliciting longer viewing durations. Changes in ratings throughout the task are positively correlated with cumulative viewing time; critically, an image's categorizability level further moderates the strength of this relationship. These results demonstrate that a particular kind of visual object recognition - the recognition of human forms - modulates aesthetic preferences at a glance, subsequent viewing patterns, as well as rating changes over time.

Attentional Weighting in the Face Processing Network: A Magnetic Response Image-guided Magnetoencephalography Study Using Multiple Cyclic Entrainments

We recorded magnetoencephalography using a neural entrainment paradigm with compound face stimuli that allowed for entraining the processing of various parts of a face (eyes, mouth) as well as changes in facial identity. Our magnetic response image-guided magnetoencephalography analyses revealed that different subnodes of the human face processing network were entrained differentially according to their functional specialization. Whereas the occipital face area was most responsive to the rate at which face parts (e.g., the mouth) changed, and face patches in the STS were mostly entrained by rhythmic changes in the eye region, the fusiform face area was the only subregion that was strongly entrained by the rhythmic changes in facial identity. Furthermore, top-down attention to the mouth, eyes, or identity of the face selectively modulated the neural processing in the respective area (i.e., occipital face area, STS, or fusiform face area), resembling behavioral cue validity effects observed in the participants' RT and detection rate data. Our results show the attentional weighting of the visual processing of different aspects and dimensions of a single face object, at various stages of the involved visual processing hierarchy.

Investigating the Time Course of Part-Based and Holistic Processing in Face Perception

Human has an exceptional ability for face recognition to keep up social network. However, it is unclear to understand the mechanisms of face recognition until now. Specifically, there is less research to examine the time course of part-based and holistic processing when these two routes trigger and finish. In the present experiments, the exposure time was manipulated to examine the time course of face processing and found evidence suggesting that holistic processing occurs shortly after part-based processing at about 200 ms, and can last for a relatively long duration up to 2,000 ms. These results may support to a dual-route model comprising holistic processing and part-based processing in face perception. Moreover, our findings were inconsistent with the previous study which suggests that no holistic processing was observed at the relatively long duration, and suspected that perceptual discriminability may have been responsible for the discrepancy.

The automaticity of face perception is influenced by familiarity

In this study, we explore the automaticity of encoding for different facial characteristics and ask whether it is influenced by face familiarity. We used a matching task in which participants had to report whether the gender, identity, race, or expression of two briefly presented faces was the same or different. The task was made challenging by allowing nonrelevant dimensions to vary across trials. To test for automaticity, we compared performance on trials in which the task instruction was given at the beginning of the trial, with trials in which the task instruction was given at the end of the trial. As a strong criterion for automatic processing, we reasoned that if perception of a given characteristic (gender, race, identity, or emotion) is fully automatic, the timing of the instruction should not influence performance. We compared automaticity for the perception of familiar and unfamiliar faces. Performance with unfamiliar faces was higher for all tasks when the instruction was given at the beginning of the trial. However, we found a significant interaction between instruction and task with familiar faces. Accuracy of gender and identity judgments to familiar faces was the same regardless of whether the instruction was given before or after the trial, suggesting automatic processing of these properties. In contrast, there was an effect of instruction for judgments of expression and race to familiar faces. These results show that familiarity enhances the automatic processing of some types of facial information more than others.

Sensitivity to Faces with Typical and Atypical Part Configurations within Regions of the Face-processing Network: An fMRI Study

Perception of faces has been shown to engage a domain-specific set of brain regions, including the occipital face area (OFA) and the fusiform face area (FFA). It is commonly held that the OFA is responsible for the detection of faces in the environment, whereas the FFA is responsible for processing the identity of the face. However, an alternative model posits that the FFA is responsible for face detection and subsequently recruits the OFA to analyze the face parts in the service of identification. An essential prediction of the former model is that the OFA is not sensitive to the arrangement of internal face parts. In the current fMRI study, we test the sensitivity of the OFA and FFA to the configuration of face parts. Participants were shown faces in which the internal parts were presented in a typical configuration (two eyes above a nose above a mouth) or in an atypical configuration (the locations of individual parts were shuffled within the face outline). Perception of the atypical faces evoked a significantly larger response than typical faces in the OFA and in a wide swath of the surrounding posterior occipitotemporal cortices. Surprisingly, typical faces did not evoke a significantly larger response than atypical faces anywhere in the brain, including the FFA (although some subthreshold differences were observed). We propose that face processing in the FFA results in inhibitory sculpting of activation in the OFA, which accounts for this region's weaker response to typical than to atypical configurations.

The influence of spatial frequency content on facial expression processing: An ERP study using rapid serial visual presentation

Spatial frequency (SF) contents have been shown to play an important role in emotion perception. This study employed event-related potentials (ERPs) to explore the time course of neural dynamics involved in the processing of facial expression conveying specific SF information. Participants completed a dual-target rapid serial visual presentation (RSVP) task, in which SF-filtered happy, fearful, and neutral faces were presented. The face-sensitive N170 component distinguished emotional (happy and fearful) faces from neutral faces in a low spatial frequency (LSF) condition, while only happy faces were distinguished from neutral faces in a high spatial frequency (HSF) condition. The later P3 component differentiated between the three types of emotional faces in both LSF and HSF conditions. Furthermore, LSF information elicited larger P1 amplitudes than did HSF information, while HSF information elicited larger N170 and P3 amplitudes than did LSF information. Taken together, these results suggest that emotion perception is selectively tuned to distinctive SF contents at different temporal processing stages.

N170 Reveals the Categorical Perception Effect of Emotional Valence

As an important attribute of facial expression, emotional valence has been well explored, but its processing mechanisms remain ambiguous. Investigating the categorical perception (CP) of emotional valence might help uncover the objective basis of the subjective dichotomy of emotional valence and identify the stage at which this processing of valence information might occur. A judgment task was used in the current study with stimuli from the within- or between-category condition, in which participants were required to decide whether two presented faces showed the same emotion. The results of the behavioral experiment revealed a significant CP effect of emotional valence, with faster RTs and greater accuracy for the between- than for the within-category stimuli. In the ERP experiment, the N170 (peaking at approximately 150-170 ms) was found to reflect the CP effect of emotional valence, with a larger amplitude for the within- than for the between-category condition. In contrast, the P1 component (peaking at approximately 100-130 ms) was insensitive to the CP effect of emotional valence. These results reveal the existence of the CP of emotional valence and indicate that the N170 is its earliest electrophysiological index. Therefore, the categorization of emotional valence not only has an objective neural basis but occurs at a relatively early stage of processing.

Newly learned categories induce pre-attentive categorical perception of faces

Face perception is modulated by categorical information in faces, which is known as categorical perception (CP) of faces. However, it remains unknown whether CP of faces is humans’ inborn capability or the result of acquired categories. Here, we examined whether and when newly learned categories affect face perception. A short-term training method was employed in which participants learned new categories of face stimuli. Behaviorally, using an AB-X discrimination task, we found that the discrimination accuracy of face pairs from different learned categories was significantly higher than that of faces from the same category. Neurally, using a visual oddball task, we found that deviant stimuli whose category differed from standard stimuli evoked a larger N170. Importantly, the visual mismatch negativity (vMMN), starting from 140 ms after stimuli onset, was stronger with the between-category deviants than with the within-category deviants under the unattended condition. Altogether, our study provides empirical evidence indicating that CP of faces could be induced by newly learned categories, and this effect occurs automatically during an early stage of processing.

Individual recognition and the 'face inversion effect' in medaka fish (Oryzias latipes)

Individual recognition (IR) is essential for maintaining various social interactions in a group, and face recognition is one of the most specialised cognitive abilities in IR. We used both a mating preference system and an electric shock conditioning experiment to test IR ability in medaka, and found that signals near the face are important. Medaka required more time to discriminate vertically inverted faces, but not horizontally shifted faces or inverted non-face objects. The ability may be comparable to the classic 'face inversion effect' in humans and some other mammals. Extra patterns added to the face also did not influence the IR. These findings suggest the possibility that the process of face recognition may differ from that used for other objects. The complex form of recognition may promote specific processing adaptations, although the mechanisms and neurological bases might differ in mammals and medaka. The ability to recognise other individuals is important for shaping animal societies.

The neural correlates of vertical disparity gradient and cue conflict in Panum's limiting case

Although Panum's limiting case has been extensively researched, only recently has it been discovered that in addition to horizontal disparity, the final perception of depth is influenced by (i) the vertical disparity gradient and (ii) the degree of cue conflict between 2D and 3D shapes. The present study examines the neural correlates of the two factors, using EEG while observers viewed several versions of stereoscopic stimuli, which depicted Panum's limiting case. In these patterns the vertical disparity gradient was varied from 0.1 to 0.6, while the degree of cue conflict was manipulated from low to high. The ERP data showed that the amplitude of the N170 component (exogenous) was modulated by the vertical disparity gradient and cue conflict. In contrast, the N270 component (endogenous) was modulated by cue conflict only. Such findings demonstrate that both factors affect the perception of depth in Panum's limiting case, but at different stages: the vertical disparity gradient at an early stage of processing (N170) and cue conflict at two stages (N170 and N270). Hence, vertical disparity gradient is related to low-level visual stimulus parameters and can modulate exogenous component, while cue conflict is related to both exogenous and endogenous components.

Attention to body-parts varies with visual preference and verb-effector associations

Theories of embodied conceptual meaning suggest fundamental relations between others' actions, language, and our own actions and visual attention processes. Prior studies have found that when people view an image of a neutral body in a scene they first look toward, in order, the head, torso, hands, and legs. Other studies show associations between action verbs and the body-effectors used in performing the action (e.g., "jump" with feet/legs; "talk" with face/head). In the present experiment, the visual attention of participants was recorded with a remote eye-tracking system while they viewed an image of an actor pantomiming an action and heard a concrete action verb. Participants manually responded whether or not the action image was a good example of the verb they heard. The eye-tracking results confirmed that participants looked at the head most, followed by the hands, and the feet least of all; however, visual attention to each of the body-parts also varied as a function of the effector associated with the spoken verb on image/verb congruent trials, particularly for verbs associated with the legs. Overall, these results suggest that language influences some perceptual processes; however, hearing auditory verbs did not alter the previously reported fundamental hierarchical sequence of directed attention, and fixations on specific body-effectors may not be essential for verb comprehension as peripheral visual cues may be sufficient to perform the task.

Face Context Influences Local Part Processing: An ERP Study

Perception of face parts on the basis of features is thought to be different from perception of whole faces, which is more based on configural information. Face context is also suggested to play an important role in face processing. To investigate how face context influences the early-stage perception of facial local parts, we used an oddball paradigm that tested perceptual stages of face processing rather than recognition. We recorded the event-related potentials (ERPs) elicited by whole faces and face parts presented in four conditions (upright-normal, upright-thatcherised, inverted-normal and inverted-thatcherised), as well as the ERPs elicited by non-face objects (whole houses and house parts) with corresponding conditions. The results showed that face context significantly affected the N170 with increased amplitudes and earlier peak latency for upright normal faces. Removing face context delayed the P1 latency but did not affect the P1 amplitude prominently for both upright and inverted normal faces. Across all conditions, neither the N170 nor the P1 was modulated by house context. The significant changes on the N170 and P1 components revealed that face context influences local part processing at the early stage of face processing and this context effect might be specific for face perception. We further suggested that perceptions of whole faces and face parts are functionally distinguished.

Face identity is encoded in the duration of N170 adaptation

Previous studies assessing the involvement of the face-sensitive N170 component of the event-related potential (ERP) in the processing of face identity have shown controversial results when assessing N170 amplitude in repetition suppression (RS) designs. On the other hand, N170 adaptation is robustly associated with the inter-stimulus interval (ISI) between immediate face repetitions. Interestingly, interactions of face identity and ISI could provide valuable information on early encoding of face identity, but have not been investigated so far. We employed a repetition suppression paradigm using identical and non-identical repetitions as well as parametrically varied ISIs between 500 msec and 2,000 msec in 27 healthy subjects to investigate N170 adaptation effects. Both face identity and varying ISIs significantly influenced N170 adaptation effects, albeit with small effects sizes. Most importantly, however, face identity and ISIs strongly interacted with rapid N170 amplitude recovery in non-identical trials, but sustained N170 adaptation in identical trials. We excluded low-level sensory contributions to the N170 adaptation effect by analyzing the P1 component and by running an additional experiment employing different stimulus sizes. This specific result strongly argues in favor of neuronal sensitivity to face identity, which is primarily mirrored in the N170 temporal decay function that essentially differentiates identical and non-identical face trials. In general, taking advantage of the temporal dimension of adaptation processes, i.e., their decay over time, provides additional dissections of neuronal function into feature-specific selectivity versus non-selectivity.

Representational Dynamics of Facial Viewpoint Encoding

Faces provide a wealth of information, including the identity of the seen person and social cues, such as the direction of gaze. Crucially, different aspects of face processing require distinct forms of information encoding. Another person's attentional focus can be derived based on a view-dependent code. In contrast, identification benefits from invariance across all viewpoints. Different cortical areas have been suggested to subserve these distinct functions. However, little is known about the temporal aspects of differential viewpoint encoding in the human brain. Here, we combine EEG with multivariate data analyses to resolve the dynamics of face processing with high temporal resolution. This revealed a distinct sequence of viewpoint encoding. Head orientations were encoded first, starting after around 60 msec of processing. Shortly afterward, peaking around 115 msec after stimulus onset, a different encoding scheme emerged. At this latency, mirror-symmetric viewing angles elicited highly similar cortical responses. Finally, about 280 msec after visual onset, EEG response patterns demonstrated a considerable degree of viewpoint invariance across all viewpoints tested, with the noteworthy exception of the front-facing view. Taken together, our results indicate that the processing of facial viewpoints follows a temporal sequence of encoding schemes, potentially mirroring different levels of computational complexity.

Early selectivity for vocal and musical sounds: electrophysiological evidence from an adaptation paradigm

There is growing interest in characterizing the neural basis of music perception and, in particular, assessing how similar, or not, it is to that of speech. To further explore this question, we employed an EEG adaptation paradigm in which we compared responses to short sounds belonging to the same category, either speech (pseudo-sentences) or music (piano or violin), depending on whether they were immediately preceded by a same- or different-category sound. We observed a larger reduction in the N100 component magnitude in response to musical sounds when they were preceded by music (either the same or different instrument) than by speech. In contrast, the N100 amplitude was not affected by the preceding stimulus category in the case of speech. For P200 component, we observed a diminution of amplitude when speech sounds were preceded speech, compared to music. No such decrease was found when we compared the responses to music sounds. These differences in the processing of speech and music are consistent with the proposal that some degree of category selectivity for these two classes of complex stimuli already occurs at early stages of auditory processing, possibly subserved by partly separated neuronal populations.

Mnemonic discrimination of similar face stimuli and a potential mechanism for the "other race" effect

Face recognition is an important component of successful social interactions in humans. A large literature in social psychology has focused on the phenomenon termed the "other race" (ORE) effect, the tendency to be more proficient with face recognition within one's own ethnic group compared with other ethnic groups. Several potential hypotheses have been proposed for this effect, including perceptual expertise, social grouping, and holistic face processing. Recent work on mnemonic discrimination (i.e., the ability to resolve mnemonic interference among similar experiences) may provide a mechanistic account for the ORE. In the current study, we examined how discrimination and generalization in the presence of mnemonic interference may contribute to the ORE. We developed a database of computerized faces divided evenly among ethnic origins (Black, Caucasian, East Asian, South Asian), as well as morphed face stimuli that varied in the amount of similarity to the original stimuli (30%, 40%, 50%, and 60% morphs). Participants first examined the original unmorphed stimuli during study, then during test were asked to judge the prior occurrence of repetitions (targets), morphed stimuli (lures), and new stimuli (foils). We examined participants' ability to correctly reject similar morphed lures and found that it increased linearly as a function of face dissimilarity. We additionally found that Caucasian participants' mnemonic discrimination-generalization functions were sharply tuned for Caucasian faces but considerably less tuned for East Asian and Black faces. These results suggest that expertise plays an important role in resolving mnemonic interference, which may offer a mechanistic account for the ORE.

Degraded perceptual and affective processing of racial out-groups: An electrophysiological approach

Human beings process perceptual and affective information of racial out-groups in a degraded manner. Relative to racial in-group members, we lack perceptual individuation of racial out-group members and empathize their pain to a less degree. To date, however, the relationship between the deficiency of individuation and the impairment of empathy in responding to racial out-groups remains elusive. By recording event-related brain potentials in response to racial in-group and out-group faces portraying pain and neutral expressions, we simultaneously measured neural activity that underpinned individuation and empathy. Deficiency in individuating members of racial out-groups, manifesting as reduced reactivity of face-sensitive N170 in the occipitotemporal region of the brain, predicted attenuation of fronto-central empathic response to the suffering of racial out-groups. Further, the individuation bias mediated the influence of racial prejudice on racial in-group bias in empathic neural responses. These findings suggest an interplay between degraded perceptual and affective processing of racial out-groups.

Integration or separation in the processing of facial properties--a computational view

A face recognition system ought to read out information about the identity, facial expression and invariant properties of faces, such as sex and race. A current debate is whether separate neural units in the brain deal with these face properties individually or whether a single neural unit processes in parallel all aspects of faces. While the focus of studies has been directed toward the processing of identity and facial expression, little research exists on the processing of invariant aspects of faces. In a theoretical framework we tested whether a system can deal with identity in combination with sex, race or facial expression using the same underlying mechanism. We used dimension reduction to describe how the representational face space organizes face properties when trained on different aspects of faces. When trained to learn identities, the system not only successfully recognized identities, but also was immediately able to classify sex and race, suggesting that no additional system for the processing of invariant properties is needed. However, training on identity was insufficient for the recognition of facial expressions and vice versa. We provide a theoretical approach on the interconnection of invariant facial properties and the separation of variant and invariant facial properties.

Does Seeing Faces of Young Black Boys Facilitate the Identification of Threatening Stimuli?

Pervasive stereotypes linking Black men with violence and criminality can lead to implicit cognitive biases, including the misidentification of harmless objects as weapons. In four experiments, we investigated whether these biases extend even to young Black boys (5-year-olds). White participants completed sequential priming tasks in which they categorized threatening and nonthreatening objects and words after brief presentations of faces of various races (Black and White) and ages (children and adults). Results consistently revealed that participants had less difficulty (i.e., faster response times, fewer errors) identifying threatening stimuli and more difficulty identifying nonthreatening stimuli after seeing Black faces than after seeing White faces, and this racial bias was equally strong following adult and child faces. Process-dissociation-procedure analyses further revealed that these effects were driven entirely by automatic (i.e., unintentional) racial biases. The collective findings suggest that the perceived threat commonly associated with Black men may generalize even to young Black boys.

Visual adaptation provides objective electrophysiological evidence of facial identity discrimination

Discrimination of facial identities is a fundamental function of the human brain that is challenging to examine with macroscopic measurements of neural activity, such as those obtained with functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). Although visual adaptation or repetition suppression (RS) stimulation paradigms have been successfully implemented to this end with such recording techniques, objective evidence of an identity-specific discrimination response due to adaptation at the level of the visual representation is lacking. Here, we addressed this issue with fast periodic visual stimulation (FPVS) and EEG recording combined with a symmetry/asymmetry adaptation paradigm. Adaptation to one facial identity is induced through repeated presentation of that identity at a rate of 6 images per second (6 Hz) over 10 sec. Subsequently, this identity is presented in alternation with another facial identity (i.e., its anti-face, both faces being equidistant from an average face), producing an identity repetition rate of 3 Hz over a 20 sec testing sequence. A clear EEG response at 3 Hz is observed over the right occipito-temporal (ROT) cortex, indexing discrimination between the two facial identities in the absence of an explicit behavioral discrimination measure. This face identity discrimination occurs immediately after adaptation and disappears rapidly within 20 sec. Importantly, this 3 Hz response is not observed in a control condition without the single-identity 10 sec adaptation period. These results indicate that visual adaptation to a given facial identity produces an objective (i.e., at a pre-defined stimulation frequency) electrophysiological index of visual discrimination between that identity and another, and provides a unique behavior-free quantification of the effect of visual adaptation.

Judged and Remembered Trustworthiness of Faces Is Enhanced by Experiencing Multisensory Synchrony and Asynchrony in the Right Order

This work builds on the enfacement effect. This effect occurs when experiencing a rhythmic stimulation on one’s cheek while seeing someone else’s face being touched in a synchronous way. This typically leads to cognitive and social-cognitive effects similar to self-other merging. In two studies, we demonstrate that this multisensory stimulation can change the evaluation of the other’s face. In the first study, participants judged the stranger’s face and similar faces as being more trustworthy after synchrony, but not after asynchrony. Synchrony interacted with the order of the stroking; hence trustworthiness only changed when the synchronous stimulation occurred before the asynchronous one. In the second study, a synchronous stimulation caused participants to remember the stranger’s face as more trustworthy, but again only when the synchronous stimulation came before the asynchronous one. The results of both studies show that order of stroking creates a context in which multisensory synchrony can affect the trustworthiness of faces.

Adaptation Duration Dissociates Category-, Image-, and Person-Specific Processes on Face-Evoked Event-Related Potentials

Several studies demonstrated that face perception is biased by the prior presentation of another face, a phenomenon termed as face-related after-effect (FAE). FAE is linked to a neural signal-reduction at occipito-temporal areas and it can be observed in the amplitude modulation of the early event-related potential (ERP) components. Recently, macaque single-cell recording studies suggested that manipulating the duration of the adaptor makes the selective adaptation of different visual motion processing steps possible. To date, however, only a few studies tested the effects of adaptor duration on the electrophysiological correlates of human face processing directly. The goal of the current study was to test the effect of adaptor duration on the image-, identity-, and generic category-specific face processing steps. To this end, in a two-alternative forced choice familiarity decision task we used five adaptor durations (ranging from 200-5000 ms) and four adaptor categories: adaptor and test were identical images-Repetition Suppression (RS); adaptor and test were different images of the Same Identity (SameID); adaptor and test images depicted Different Identities (DiffID); the adaptor was a Fourier phase-randomized image (No). Behaviorally, a strong priming effect was observed in both accuracy and response times for RS compared with both DiffID and No. The electrophysiological results suggest that rapid adaptation leads to a category-specific modulation of P100, N170, and N250. In addition, both identity and image-specific processes affected the N250 component during rapid adaptation. On the other hand, prolonged (5000 ms) adaptation enhanced, and extended category-specific adaptation processes over all tested ERP components. Additionally, prolonged adaptation led to the emergence of image-, and identity-specific modulations on the N170 and P2 components as well. In other words, there was a clear dissociation among category, identity-, and image-specific processing steps in the case of longer (3500 and 5000 ms) but not for shorter durations (< 3500 ms), reflected in the gradual reduction of N170 and enhancement of P2 in the No, DiffID, SameID, and RS conditions. Our findings imply that by manipulating adaptation duration one can dissociate the various steps of human face processing, reflected in the ERP response.

A kiss is not a kiss: visually evoked neuromagnetic fields reveal differential sensitivities to brief presentations of kissing couples

With a few exceptions, the literature on face recognition and its neural basis derives from the presentation of single faces. However, in many ecologically typical situations, we see more than one face, in different communicative contexts. One of the principal ways in which we interact using our faces is kissing. Although there is no obvious taxonomy of kissing, we kiss in various interpersonal situations (greeting, ceremony, sex), with different goals and partners. Here, we assess the visual cortical responses elicited by viewing different couples kissing with different intents. The study thus lies at the nexus of face recognition, action recognition, and social neuroscience. Magnetoencephalography data were recorded from nine participants in a passive viewing paradigm. We presented images of couples kissing, with the images differing along two dimensions, kiss type and couple type. We quantified event-related field amplitudes and latencies. In each participant, the canonical sequence of event-related fields was observed, including an M100, an M170, and a later M400 response. The earliest two responses were significantly modulated in latency (M100) or amplitude (M170) by the sex composition of the images (with male-male and female-female pairings yielding faster latency M100 and larger amplitude M170 responses). In contrast, kiss type showed no modulation of any brain response. The early cortical-evoked fields that we typically associate with the presentation and analysis of single faces are differentially sensitive to complex social and action information in face pairs that are kissing. The early responses, typically associated with perceptual analysis, exhibit a consistent grouping and suggest a high and rapid sensitivity to the composition of the kissing pairs.