A face-responsive potential recorded from the human scalp

Machine learning and EEG can classify passive viewing of discrete categories of visual stimuli but not the observation of pain

Previous studies have demonstrated the potential of machine learning (ML) in classifying physical pain from non-pain states using electroencephalographic (EEG) data. However, the application of ML to EEG data to categorise the observation of pain versus non-pain images of human facial expressions or scenes depicting pain being inflicted has not been explored. The present study aimed to address this by training Random Forest (RF) models on cortical event-related potentials (ERPs) recorded while participants passively viewed faces displaying either pain or neutral expressions, as well as action scenes depicting pain or matched non-pain (neutral) scenarios. Ninety-one participants were recruited across three samples, which included a model development group (n = 40) and a cross-subject validation group (n = 51). Additionally, 25 participants from the model development group completed a second experimental session, providing a within-subject temporal validation sample. The analysis of ERPs revealed an enhanced N170 component in response to faces compared to action scenes. Moreover, an increased late positive potential (LPP) was observed during the viewing of pain scenes compared to neutral scenes. Additionally, an enhanced P3 response was found when participants viewed faces displaying pain expressions compared to neutral expressions. Subsequently, three RF models were developed to classify images into faces and scenes, neutral and pain scenes, and neutral and pain expressions. The RF model achieved classification accuracies of 75%, 64%, and 69% for cross-validation, cross-subject, and within-subject classifications, respectively, along with reasonably calibrated predictions for the classification of face versus scene images. However, the RF model was unable to classify pain versus neutral stimuli above chance levels when presented with subsequent tasks involving images from either category. These results expand upon previous findings by externally validating the use of ML in classifying ERPs related to different categories of visual images, namely faces and scenes. The results also indicate the limitations of ML in distinguishing pain and non-pain connotations using ERP responses to the passive viewing of visually similar images.

Neural timing of the other-race effect across the lifespan: A review

Face race influences the way we process faces, so that faces of a different ethnic group are processed for identity less efficiently than faces of one's ethnic group - a phenomenon known as the Other-Race Effect (ORE). Although widely replicated, the ORE is still poorly characterized in terms of its development and the underlying mechanisms. In the last two decades, the Event-Related Potential (ERP) technique has brought insight into the mechanisms underlying the ORE and has demonstrated potential to clarify its development. Here, we review the ERP evidence for a differential neural processing of own-race and other-race faces throughout the lifespan. In infants, race-related processing differences emerged at the N290 and P400 (structural encoding) stages. In children, race affected the P100 (early processing, attention) perceptual stage and was implicitly encoded at the N400 (semantic processing) stage. In adults, processing difficulties for other-race faces emerged at the N170 (structural encoding), P200 (configuration processing) and N250 (accessing individual representations) perceptual stages. Early in processing, race was implicitly encoded from other-race faces (N100, P200 attentional biases) and in-depth processing preferentially applied to own-race faces (N200 attentional bias). Encoding appeared less efficient (Dm effects) and retrieval less recollection-based (old/new effects) for other-race faces. Evidence admits the contribution of perceptual, attentional, and motivational processes to the development and functioning of the ORE, offering no conclusive support for perceptual or socio-cognitive accounts. Cross-racial and non-cross-racial studies provided convergent evidence. Future research would need to include less represented ethnic populations and the developmental population.

Dissociable effects of emotional stimuli on electrophysiological indices of time and decision-making

Previous research has demonstrated that emotional faces affect time perception, however, the underlying mechanisms are not fully understood. Earlier attempts focus on effects at the different stages of the pacemaker-accumulator model (clock, memory, and/or decision-making) including, an increase in pacemaker rate or accumulation rate via arousal or attention, respectively, or by biasing decision-making. A visual temporal bisection task with sub-second intervals was conducted in two groups to further investigate these effects; one group was strictly behavioral whereas the second included a 64-channel electroencephalogram (EEG). To separate the influence of face and timing responses, participants timed a visual stimulus, temporally flanked (before and after) by two faces, either negative or neutral, creating three trial-types: Neg→Neut, Neut→Neg, or Neut→Neut. We found a leftward shift in bisection point (BP) in Neg→Neut relative to Neut→Neut suggests an overestimation of the temporal stimulus when preceded by a negative face. Neurally, we found the face-responsive N170 was larger for negative faces and the N1 and contingent negative variation (CNV) were larger when the temporal stimulus was preceded by a negative face. Additionally, there was an interaction effect between condition and response for the late positive component of timing (LPCt) and a significant difference between response (short/long) in the neutral condition. We concluded that a preceding negative face affects the clock stage leading to more pulses being accumulated, either through attention or arousal, as indexed by a larger N1, CNV, and N170; whereas viewing a negative face after impacted decision-making mechanisms, as evidenced by the LPCt.

Neural specialization to human faces at the age of 7 months

Sensitivity to human faces has been suggested to be an early emerging capacity that promotes social interaction. However, the developmental processes that lead to cortical specialization to faces has remained unclear. The current study investigated both cortical sensitivity and categorical specificity through event-related potentials (ERPs) previously implicated in face processing in 7-month-old infants (N290) and adults (N170). Using a category-specific repetition/adaptation paradigm, cortical specificity to human faces, or control stimuli (cat faces), was operationalized as changes in ERP amplitude between conditions where a face probe was alternated with categorically similar or dissimilar adaptors. In adults, increased N170 for human vs. cat faces and category-specific release from adaptation for face probes alternated with cat adaptors was found. In infants, a larger N290 was found for cat vs. human probes. Category-specific repetition effects were also found in infant N290 and the P1-N290 peak-to-peak response where latter indicated category-specific release from adaptation for human face probes resembling that found in adults. The results suggest cortical specificity to human faces during the first year of life. Encoding of unfamiliar cat stimuli might explain N290 amplification found in infants.

Neural Processing Differences of Facial Emotions Between Human and Vehicles: Evidence From an Event-Related Potential Study

Vehicle “faces” are a crucial factor influencing consumer intention to purchase gasoline and electric vehicles. However, little empirical evidence has demonstrated whether people process a vehicle’s face similarly to a human’s face. We investigated the neural processing relationship among human facial emotions and facial emotions of gasoline and electric vehicles using a 2 (emotional) × 3 (face type) repeated measures design and electroencephalograph (EEG) recordings. The results showed that human faces appear to share a partly similar neural processing mechanism in the latency of 100–300 ms, and that both human and vehicle faces elicited the ERP components N170, EPN, and P2. The large EPN and P2 suggest that gasoline vehicle facial emotions can be perceived more efficiently than those of electric vehicles. These findings provide an insight for vehicle designers to better understand the facial emotions presented by cars.

Investigating the Single Trial Detectability of Cognitive Face Processing by a Passive Brain-Computer Interface

An automated recognition of faces enables machines to visually identify a person and to gain access to non-verbal communication, including mimicry. Different approaches in lab settings or controlled realistic environments provided evidence that automated face detection and recognition can work in principle, although applications in complex real-world scenarios pose a different kind of problem that could not be solved yet. Specifically, in autonomous driving-it would be beneficial if the car could identify non-verbal communication of pedestrians or other drivers, as it is a common way of communication in daily traffic. Automated identification from observation whether pedestrians or other drivers communicate through subtle cues in mimicry is an unsolved problem so far, as intent and other cognitive factors are hard to derive from observation. In contrast, communicating persons usually have clear understanding whether they communicate or not, and such information is represented in their mindsets. This work investigates whether the mental processing of faces can be identified through means of a Passive Brain-Computer Interface (pBCI). This then could be used to support the cars' autonomous interpretation of facial mimicry of pedestrians to identify non-verbal communication. Furthermore, the attentive driver can be utilized as a sensor to improve the context awareness of the car in partly automated driving. This work presents a laboratory study in which a pBCI is calibrated to detect responses of the fusiform gyrus in the electroencephalogram (EEG), reflecting face recognition. Participants were shown pictures from three different categories: faces, abstracts, and houses evoking different responses used to calibrate the pBCI. The resulting classifier could distinguish responses to faces from that evoked by other stimuli with accuracy above 70%, in a single trial. Further analysis of the classification approach and the underlying data identified activation patterns in the EEG that corresponds to face recognition in the fusiform gyrus. The resulting pBCI approach is promising as it shows better-than-random accuracy and is based on relevant and intended brain responses. Future research has to investigate whether it can be transferred from the laboratory to the real world and how it can be implemented into artificial intelligences, as used in autonomous driving.

The Processing of Visual Signals in Major Depressive Disorder

The balanced processing of the internal mental world and the external world is a crucial aspect of everyday well-being. An extensive control of the internal emotional and cognitive world that often results in an internal expression of distress is a common feature of internalizing disorders. However, how depression affects the processing of the external world is still an open question. We, therefore, tested the processing of visual signals in major depressive disorder (MDD). To this end, we recorded the electroencephalogram of 38 MDD patients and 38 controls, while they performed a response-choice task with informative feedback and a passive viewing task. MDD patients differed significantly from controls in the early information processing of visual stimuli. The vertex positive potential (VPP) evoked by feedback in the response-choice task and pictures in the passive viewing task were smaller in MDD patients than in controls. This outcome suggests that depression might subtract attentional resources from external signal processing, with potential consequences in various cognitive domains.

Is human face recognition lateralized to the right hemisphere due to neural competition with left-lateralized visual word recognition? A critical review

The right hemispheric lateralization of face recognition, which is well documented and appears to be specific to the human species, remains a scientific mystery. According to a long-standing view, the evolution of language, which is typically substantiated in the left hemisphere, competes with the cortical space in that hemisphere available for visuospatial processes, including face recognition. Over the last decade, a specific hypothesis derived from this view according to which neural competition in the left ventral occipito-temporal cortex with selective representations of letter strings causes right hemispheric lateralization of face recognition, has generated considerable interest and research in the scientific community. Here, a systematic review of studies performed in various populations (infants, children, literate and illiterate adults, left-handed adults) and methodologies (behavior, lesion studies, (intra)electroencephalography, neuroimaging) offers little if any support for this reading lateralized neural competition hypothesis. Specifically, right-lateralized face-selective neural activity already emerges at a few months of age, well before reading acquisition. Moreover, consistent evidence of face recognition performance and its right hemispheric lateralization being modulated by literacy level during development or at adulthood is lacking. Given the absence of solid alternative hypotheses and the key role of neural competition in the sensory-motor cortices for selectivity of representations, learning, and plasticity, a revised language-related neural competition hypothesis for the right hemispheric lateralization of face recognition should be further explored in future research, albeit with substantial conceptual clarification and advances in methodological rigor.

Comparison of Scalp ERP to Faces in Macaques and Humans

Face recognition is an essential activity of social living, common to many primate species. Underlying processes in the brain have been investigated using various techniques and compared between species. Functional imaging studies have shown face-selective cortical regions and their degree of correspondence across species. However, the temporal dynamics of face processing, particularly processing speed, are likely different between them. Across sensory modalities activation of primary sensory cortices in macaque monkeys occurs at about 3/5 the latency of corresponding activation in humans, though this human simian difference may diminish or disappear in higher cortical regions. We recorded scalp event-related potentials (ERPs) to presentation of faces in macaques and estimated the peak latency of ERP components. Comparisons of latencies between macaques (112 ms) and humans (192 ms) suggested that the 3:5 ratio could be preserved in higher cognitive regions of face processing between those species.

The N170 is Sensitive to Long-term (Personal) Familiarity of a Face Identity

The N170 is a large deflection of the human electroencephalogram (EEG), peaking at about 170 milliseconds over the occipito-temporal cortex after the sudden onset of a face stimulus. The N170 reflects perceptual awareness of a face and its onset corresponds to the emergence of reliable face-selectivity in the human brain. However, whether sensitivity to the long-term familiarity of a face identity emerges already at this early time-point remains debated. Here we provide a brief survey of the 45 published studies comparing the N170 response to unfamiliar and familiar (famous, experimentally familiarized, personally familiar and own) faces. Even though effects of familiarity on the N170 are relatively small and inconsistent across studies, this overview indicates that face familiarity significantly increases the N170 amplitude. This effect is especially present for personally familiar faces, learned in natural conditions. In the human brain, effects linked to familiarity with specific facial identities therefore appear to emerge between 150 and 200 ms in occipito-temporal brain regions, i.e., shortly after the onset of face-selectivity but at the same time as the earliest high-level effects of immediate unfamiliar face identity repetition. This observation challenges standard neurocognitive models with a clear-cut distinction between perceptual and memory stages in human face recognition.

Flexing Gender Perception: Brain Potentials Reveal the Cognitive Permeability of Grammatical Information

A growing body of recent research suggests that verbal categories, particularly labels, impact categorization and perception. These findings are commonly interpreted as demonstrating the involvement of language on cognition; however, whether these assumptions hold true for grammatical structures has yet to be investigated. In the present study, we investigated the extent to which linguistic information, namely, grammatical gender categories, structures cognition to subsequently influence categorical judgments and perception. In a nonverbal categorization task, French-English bilinguals and monolingual English speakers made gender-associated judgments about a set of image pairs while event-related potentials were recorded. The image sets were composed of an object paired with either a female or male face, wherein the object was manipulated for their conceptual gender relatedness and grammatical gender congruency to the sex of the following target face. The results showed that grammatical gender modulated the N1 and P2/VPP, as well as the N300 exclusively for the French-English bilinguals, indicating the inclusion of language in the mechanisms associated with attentional bias and categorization. In contrast, conceptual gender information impacted the monolingual English speakers in the later N300 time window given the absence of a comparable grammatical feature. Such effects of grammatical categories in the early perceptual stream have not been found before, and further provide grounds to suggest that language shapes perception.

Lack of awareness despite complex visual processing: Evidence from event-related potentials in a case of selective metamorphopsia

Visual awareness is thought to result from integration of low- and high-level processing; instances of integration failure provide a crucial window into the cognitive and neural bases of awareness. We present neurophysiological evidence of complex cognitive processing in the absence of awareness, raising questions about the conditions necessary for visual awareness. We describe an individual with a neurodegenerative disease who exhibits impaired visual awareness for the digits 2 to 9, and stimuli presented in close proximity to these digits, due to perceptual distortion. We identified robust event-related potential responses indicating 1) face detection with the N170 component and 2) task-dependent target-word detection with the P3b component, despite no awareness of the presence of faces or target words. These data force us to reconsider the relationship between neural processing and visual awareness; even stimuli processed by a workspace-like cognitive system can remain inaccessible to awareness. We discuss how this finding challenges and constrains theories of visual awareness.

The altered early components and the decisive later process underlying attention bias modification in social anxiety: evidence from event-related potentials

Attention bias modification (ABM) is a potential intervention in relieving social anxiety symptoms, while its underlying neural mechanisms are not yet understood. The current study included 63 college students with social anxiety. Participants were assigned to the attention modification program (AMP, n = 20), the attention control condition (ACC, n = 20) and the passive waiting group (PW, n = 23). Questionnaires and the emotional Stroop task with EEG recordings were used to assess whether and how the 4-week ABM period affected emotional symptoms and specific emotional processing. Results showed that the two training groups (AMP and ACC) produced comparable emotional improvements and both showed a decrease in negative bias compared with the PW group. The ERP results indicated that despite no significant ERP changes in the PW group, the ACC group exhibited a greater N1, whereas the AMP group exhibited a reduced VPP at the post-test stage compared to the pre-test stage. Besides, both training groups showed a similar late positive potential (LPP) reduction. Notably, the reduction in LPP was positively correlated with behavioral and symptom improvement. Thus, manipulations unique to ABM (face-target contingency) primarily modulate the early attention distribution of material-related stimuli. However, the clinical benefits of attention training may be due to later cognitive-affective mechanisms.

Treatment-related changes towards normalization of the abnormal external signal processing in panic disorder

Despite the scientific consensus on the efficacy of psychotherapy for the treatment of psychological disorders, the evidence of treatment-related changes towards normalization of abnormal brain functions in patients is mixed. In the present experiment, we investigated whether treatment can affect early information processing, by testing abnormal event-related potentials (ERPs) evoked by internal and external signals in panic disorder. Sixteen patients with panic disorder and comorbid personality disorder and sixteen control participants performed a response-choice task and a passive viewing task in two testing sessions, separated by around 14 months. During this period, patients received psychological treatment. In agreement with previous studies of performance monitoring, the abnormal amplitude of the Ne/ERN–an index of error processing based on internal signals—did not change between the first and second testing session. However, treatment-related changes were evident for the abnormal vertex positive potential (VPP) evoked by external signals in the response-choice task and the passive viewing task. In patients, the VPP was smaller in the second session compared to the first session, whereas no significant changes occurred in controls. This result supplies evidence of treatment-related changes towards normalization in the early information processing of external visual stimuli in panic disorder.

Changes in face and face pareidolia processing in patients with migraine: an ERP study

Migraine is a multifactorial brain disorder characterized by recurrent disabling headache attacks. One of the possible mechanisms in the pathogenesis of migraine may be a decrease in inhibitory cortical stimuli in the primary visual cortex attributable to cortical hyperexcitability. The aim of this study was to investigate the neural correlates underlying face and face pareidolia processing in terms of the event-related potential (ERP) components, N170, vertex positive potential (VPP), and N250, in patients with migraine. In total, 40 patients with migraine without aura, 23 patients with migraine and aura, and 30 healthy controls were enrolled. We recorded ERPs during the presentation of face and face pareidolia images. N170, VPP, and N250 mean amplitudes and latencies were examined. N170 was significantly greater in patients with migraine with aura than in healthy controls. VPP amplitude was significantly greater in patients with migraine without aura than in healthy controls. The face stimuli evoked significantly earlier VPP responses to faces (168.7 ms, SE = 1.46) than pareidolias (173.4 ms, SE = 1.41) in patients with migraine with aura. We did not find a significant difference between N250 amplitude for face and face pareidolia processing. A significant difference was observed between the groups for pareidolia in terms of N170 [F(2,86) = 14,75, P < 0.001] and VPP [F(2,86) = 16.43, P < 0.001] amplitudes. Early ERPs are a valuable tool to study the neural processing of face processing in patients with migraine to demonstrate visual cortical hyperexcitability.NEW & NOTEWORTHY Event-related potentials (ERPs) are important for understanding face and face pareidolia processing in patients with migraine. N170, vertex positive potential (VPP), and N250 ERPs were investigated. N170 was revealed as a potential component of cortical excitability for face and face pareidolia processing in patients with migraine.

Electrophysiological evidence of altered facial expressions recognition in Alzheimer's disease: A comprehensive ERP study

OBJECTIVES

The present study aims to evaluate the amplitude and latency of event-related potentials (ERPs) P100, N170, VPP and N230 in individuals with Alzheimer's disease (AD) compared to healthy elderly controls, using a passive viewing task of emotional facial expressions.

METHODS

Twenty-four individuals with mild to moderate AD and 23 demographically matched healthy elderly controls were included in the study. ERP P100, N170, VPP and N230 amplitude and latency values were compared between groups.

RESULTS

The categorization of emotional facial expressions was intact; yet, increased P100 amplitude and latency, decreased N170 amplitude, and increased VPP amplitude were observed in AD compared to controls. Increased N230 amplitude and latency were observed in response to angry expressions, while neutral expressions elicited decreased amplitude and latency.

CONCLUSIONS

Increased P100 amplitude and latency may reflect reduced amygdala volume and disruptions in the visual system, while decreased N170 and increased VPP amplitudes may reflect impaired perceptual processing, mitigated by a greater involvement of prefrontal areas for task performance in AD.

SIGNIFICANCE

This study is the first to report a complex pattern of ERPs to emotional facial expressions in individuals with AD.

Internal and external signal processing in patients with panic disorder: An event-related potential (ERP) study

Self-absorption describes a pathological tendency towards the internal mental world (internalization) that often conflicts with the accurate monitoring of the external world. In performance monitoring, an augmented electrophysiological response evoked by internal signals in patients with anxiety or depressive disorder seems to reflect this tendency. Specifically, the error-related negativity (Ne/ERN), an index of error processing based on internal signals, is larger in patients compared to controls. In the present experiment, we investigated whether the preferential processing of internal signals in patients is linked to diminished and inflexible external signal processing. To this end, the electrophysiological response evoked by external signals was analysed in patients with panic disorder and healthy controls. Participants performed a choice-response task, where informative or uninformative feedback followed each response, and a passive viewing task. As a replication of previous studies, patients presented an augmented Ne/ERN, indexing enhanced processing of internal signals related to errors. Furthermore, the vertex positive potential (VPP) evoked by visual stimuli was larger in patients than in controls, suggesting enhanced attention to external signals. Moreover, patients and controls showed similar sensitivity to the feedback information content, indicating a normal flexibility in the allocation of monitoring resources to external signals depending on how informative these signals are for performance monitoring. These results suggest that the tendency towards internal signals in patients with panic disorder does not hinder the flexible processing of external signals. On the contrary, external signals seem to attract enhanced processing in patients compared to controls.

The inferior occipital gyrus is a major cortical source of the face-evoked N170: Evidence from simultaneous scalp and intracerebral human recordings

The sudden onset of a face image leads to a prominent face-selective response in human scalp electroencephalographic (EEG) recordings, peaking 170 ms after stimulus onset at occipito-temporal (OT) scalp sites: the N170 (or M170 in magnetoencephalography). According to a widely held view, the main cortical source of the N170 lies in the fusiform gyrus (FG), whereas the posteriorly located inferior occipital gyrus (IOG) would rather generate earlier face-selective responses. Here, we report neural responses to upright and inverted faces recorded in a unique patient using multicontact intracerebral electrodes implanted in the right IOG and in the OT sulcus above the right lateral FG (LFG). Simultaneous EEG recordings on the scalp identified the N170 over the right OT scalp region. The latency and amplitude of this scalp N170 were correlated at the single-trial level with the N170 recorded in the lateral IOG, close to the scalp lateral occipital surface. In addition, a positive component maximal around the latency of the N170 (a P170) was prominent above the internal LFG, whereas this region typically generates an N170 (or "N200") over its external/ventral surface. This suggests that electrophysiological responses in the LFG manifest as an equivalent dipole oriented mostly along the vertical axis with likely minimal projection to the lateral OT scalp region. Altogether, these observations provide evidence that the IOG is a major cortical generator of the face-selective scalp N170, qualifying the potential contribution of the FG and questioning a strict serial spatiotemporal organization of the human cortical face network.

Maturational trajectories of local and long-range functional connectivity in autism during face processing

Autism spectrum disorder (ASD) is characterized neurophysiologically by, among other things, functional connectivity abnormalities in the brain. Recent evidence suggests that the nature of these functional connectivity abnormalities might not be uniform throughout maturation. Comparing between adolescents and young adults (ages 14-21) with ASD and age- and IQ-matched typically developing (TD) individuals, we previously documented, using magnetoencephalography (MEG) data, that local functional connectivity in the fusiform face areas (FFA) and long-range functional connectivity between FFA and three higher order cortical areas were all reduced in ASD. Given the findings on abnormal maturation trajectories in ASD, we tested whether these results extend to preadolescent children (ages 7-13). We found that both local and long-range functional connectivity were in fact normal in this younger age group in ASD. Combining the two age groups, we found that local and long-range functional connectivity measures were positively correlated with age in TD, but negatively correlated with age in ASD. Last, we showed that local functional connectivity was the primary feature in predicting age in ASD group, but not in the TD group. Furthermore, local functional connectivity was only correlated with ASD severity in the older group. These results suggest that the direction of maturation of functional connectivity for processing of faces from childhood to young adulthood is itself abnormal in ASD, and that during the processing of faces, these trajectory abnormalities are more pronounced for local functional connectivity measures than they are for long-range functional connectivity measures.

Fear boosts the early neural coding of faces

The rapid extraction of facial identity and emotional expressions is critical for adapted social interactions. These biologically relevant abilities have been associated with early neural responses on the face sensitive N170 component. However, whether all facial expressions uniformly modulate the N170, and whether this effect occurs only when emotion categorization is task-relevant, is still unclear. To clarify this issue, we recorded high-resolution electrophysiological signals while 22 observers perceived the six basic expressions plus neutral. We used a repetition suppression paradigm, with an adaptor followed by a target face displaying the same identity and expression (trials of interest). We also included catch trials to which participants had to react, by varying identity (identity-task), expression (expression-task) or both (dual-task) on the target face. We extracted single-trial Repetition Suppression (stRS) responses using a data-driven spatiotemporal approach with a robust hierarchical linear model to isolate adaptation effects on the trials of interest. Regardless of the task, fear was the only expression modulating the N170, eliciting the strongest stRS responses. This observation was corroborated by distinct behavioral performance during the catch trials for this facial expression. Altogether, our data reinforce the view that fear elicits distinct neural processes in the brain, enhancing attention and facilitating the early coding of faces.

When appearance does not match accent: neural correlates of ethnicity-related expectancy violations

Most research on ethnicity in neuroscience and social psychology has focused on visual cues. However, accents are central social markers of ethnicity and strongly influence evaluations of others. Here, we examine how varying auditory (vocal accent) and visual (facial appearance) information about others affects neural correlates of ethnicity-related expectancy violations. Participants listened to standard German and Turkish-accented speakers and were subsequently presented with faces whose ethnic appearance was either congruent or incongruent to these voices. We expected that incongruent targets (e.g. German accent/Turkish face) would be paralleled by a more negative N2 event-related brain potential (ERP) component. Results confirmed this, suggesting that incongruence was related to more effortful processing of both Turkish and German target faces. These targets were also subjectively judged as surprising. Additionally, varying lateralization of ERP responses for Turkish and German faces suggests that the underlying neural generators differ, potentially reflecting different emotional reactions to these targets. Behavioral responses showed an effect of violated expectations: German-accented Turkish-looking targets were evaluated as most competent of all targets. We suggest that bringing together neural and behavioral measures of expectancy violations, and using both visual and auditory information, yields a more complete picture of the processes underlying impression formation.

Processing of Internal and External Signals for Performance Monitoring in the Context of Emotional Faces

Performance monitoring can be based on internal or external signals. We recorded event-related potentials (ERPs) to investigate whether relating performance to external signals affects internal performance monitoring. Thirty participants performed a task in which responses were followed by faces whose expressions were partially contingent upon performance. Instructions given to half of the participants mentioned a link between task performance and the upcoming face expression. Instructed participants showed smaller error-related negativity (Ne/ERN) to erroneous responses and larger N170 to faces as compared to participants in the not-instructed group. In addition, we observed a correlation between ΔNe/ERN and P1-latency benefit for angry faces after errors. Taken together, processing of internally generated signals for performance monitoring is reduced by instructions referring to an emotional face. Furthermore, we relate the correlation between the magnitude of internal monitoring and facilitation in processing angry faces to priming induced by the negative affective meaning of errors.

Mental imagery of face enhances face-sensitive event-related potentials to ambiguous visual stimuli

Visual mental imagery forms mental representations of visual objects when correspondent stimuli are absent and shares some characters with visual perception. Both the vertex-positive-potential (VPP) and N170 components of event-related potentials (ERPs) to visual stimuli have a remarkable preference to faces. This study investigated whether visual mental imagery modulates the face-sensitive VPP and/or N170 components. The results showed that with significantly larger amplitudes under the face-imagery condition than the house-imagery condition, the VPP and P2 responses, but not the N170 component, were elicited by phase-randomized ambiguous stimuli. Thus, the brain substrates underlying VPP are not completely identical to those underlying N170, and the VPP/P2 manifestation of the category selectivity in imagery probably reflects an integration of top-down mental imagery signals (from the prefrontal cortex) and bottom-up perception signals (from the early visual cortex) in the occipito-temporal cortex where VPP and P2 originate.

Anticipatory representations of reward and threat in perceptual areas from preadolescence to late adolescence

This study examined whether changes in perceptual processes can partially account for the increase in reward-orientated behaviour during adolescence. This was investigated by examining reinforcement-dependent potentiation to discriminative stimuli (S^D) that predicted rewarding or threatening outcomes. To that end, perceptual event-related potentials that are modulated by motivationally salient stimuli, the N170 and Late Positive Potential (LPP), were recorded from 30 preadolescents (9–12 years), 30 adolescents (13–17 years), and 34 late adolescents (18–23 years) while they completed an instrumental task in which they emitted or omitted a motor response to obtain rewards and avoid losses. The LPP, but not the N170, showed age, but not gender, differences in reinforcement-dependent potentiation; preadolescents, adolescents, and late adolescents showed potentiation to S^D that predicted a threat, whereas only preadolescents showed potentiation to S^D that predicted a reward. Notably, the magnitude of threat-related LPP reinforcement-dependent potentiation decreased during the course of adolescence. In addition, greater sensation seeking was associated with greater LPP amplitudes in preadolescent males, but smaller LPP amplitudes in late adolescent males. Critically, these findings provide initial evidence for developmental differences in value-related coding in perceptual areas, where adolescents show greater perceptual biases to avoidance-related cues than to reward-related cues.

Face-specific and domain-general visual processing deficits in children with developmental prosopagnosia

Evidence suggests that face and object recognition depend on distinct neural circuitry within the visual system. Work with adults with developmental prosopagnosia (DP) demonstrates that some individuals have preserved object recognition despite severe face recognition deficits. This face selectivity in adults with DP indicates that face- and object-processing systems can develop independently, but it is unclear at what point in development these mechanisms are separable. Determining when individuals with DP first show dissociations between faces and objects is one means to address this question. In the current study, we investigated face and object processing in six children with DP (5-12-years-old). Each child was assessed with one face perception test, two different face memory tests, and two object memory tests that were matched to the face memory tests in format and difficulty. Scores from the DP children on the matched face and object tasks were compared to within-subject data from age-matched controls. Four of the six DP children, including the 5-year-old, showed evidence of face-specific deficits, while one child appeared to have more general visual-processing deficits. The remaining child had inconsistent results. The presence of face-specific deficits in children with DP suggests that face and object perception depend on dissociable processes in childhood.

Electrophysiological Anomalies in Face-Name Memory Encoding in Young Binge Drinkers

A growing body of evidence indicates that the intake of large amounts of alcohol during one session may have structural and functional effects on the still-maturing brains of young people. These effects are particularly pronounced in prefrontal and hippocampal regions, which appear to be especially sensitive to the neurotoxic effects of alcohol. However, to date, few studies have used the event-related potentials (ERPs) technique to analyze the relationship between binge drinking (BD) and associative memory. The objective of this study was to examine brain activity during memory encoding using the Subsequent memory paradigm in subjects who have followed a BD pattern of alcohol consumption for at least 2 years. A total of 50 undergraduate students (mean age = 20.6 years), i.e., 25 controls (12 females) and 25 binge drinkers (BDs; 11 females), with no personal or family history of alcoholism or psychopathological disorders, performed a visual face-name association memory task. The task used enables assessment of the Difference due to memory effect (Dm), a measure of memory encoding based on comparison of the neural activity associated with subsequent successful and unsuccessful retrieval. In ERP studies, study items that are subsequently remembered elicit larger positive amplitudes at midline parieto-frontal sites than those items that are subsequently forgotten. The Dm effect generally appears in the latency range of about 300-800 ms. The results showed a Dm effect in posterior regions in the 350-650 ms latency range in the Control group. However, in the BD group, no significant differences were observed in the electrophysiological brain activity between remembered and forgotten items during the encoding process. No differences between groups were found in behavioral performance. These findings show that young BDs display abnormal pattern of ERP brain activity during the encoding phase of a visual face-name association task, possibly suggesting a different neural signature of successful memory encoding.

Facial identity and emotional expression as predictors during economic decisions

Two sources of information most relevant to guide social decision making are the cooperative tendencies associated with different people and their facial emotional displays. This electrophysiological experiment aimed to study how the use of personal identity and emotional expressions as cues impacts different stages of face processing and their potential isolated or interactive processing. Participants played a modified trust game with 8 different alleged partners, and in separate blocks either the identity or the emotions carried information regarding potential trial outcomes (win or loss). Behaviorally, participants were faster to make decisions based on identity compared to emotional expressions. Also, ignored (nonpredictive) emotions interfered with decisions based on identity in trials where these sources of information conflicted. Electrophysiological results showed that expectations based on emotions modulated processing earlier in time than those based on identity. Whereas emotion modulated the central N1 and VPP potentials, identity judgments heightened the amplitude of the N2 and P3b. In addition, the conflict that ignored emotions generated was reflected on the N170 and P3b potentials. Overall, our results indicate that using identity or emotional cues to predict cooperation tendencies recruits dissociable neural circuits from an early point in time, and that both sources of information generate early and late interactive patterns.

Optimizing the Face Paradigm of BCI System by Modified Mismatch Negative Paradigm

Many recent studies have focused on improving the performance of event-related potential (ERP) based brain computer interfaces (BCIs). The use of a face pattern has been shown to obtain high classification accuracies and information transfer rates (ITRs) by evoking discriminative ERPs (N200 and N400) in addition to P300 potentials. Recently, it has been proved that the performance of traditional P300-based BCIs could be improved through a modification of the mismatch pattern. In this paper, a mismatch inverted face pattern (MIF-pattern) was presented to improve the performance of the inverted face pattern (IF-pattern), one of the state of the art patterns used in visual-based BCI systems. Ten subjects attended in this experiment. The result showed that the mismatch inverted face pattern could evoke significantly larger vertex positive potentials (p < 0.05) and N400s (p < 0.05) compared to the inverted face pattern. The classification accuracy (mean accuracy is 99.58%) and ITRs (mean bit rate is 27.88 bit/min) of the mismatch inverted face pattern was significantly higher than that of the inverted face pattern (p < 0.05).

Tracking the Timecourse of Social Perception: The Effects of Racial Cues on Event-Related Brain Potentials

Event-related potentials were used to track social perception processes associated with viewing faces of racial ingroup and outgroup members. Activity associated with three distinct processes was detected. First, peaking at approximately 170 ms, faces were distinguished from nonface stimuli. Second, peaking at approximately 250 ms, ingroup members were differentiated from outgroup members, with a larger component suggesting greater attention to ingroup members. This effect may reflect the spontaneous application of a deeper level of processing to ingroup members. Third, peaking at approximately 520 ms, evaluative differentiation of ingroup and outgroup members occurred, with greater ingroup bias displayed by those with higher levels of prejudice on an explicit measure. Together, the results demonstrate the promise of using neural processes to track the presence, timing, and degree of activation of components relevant to social perception, prejudice, and stereotyping.

Prior Knowledge about Display Inversion in Biological Motion Perception

Display inversion severely impedes veridical perception of point-light biological motion (Pavlova and Sokolov, 2000 Perception & Psychophysics62 889–899; Sumi, 1984 Perception13 283–286). Here, by using a spontaneous-recognition paradigm, we ask whether prior information about display orientation improves biological motion perception. Participants were shown a set of 180° inverted point-light stimuli depicting a human walker and quadrupeds (dogs). In experiment 1, one group of observers was not aware of the orientation of stimuli, whereas the other group was told beforehand that stimuli will be presented upside down. In experiment 2, independent groups of participants informed about stimulus orientation saw the same set of stimuli, in each of which either a moving or a static background line was inserted. The findings indicate that information about display inversion is insufficient for reliable recognition of inverted point-light biological motion. Instead, prior information facilitates display recognition only when it is complemented by additional contextual elements. It appears that visual impressions from inverted point-light stimuli remain impenetrable with respect to one's knowledge about display orientation. The origins of orientation specificity in biological motion perception are discussed in relation to the recent neuroimaging data obtained with point-light stimuli and fragmented Mooney faces.

Women are better at seeing faces where there are none: an ERP study of face pareidolia

Event-related potentials (ERPs) were recorded in 26 right-handed students while they detected pictures of animals intermixed with those of familiar objects, faces and faces-in-things (FITs). The face-specific N170 ERP component over the right hemisphere was larger in response to faces and FITs than to objects. The vertex positive potential (VPP) showed a difference in FIT encoding processes between males and females at frontal sites; while for men, the FIT stimuli elicited a VPP of intermediate amplitude (between that for faces and objects), for women, there was no difference in VPP responses to faces or FITs, suggesting a marked anthropomorphization of objects in women. SwLORETA source reconstructions carried out to estimate the intracortical generators of ERPs in the 150-190 ms time window showed how, in the female brain, FIT perception was associated with the activation of brain areas involved in the affective processing of faces (right STS, BA22; posterior cingulate cortex, BA22; and orbitofrontal cortex, BA10) in addition to regions linked to shape processing (left cuneus, BA18/30). Conversely, in the men, the activation of occipito/parietal regions was prevalent, with a considerably smaller activation of BA10. The data suggest that the female brain is more inclined to anthropomorphize perfectly real objects compared to the male brain.

Dynamic Functional Brain Connectivity for Face Perception

Face perception is mediated by a distributed brain network comprised of the core system at occipito-temporal areas and the extended system at other relevant brain areas involving bilateral hemispheres. In this study we explored how the brain connectivity changes over the time for face-sensitive processing. We investigated the dynamic functional connectivity in face perception by analyzing time-dependent EEG phase synchronization in four different frequency bands: theta (4-7 Hz), alpha (8-14 Hz), beta (15-24 Hz), and gamma (25-45 Hz) bands in the early stages of face processing from 30 to 300 ms. High-density EEG were recorded from subjects who were passively viewing faces, buildings, and chairs. The dynamic connectivity within the core system and between the extended system were investigated. Significant differences between faces and non-faces mainly appear in theta band connectivity: (1) at the time segment of 90-120 ms between parietal area and occipito-temporal area in the right hemisphere, and (2) at the time segment of 150-180 ms between bilateral occipito-temporal areas. These results indicate (1) the importance of theta-band connectivity in the face-sensitive processing, and (2) that different parts of network are involved for the initial stage of face categorization and the stage of face structural encoding.

Perceptual face processing in developmental prosopagnosia is not sensitive to the canonical location of face parts

Individuals with developmental prosopagnosia (DP) are strongly impaired in recognizing faces, but it is controversial whether this deficit is linked to atypical visual-perceptual face processing mechanisms. Previous behavioural studies have suggested that face perception in DP might be less sensitive to the canonical spatial configuration of face parts in upright faces. To test this prediction, we recorded event-related brain potentials (ERPs) to intact upright faces and to faces with spatially scrambled parts (eyes, nose, and mouth) in a group of ten participants with DP and a group of ten age-matched control participants with normal face recognition abilities. The face-sensitive N170 component and the vertex positive potential (VPP) were both enhanced and delayed for scrambled as compared to intact faces in the control group. In contrast, N170 and VPP amplitude enhancements to scrambled faces were absent in the DP group. For control participants, the N170 to scrambled faces was also sensitive to feature locations, with larger and delayed N170 components contralateral to the side where all features appeared in a non-canonical position. No such differences were present in the DP group. These findings suggest that spatial templates of the prototypical feature locations within an upright face are selectively impaired in DP.

A new Mooney test

Since its introduction in 1957, the Mooney test has continued to see active use in studies of visual perception, in studies using brain imaging, and in clinical research. Mooney's original version is of limited length, however, and was designed to be administered by time-consuming personal interview. We have developed a new, extended version of the Mooney test that is suitable for online testing and for use in a test-retest paradigm. The Mooney-Verhallen Test (MVT) comprises 144 trials, takes on average less than 10 min to complete, and has a Spearman-Brown-corrected test-retest reliability of ρ = .89. We outline our methods for developing the stimuli and for selecting the final stimulus set, and we present the results from two rounds of testing on two independent samples of 374 participants and 505 participants, respectively. The test is freely available for scientific use.

The 170ms Response to Faces as Measured by MEG (M170) Is Consistently Altered in Congenital Prosopagnosia

Modularity of face processing is still a controversial issue. Congenital prosopagnosia (cPA), a selective and lifelong impairment in familiar face recognition without evidence of an acquired cerebral lesion, offers a unique opportunity to support this fundamental hypothesis. However, in spite of the pronounced behavioural impairment, identification of a functionally relevant neural alteration in congenital prosopagnosia by electrophysiogical methods has not been achieved so far. Here we show that persons with congenital prosopagnosia can be distinguished as a group from unimpaired persons using magnetoencephalography. Early face-selective MEG-responses in the range of 140 to 200ms (the M170) showed prolonged latency and decreased amplitude whereas responses to another category (houses) were indistinguishable between subjects with congenital prosopagnosia and unimpaired controls. Latency and amplitude of face-selective EEG responses (the N170) which were simultaneously recorded were statistically indistinguishable between subjects with cPA and healthy controls which resolves heterogeneous and partly conflicting results from existing studies. The complementary analysis of categorical differences (evoked activity to faces minus evoked activity to houses) revealed that the early part of the 170ms response to faces is altered in subjects with cPA. This finding can be adequately explained in a common framework of holistic and part-based face processing. Whereas a significant brain-behaviour correlation of face recognition performance and the size of the M170 amplitude is found in controls a corresponding correlation is not seen in subjects with cPA. This indicates functional relevance of the alteration found for the 170ms response to faces in cPA and pinpoints the impairment of face processing to early perceptual stages.

A Representational Similarity Analysis of the Dynamics of Object Processing Using Single-Trial EEG Classification

The recognition of object categories is effortlessly accomplished in everyday life, yet its neural underpinnings remain not fully understood. In this electroencephalography (EEG) study, we used single-trial classification to perform a Representational Similarity Analysis (RSA) of categorical representation of objects in human visual cortex. Brain responses were recorded while participants viewed a set of 72 photographs of objects with a planned category structure. The Representational Dissimilarity Matrix (RDM) used for RSA was derived from confusions of a linear classifier operating on single EEG trials. In contrast to past studies, which used pairwise correlation or classification to derive the RDM, we used confusion matrices from multi-class classifications, which provided novel self-similarity measures that were used to derive the overall size of the representational space. We additionally performed classifications on subsets of the brain response in order to identify spatial and temporal EEG components that best discriminated object categories and exemplars. Results from category-level classifications revealed that brain responses to images of human faces formed the most distinct category, while responses to images from the two inanimate categories formed a single category cluster. Exemplar-level classifications produced a broadly similar category structure, as well as sub-clusters corresponding to natural language categories. Spatiotemporal components of the brain response that differentiated exemplars within a category were found to differ from those implicated in differentiating between categories. Our results show that a classification approach can be successfully applied to single-trial scalp-recorded EEG to recover fine-grained object category structure, as well as to identify interpretable spatiotemporal components underlying object processing. Finally, object category can be decoded from purely temporal information recorded at single electrodes.

Interactions between the perception of age and ethnicity in faces: an event-related potential study

Face perception models propose that different facial attributes are processed by anatomically distinct neural pathways that partially overlap. Whether these attributes interact functionally is an open question. Our goal was to determine if there are interactions between age and ethnicity processing and, if so, at what temporal epoch these interactions are evident. We monitored event-related potentials on electroencephalography while subjects categorized faces by age or ethnicity in two conditions: a baseline in which the other of these two properties not being categorized was held constant and an interference condition in which it also varied, as modelled after the Garner interference paradigm. We found that, when participants were categorizing faces by age, variations in ethnicity increased the amplitude of the right face-selective N170 component. When subjects were categorizing faces by ethnicity, variations in age did not alter the N170. We concluded that there is an asymmetric pattern of influence between age and ethnicity on early face-specific stages of visual processing, which has parallels with behavioural evidence of asymmetric interactions between identity and expression processing of faces.

Brain Signals of Face Processing as Revealed by Event-Related Potentials

We analyze the functional significance of different event-related potentials (ERPs) as electrophysiological indices of face perception and face recognition, according to cognitive and neurofunctional models of face processing. Initially, the processing of faces seems to be supported by early extrastriate occipital cortices and revealed by modulations of the occipital P1. This early response is thought to reflect the detection of certain primary structural aspects indicating the presence grosso modo of a face within the visual field. The posterior-temporal N170 is more sensitive to the detection of faces as complex-structured stimuli and, therefore, to the presence of its distinctive organizational characteristics prior to within-category identification. In turn, the relatively late and probably more rostrally generated N250r and N400-like responses might respectively indicate processes of access and retrieval of face-related information, which is stored in long-term memory (LTM). New methods of analysis of electrophysiological and neuroanatomical data, namely, dynamic causal modeling, single-trial and time-frequency analyses, are highly recommended to advance in the knowledge of those brain mechanisms concerning face processing.

Rapid P300 brain-computer interface communication with a head-mounted display

Visual ERP (P300) based brain-computer interfaces (BCIs) allow for fast and reliable spelling and are intended as a muscle-independent communication channel for people with severe paralysis. However, they require the presentation of visual stimuli in the field of view of the user. A head-mounted display could allow convenient presentation of visual stimuli in situations, where mounting a conventional monitor might be difficult or not feasible (e.g., at a patient's bedside). To explore if similar accuracies can be achieved with a virtual reality (VR) headset compared to a conventional flat screen monitor, we conducted an experiment with 18 healthy participants. We also evaluated it with a person in the locked-in state (LIS) to verify that usage of the headset is possible for a severely paralyzed person. Healthy participants performed online spelling with three different display methods. In one condition a 5 × 5 letter matrix was presented on a conventional 22 inch TFT monitor. Two configurations of the VR headset were tested. In the first (glasses A), the same 5 × 5 matrix filled the field of view of the user. In the second (glasses B), single letters of the matrix filled the field of view of the user. The participant in the LIS tested the VR headset on three different occasions (glasses A condition only). For healthy participants, average online spelling accuracies were 94% (15.5 bits/min) using three flash sequences for spelling with the monitor and glasses A and 96% (16.2 bits/min) with glasses B. In one session, the participant in the LIS reached an online spelling accuracy of 100% (10 bits/min) using the glasses A condition. We also demonstrated that spelling with one flash sequence is possible with the VR headset for healthy users (mean: 32.1 bits/min, maximum reached by one user: 71.89 bits/min at 100% accuracy). We conclude that the VR headset allows for rapid P300 BCI communication in healthy users and may be a suitable display option for severely paralyzed persons.

Objective measurement of visual resolution using the P300 to self-facial images

OBJECTIVE

To assess visual acuity objectively "beyond V1", the P300 event-related potential is a promising candidate and closely associated with conscious perception. However, the P300 can be willfully modulated, a disadvantage for objective visual acuity estimation. Faces are very salient stimuli and difficult to ignore. Here, we present a P300-type paradigm to assess visual acuity with faces.

METHODS

Gray-scale portraits of the respective subject served as oddball stimuli (probability 1/7), scrambled versions of these as the standard stimuli (probability 6/7). Furthermore, stimuli were spatially high-pass filtered (at 0, 2.2, 4.2 and 8.3 cpd), making them recognizable only with sufficient acuity. Acuity was systematically reduced by dioptric blur, chosen individually to render faces unrecognizable when high-passed at ≥ 4.2 cpd. EEG was recorded from 11 subjects at 32 scalp positions and re-referenced to the average of TP9 and TP10. One of the rare face variants was designated as target, for which a button had to be pressed.

RESULTS

The event-related potential was dominated by the P300 at 300-800 ms. All subjects had a significant (P < 0.05) P300 for 0- to 8.3-cpd filtering. When vision was blurred, the fraction of significant P300 responses to 8.3-cpd filtered faces dropped to 18%, but stayed at 100% for 4.2 cpd. Another component, the vertex positive potential (VPP) at 170 ms, was undetectable in most participants with blur and all levels of filtering, even when the images were recognizable.

CONCLUSIONS

The study demonstrates the feasibility of a face-based P300 approach to objectively assess visual acuity. The sensitivity to stimulus degradation was comparable to that of a grating-based approach as previously reported. An unexpected finding was the differing behavior of the P300 and the VPP. The VPP was quite sensitive to high-pass filtering, while the P300 sustained stronger filtering, although for its generation, the faces must also be discriminated from scrambled faces.