The N250 event-related potential as an index of face familiarity: a replication study

Asymmetries in event-related potentials part 1: A systematic review of face processing studies

The human brain shows distinct lateralized activation patterns for a range of cognitive processes. One such function, which is thought to be lateralized to the right hemisphere (RH), is human face processing. Its importance for social communication and interaction has led to a plethora of studies investigating face processing in health and disease. Temporally highly resolved methods, like event-related potentials (ERPs), allow for a detailed characterization of different processing stages and their specific lateralization patterns. This systematic review aimed at disentangling some of the contradictory findings regarding the RH specialization in face processing focusing on ERP research in healthy participants. Two databases were searched for studies that investigated left and right electrodes while participants viewed (mostly neutral) facial stimuli. The included studies used a variety of different tasks, which ranged from passive viewing to memorizing faces. The final data selection highlights, that strongest lateralization to the RH was found for the N170, especially for right-handed young male participants. Left-handed, female, and older participants showed less consistent lateralization patterns. Other ERP components like the P1, P2, N2, P3, and the N400 were overall less clearly lateralized. The current review highlights that many of the assumed lateralization patterns are less clear than previously thought and that the variety of stimuli, tasks, and EEG setups used, might contribute to the ambiguous findings.

Shared neural codes of recognition memory

Recognition memory research has identified several electrophysiological indicators of successful memory retrieval, known as old-new effects. These effects have been observed in different sensory domains using various stimulus types, but little attention has been given to their similarity or distinctiveness and the underlying processes they may share. Here, a data-driven approach was taken to investigate the temporal evolution of shared information content between different memory conditions using openly available EEG data from healthy human participants of both sexes, taken from six experiments. A test dataset involving personally highly familiar and unfamiliar faces was used. The results show that neural signals of recognition memory for face stimuli were highly generalized starting from around 200 ms following stimulus onset. When training was performed on non-face datasets, an early (around 200-300 ms) to late (post-400 ms) differentiation was observed over most regions of interest. Successful cross-classification for non-face stimuli (music and object/scene associations) was most pronounced in late period. Additionally, a striking dissociation was observed between familiar and remembered objects, with shared signals present only in the late window for correctly remembered objects, while cross-classification for familiar objects was successful in the early period as well. These findings suggest that late neural signals of memory retrieval generalize across sensory modalities and stimulus types, and the dissociation between familiar and remembered objects may provide insight into the underlying processes.

P1, N170, and N250 Event-related Potential Components Reflect Temporal Perception Processing in Face and Body Personal Identification

Human faces and bodies represent various socially important signals. Although adults encounter numerous new people in daily life, they can recognize hundreds to thousands of different individuals. However, the neural mechanisms that differentiate one person from another person are unclear. This study aimed to clarify the temporal dynamics of the cognitive processes of face and body personal identification using face-sensitive ERP components (P1, N170, and N250). The present study performed three blocks (face-face, face-body, and body-body) of different ERP adaptation paradigms. Furthermore, in the above three blocks, ERP components were used to compare brain biomarkers under three conditions (same person, different person of the same sex, and different person of the opposite sex). The results showed that the P1 amplitude for the face-face block was significantly greater than that for the body-body block, that the N170 amplitude for a different person of the same sex condition was greater than that for the same person condition in the right hemisphere only, and that the N250 amplitude gradually increased as the degree of face and body sex-social categorization grew closer (i.e., same person condition > different person of the same sex condition > different person of the opposite sex condition). These results suggest that early processing of the face and body processes the face and body separately and that structural encoding and personal identification of the face and body process the face and body collaboratively.

Effects of the presentation order of stimulations in sequential ERP/SSVEP Hybrid Brain-Computer Interface

Hybrid Brain-Computer Interface (hBCI) combines multiple neurophysiology modalities or paradigms to speed up the output of a single command or produce multiple ones simultaneously. Concurrent hBCIs that employ endogenous and exogenous paradigms are limited by the reduced set of possible commands. Conversely, the fusion of different exogenous visual evoked potentials demonstrated impressive performances; however, they suffer from limited portability. Yet, sequential hBCIs did not receive much attention mainly due to slower transfer rate and user fatigue during prolonged BCI use (Lorenz et al 2014 J. Neural Eng. 11 035007). Moreover, the crucial factors for optimizing the hybridization remain under-explored. In this paper, we test the feasibility of sequential Event Related-Potentials (ERP) and Steady-State Visual Evoked Potentials (SSVEP) hBCI and study the effect of stimulus order presentation between ERP-SSVEP and SSVEP-ERP for the control of directions and speed of powered wheelchairs or mobile robots with 15 commands. Exploiting the fast single trial face stimulus ERP, SSVEP and modern efficient convolutional neural networks, the configuration with SSVEP presented at first achieved significantly (p < 0.05) higher average accuracy rate with 76.39% ( ± 7.30 standard deviation) hybrid command accuracy and an average Information Transfer Rate (ITR) of 25.05 ( ± 5.32 standard deviation) bits per minute (bpm). The results of the study demonstrate the suitability of a sequential SSVEP-ERP hBCI with challenging dry electroencephalography (EEG) electrodes and low-compute capacity. Although it presents lower ITR than concurrent hBCIs, our system presents an alternative in small screen settings when the conditions for concurrent hBCIs are difficult to satisfy.

Involuntary memories are not déjà vu

The proposed framework can benefit from integrating predictive processing into the explanation of déjà vu which corresponds to interrupted prediction. Déjà vu is also accompanied by familiarity. However, considerable ambiguity is inherent in familiarity, which necessitates elaboration of this construct. Research findings on involuntary autobiographical memories and déjà vu show discrepancies, and clustering these constructs can be counterproductive for research.

Explicit face memory abilities are positively related to the non-intentional encoding of faces: Behavioral and ERP evidence

Individual differences in face memory abilities have been shown to be related to individual differences in brain activity. The present study investigated brain-behavior relationships for the N250 component in event-related brain potentials, which is taken as a neural sign of face familiarity. We used a task in which a designated, typical target face and several (high- and low-distinctive) nontarget faces had to be distinguished during multiple presentations across a session. Separately, face memory/recognition abilities were measured with easy versus difficult tasks. We replicated an increase of the N250 amplitude to the target face across the session and observed a similar increase for the non-target faces, indicating the build-up of memory representations also for these faces. On the interindividual level, larger across-session N250 amplitude increases to low-distinctive non-target faces were related to faster face recognition as measured in an easy task. These findings indicate that non-intentional encoding of non-target faces into memory is associated with the swift recognition of explicitly learned faces; that is, there is shared variance of incidental and intentional face memory.

Increased N250 elicited by facial familiarity: An ERP study including the face inversion effect and facial emotion processing

The present study aims to explore how familiarity modulates the neural processing of faces under different conditions: upright or inverted, neutral or emotional. To this purpose, 32 participants (25 female; age: M = 27.7 years, SD = 9.3) performed two face/emotion identification tasks during EEG recording. In the first task, to study facial processing, three different categories of facial stimuli were presented during a target detection task: famous familiar faces, faces of loved ones, and unfamiliar faces. To explore the face inversion effect according to each level of familiarity, these facial stimuli were also presented upside down. In the second task, to study emotional face processing, an emotional identification task on personally familiar and unfamiliar faces was conducted. The behavioural results showed an improved performance in the identification of facial expressions of emotion with the increase of facial familiarity, consistent with the previous literature. Regarding electrophysiological results, we found increased amplitudes of the P100, N170, and N250 for inverted compared to upright faces, independently of their degree of familiarity. Moreover, we did not find familiarity effects at the P100 and N170 time-windows, but we found that N250 amplitude was larger for personally familiar compared to unfamiliar faces. This result supports the reasoning that the facial familiarity increases the neural activity during the N250 time-window, which may be explained by the processing of additional information prompted by the viewing of our loved ones faces, in contrast to what happens with unfamiliar individuals.

Differences between high and low performers in face recognition in electrophysiological correlates of face familiarity and distance-to-norm

Valentine's influential norm-based multidimensional face-space model (nMDFS) predicts that perceived distinctiveness of a face increases with its distance to the norm. Occipito-temporal event-related potentials (ERPs) have been recently shown to respond selectively to variations in distance-to-norm (P200) or familiarity (N250, late negativity), respectively (Wuttke & Schweinberger, 2019). Despite growing evidence on interindividual differences in face perception skills at the behavioral level, little research has focused on their electrophysiological correlates. To reveal potential interindividual differences in face spaces, we contrasted high and low performers in face recognition in regards to distance-to-norm (P200) and familiarity (N250). We replicated both the P200 distance-to-norm and the N250 familiarity effect. Importantly, we observed: i) reduced responses in low compared to high performers of face recognition, especially in terms of smaller distance-to-norm effects in the P200, possibly indicating less 'expanded' face spaces in low compared to high performers; ii) increased N250 responses to familiar original faces in high performers, suggesting more robust face identity representations. In summary, these findings suggest the contribution of both early norm-based face coding and robust face representations to individual face recognition skills, and indicate that ERPs can offer a promising route to understand individual differences in face perception and their neurocognitive correlates.

Real and Deepfake Face Recognition: An EEG Study on Cognitive and Emotive Implications

The human brain's role in face processing (FP) and decision making for social interactions depends on recognizing faces accurately. However, the prevalence of deepfakes, AI-generated images, poses challenges in discerning real from synthetic identities. This study investigated healthy individuals' cognitive and emotional engagement in a visual discrimination task involving real and deepfake human faces expressing positive, negative, or neutral emotions. Electroencephalographic (EEG) data were collected from 23 healthy participants using a 21-channel dry-EEG headset; power spectrum and event-related potential (ERP) analyses were performed. Results revealed statistically significant activations in specific brain areas depending on the authenticity and emotional content of the stimuli. Power spectrum analysis highlighted a right-hemisphere predominance in theta, alpha, high-beta, and gamma bands for real faces, while deepfakes mainly affected the frontal and occipital areas in the delta band. ERP analysis hinted at the possibility of discriminating between real and synthetic faces, as N250 (200-300 ms after stimulus onset) peak latency decreased when observing real faces in the right frontal (LF) and left temporo-occipital (LTO) areas, but also within emotions, as P100 (90-140 ms) peak amplitude was found higher in the right temporo-occipital (RTO) area for happy faces with respect to neutral and sad ones.

A label isn't just a label: Brief training leads to label-dependent visuo-cortical processing in adults

The current study examines the extent to which hearing individual-level names (e.g., Jimmy) and category-level labels (e.g., Hitchel) paired with novel objects impacts neural responses across a brief (6 min) learning period. Event-related Potentials (ERPs) were recorded while adult participants (n = 44) viewed and heard exemplars of two different species of named novel objects. ERPs were examined for each labeling condition and compared across the first and second half of the learning trials (∼3 min/half). Mean amplitude decreased for the P1 and increased for the N170 from the first to the second half of trials. The decrease in P1 was right lateralized. In addition, the P1 amplitude recorded over right occipitotemporal regions was greater than left occipitotemporal areas, but only for objects paired with individual-level labels. Category-level labels did not show regional P1 differences. The N250 component was greatest over the right occipitotemporal region and was enhanced for objects labeled with individual-level relative to category-level names during the second half of trials. Overall, these findings highlight the unfolding of label-dependent visual processing across a short training period in adults. The results suggest that linguistic labels have an important, top-down impact, on visual processing and that label specificity shapes visuo-cortical responses within a 6-min learning period.

Changepoint Detection in Noisy Data Using a Novel Residuals Permutation-Based Method (RESPERM): Benchmarking and Application to Single Trial ERPs

An important problem in many fields dealing with noisy time series, such as psychophysiological single trial data during learning or monitoring treatment effects over time, is detecting a change in the model underlying a time series. Here, we present a new method for detecting a single changepoint in a linear time series regression model, termed residuals permutation-based method (RESPERM). The optimal changepoint in RESPERM maximizes Cohen’s effect size with the parameters estimated by the permutation of residuals in a linear model. RESPERM was compared with the SEGMENTED method, a well-established and recommended method for detecting changepoints, using extensive simulated data sets, varying the amount and distribution characteristics of noise and the location of the change point. In time series with medium to large amounts of noise, the variance of the detected changepoint was consistently smaller for RESPERM than SEGMENTED. Finally, both methods were applied to a sample dataset of single trial amplitudes of the N250 ERP component during face learning. In conclusion, RESPERM appears to be well suited for changepoint detection especially in noisy data, making it the method of choice in neuroscience, medicine and many other fields.

Deep Convolutional Neural Network-Based Visual Stimuli Classification Using Electroencephalography Signals of Healthy and Alzheimer’s Disease Subjects

Visual perception is an important part of human life. In the context of facial recognition, it allows us to distinguish between emotions and important facial features that distinguish one person from another. However, subjects suffering from memory loss face significant facial processing problems. If the perception of facial features is affected by memory impairment, then it is possible to classify visual stimuli using brain activity data from the visual processing regions of the brain. This study differentiates the aspects of familiarity and emotion by the inversion effect of the face and uses convolutional neural network (CNN) models (EEGNet, EEGNet SSVEP (steady-state visual evoked potentials), and DeepConvNet) to learn discriminative features from raw electroencephalography (EEG) signals. Due to the limited number of available EEG data samples, Generative Adversarial Networks (GAN) and Variational Autoencoders (VAE) are introduced to generate synthetic EEG signals. The generated data are used to pretrain the models, and the learned weights are initialized to train them on the real EEG data. We investigate minor facial characteristics in brain signals and the ability of deep CNN models to learn them. The effect of face inversion was studied, and it was observed that the N170 component has a considerable and sustained delay. As a result, emotional and familiarity stimuli were divided into two categories based on the posture of the face. The categories of upright and inverted stimuli have the smallest incidences of confusion. The model’s ability to learn the face-inversion effect is demonstrated once more.