Mechanisms of face perception in humans: A magneto- and electro-encephalographic study

Extensive Visual Training in Adulthood Reduces an Implicit Neural Marker of the Face Inversion Effect

Face identity recognition (FIR) in humans is supported by specialized neural processes whose function is spectacularly impaired when simply turning a face upside-down: the face inversion effect (FIE). While the FIE appears to have a slow developmental course, little is known about the plasticity of the neural processes involved in this effect-and in FIR in general-at adulthood. Here, we investigate whether extensive training (2 weeks, ~16 h) in young human adults discriminating a large set of unfamiliar inverted faces can reduce an implicit neural marker of the FIE for a set of entirely novel faces. In all, 28 adult observers were trained to individuate 30 inverted face identities presented under different depth-rotated views. Following training, we replicate previous behavioral reports of a significant reduction (56% relative accuracy rate) in the behavioral FIE as measured with a challenging four-alternative delayed-match-to-sample task for individual faces across depth-rotated views. Most importantly, using EEG together with a validated frequency tagging approach to isolate a neural index of FIR, we observe the same substantial (56%) reduction in the neural FIE at the expected occipito-temporal channels. The reduction in the neural FIE correlates with the reduction in the behavioral FIE at the individual participant level. Overall, we provide novel evidence suggesting a substantial degree of plasticity in processes that are key for face identity recognition in the adult human brain.

The Neural Correlates of Developmental Prosopagnosia: Twenty-Five Years on

Faces play a crucial role in social interactions. Developmental prosopagnosia (DP) refers to the lifelong difficulty in recognizing faces despite the absence of obvious signs of brain lesions. In recent decades, the neural substrate of this condition has been extensively investigated. While early neuroimaging studies did not reveal significant functional and structural abnormalities in the brains of individuals with developmental prosopagnosia (DPs), recent evidence identifies abnormalities at multiple levels within DPs' face-processing networks. The current work aims to provide an overview of the convergent and contrasting findings by examining twenty-five years of neuroimaging literature on the anatomo-functional correlates of DP. We included 55 original papers, including 63 studies that compared the brain structure (MRI) and activity (fMRI, EEG, MEG) of healthy control participants and DPs. Despite variations in methods, procedures, outcomes, sample selection, and study design, this scoping review suggests that morphological, functional, and electrophysiological features characterize DPs' brains, primarily within the ventral visual stream. Particularly, the functional and anatomical connectivity between the Fusiform Face Area and the other face-sensitive regions seems strongly impaired. The cognitive and clinical implications as well as the limitations of these findings are discussed in light of the available knowledge and challenges in the context of DP.

Maturational trajectory of fusiform gyrus neural activity when viewing faces: From 4 months to 4 years old

Infant and young child electrophysiology studies have provided information regarding the maturation of face-encoding neural processes. A limitation of previous research is that very few studies have examined face-encoding processes in children 12-48 months of age, a developmental period characterized by rapid changes in the ability to encode facial information. The present study sought to fill this gap in the literature via a longitudinal study examining the maturation of a primary node in the face-encoding network-the left and right fusiform gyrus (FFG). Whole-brain magnetoencephalography (MEG) data were obtained from 25 infants with typical development at 4-12 months, and with follow-up MEG exams every ∼12 months until 3-4 years old. Children were presented with color images of Face stimuli and visual noise images (matched on spatial frequency, color distribution, and outer contour) that served as Non-Face stimuli. Using distributed source modeling, left and right face-sensitive FFG evoked waveforms were obtained from each child at each visit, with face-sensitive activity identified via examining the difference between the Non-Face and Face FFG timecourses. Before 24 months of age (Visits 1 and 2) the face-sensitive FFG M290 response was the dominant response, observed in the left and right FFG ∼250-450 ms post-stimulus. By 3-4 years old (Visit 4), the left and right face-sensitive FFG response occurred at a latency consistent with a face-sensitive M170 response ∼100-250 ms post-stimulus. Face-sensitive left and right FFG peak latencies decreased as a function of age (with age explaining greater than 70% of the variance in face-sensitive FFG latency), and with an adult-like FFG latency observed at 3-4 years old. Study findings thus showed face-sensitive FFG maturational changes across the first 4 years of life. Whereas a face-sensitive M290 response was observed under 2 years of age, by 3-4 years old, an adult-like face-sensitive M170 response was observed bilaterally. Future studies evaluating the maturation of face-sensitive FFG activity in infants at risk for neurodevelopmental disorders are of interest, with the present findings suggesting age-specific face-sensitive neural markers of a priori interest.

Face pareidolia in the brain: Impact of gender and orientation

Research on face sensitivity is of particular relevance during the rapidly evolving Covid-19 pandemic leading to social isolation, but also calling for intact interaction and sharing. Humans possess high sensitivity even to a coarse face scheme, seeing faces in non-face images where real faces do not exist. The advantage of non-face images is that single components do not trigger face processing. Here by implementing a novel set of Face-n-Thing images, we examined (i) how face tuning alters with changing display orientation, and (ii) whether it is affected by observers’ gender. Young females and males were presented with a set of Face-n-Thing images either with canonical upright orientation or inverted 180° in the image plane. Face impression was substantially impeded by display inversion. Furthermore, whereas with upright display orientation, no gender differences were found, with inversion, Face-n-Thing images elicited face impression in females significantly more often. The outcome sheds light on the origins of the face inversion effect in general. Moreover, the findings open a way for examination of face sensitivity and underwriting brain networks in neuropsychiatric conditions related to the current pandemic (such as depression and anxiety), most of which are gender/sex-specific.

Neuromagnetic correlates of hemispheric specialization for face and word recognition

The adult human brain appears to have specialized and independent neural systems for the visual processing of faces and words: greater selectivity for faces in the right hemisphere (RH) while greater selectivity for words in the left hemisphere (LH). Nevertheless, the nature of functional differences between the hemispheres is still largely unknown. To elucidate the hemispheric specialization for face and word recognition, event-related magnetic fields (ERFs) were recorded in young adults while they passively viewed faces and words presented either in the right visual field or in the left visual field. If the neural correlates of face recognition and word recognition reflect the same lateralization profile, then the lateralization of the magnetic source of the M170 component should follow a similar profile, with a greater M170 response for faces in the RH and a greater M170 response for words in the LH. We observed the expected finding of a larger M170 in the LH for words. Unexpectedly, a larger M170 response in the RH for faces was not found. Thus, the hemispheric organization of face recognition is different from that of word recognition in terms of specificity.

The sequence of cortical activity inferred by response latency variability in the human ventral pathway of face processing

Variability in neuronal response latency has been typically considered caused by random noise. Previous studies of single cells and large neuronal populations have shown that the temporal variability tends to increase along the visual pathway. Inspired by these previous studies, we hypothesized that functional areas at later stages in the visual pathway of face processing would have larger variability in the response latency. To test this hypothesis, we used magnetoencephalographic data collected when subjects were presented with images of human faces. Faces are known to elicit a sequence of activity from the primary visual cortex to the fusiform gyrus. Our results revealed that the fusiform gyrus showed larger variability in the response latency compared to the calcarine fissure. Dynamic and spectral analyses of the latency variability indicated that the response latency in the fusiform gyrus was more variable than in the calcarine fissure between 70 ms and 200 ms after the stimulus onset and between 4 Hz and 40 Hz, respectively. The sequential processing of face information from the calcarine sulcus to the fusiform sulcus was more reliably detected based on sizes of the response variability than instants of the maximal response peaks. With two areas in the ventral visual pathway, we show that the variability in response latency across brain areas can be used to infer the sequence of cortical activity.

Modulation of Neural Oscillatory Activity during Dynamic Face Processing

Various neuroimaging and neurophysiological methods have been used to examine neural activation patterns in response to faces. However, much of previous research has relied on static images of faces, which do not allow a complete description of the temporal structure of face-specific neural activities to be made. More recently, insights are emerging from fMRI studies about the neural substrates that underpin our perception of naturalistic dynamic face stimuli, but the temporal and spectral oscillatory activity associated with processing dynamic faces has yet to be fully characterized. Here, we used MEG and beamformer source localization to examine the spatiotemporal profile of neurophysiological oscillatory activity in response to dynamic faces. Source analysis revealed a number of regions showing enhanced activation in response to dynamic relative to static faces in the distributed face network, which were spatially coincident with regions that were previously identified with fMRI. Furthermore, our results demonstrate that perception of realistic dynamic facial stimuli activates a distributed neural network at varying time points facilitated by modulations in low-frequency power within alpha and beta frequency ranges (8-30 Hz). Naturalistic dynamic face stimuli may provide a better means of representing the complex nature of perceiving facial expressions in the real world, and neural oscillatory activity can provide additional insights into the associated neural processes.

Temporal dynamics of neural activity underlying unconscious processing of manipulable objects

The primate visual system is assumed to comprise two main pathways: a ventral pathway for shape and color perception and a dorsal pathway for spatial processing and visuomotor control. Previous studies consistently reported strong activation in the dorsal pathway(especially in the inferior parietal region) induced by manipulable object images such as tools. However, it is controversial whether the dorsal pathway retains this preferential activity to tool images under unconscious perception. In the present study, we used magnetoencephalography (MEG) and investigated spatio-temporal dynamics of neural responses to visible and invisible tool images. A presentation of visible tool images elicited a strong neural response over the parietal regions in the left hemisphere peaking at 400 msec. This response unique to the processing of tool information in the left parietal regions was still observed when conscious perception of tool images was inhibited by interocular suppression. Furthermore, analyses of neural oscillation signals revealed a suppression of m rhythm (8-13 Hz), a neural index of movement execution or imagery,induced by both visible and invisible tools. Those results indicated that the neural circuit to process the tool information was preserved under unconscious perception, highlighting an implicit aspect of the dorsal pathway.

A magnetoencephalographic study of face processing: M170, gamma-band oscillations and source localization

EEG studies suggested that the N170 ERP and Gamma-band responses to faces reflect early and later stages of a multiple-level face-perception mechanism, respectively. However, these conclusions should be considered cautiously because EEG-recorded Gamma may be contaminated by noncephalic activity such as microsaccades. Moreover, EEG studies of Gamma cannot easily reveal its intracranial sources. Here we recorded MEG rather than EEG, assessed the sources of the M170 and Gamma oscillations using beamformer, and explored the sensitivity of these neural manifestations to global, featural and configural information in faces. The M170 was larger in response to faces and face components than in response to watches. Scrambling the configuration of the inner components of the face even if presented without the face contour reduced and delayed the M170. The amplitude of MEG Gamma oscillations (30-70 Hz) was higher than baseline during an epoch between 230-570 ms from stimulus onset and was particularly sensitive to the configuration of the stimuli, regardless of their category. However, in the lower part of this frequency range (30-40 Hz) only physiognomic stimuli elevated the MEG above baseline. Both the M170 and Gamma were generated in a posterior-ventral network including the fusiform, inferior-occipital and lingual gyri, all in the right hemisphere. The generation of Gamma involved additional sources in the visual system, bilaterally. We suggest that the evoked M170 manifests a face-perception mechanism based on the global characteristics of face, whereas the induced Gamma oscillations are associated with the integration of visual input into a pre-existent coherent perceptual representation.

Human Scalp Electroencephalography Reveals that Repetition Suppression Varies with Expectation

Repetitions of a sensory event elicit lower levels of brain activity than its initial presentation (“repetition suppression,” RS). According to one view, RS depends on the biophysics of neuronal discharge, and is thus an automatic consequence of stimulus processing (“fatigue” model). Another account suggests that RS depends on the statistical structure of the environment, and occurs when repeated stimuli are less surprising than novel stimuli (“surprise reduction” model). In support of the latter view, functional magnetic resonance imaging studies have shown that RS is modulated by the local probability of repetition. However, single-cell recordings from macaque inferotemporal area (IT) have failed to replicate this finding. Here, we recorded scalp electroencephalography from human participants viewing pairs of faces that repeated (face₁–face₁) or alternated (face₁–face₂), in contexts in which repetitions were expected or unexpected. As previously described, event-related potentials in the range of 100–400 ms recorded at posterior electrode sites and at the vertex differed between repetitions and alternations. Critically, at central electrodes, we observed that the difference between repeated and alternating stimuli was attenuated when repetitions were unexpected, as predicted by the surprise reduction model. These findings demonstrate that the modulation of RS by repetition probability is observable using direct neural recording methods in human participants, and that it occurs relatively late (>300 ms) post-stimulus. Finally, we found that theta-band (4–8 Hz) spectral power over central electrodes varied with the three-way interaction between of repetition, expectation, and the rate of change of the environment, consistent with recent reports that frontal theta may be a hallmark of learning processes originating in the anterior cingulate and medial prefrontal cortex.

Effects of inverting contour and features on processing for static and dynamic face perception: an MEG study

We investigated the effects of inverting facial contour (hair and chin) and features (eyes, nose and mouth) on processing for static and dynamic face perception using magnetoencephalography (MEG). We used apparent motion, in which the first stimulus (S1) was replaced by a second stimulus (S2) with no interstimulus interval and subjects perceived visual motion, and presented three conditions as follows: (1) U&U: Upright contour and Upright features, (2) U&I: Upright contour and Inverted features, and (3) I&I: Inverted contour and Inverted features. In static face perception (S1 onset), the peak latency of the fusiform area's activity, which was related to static face perception, was significantly longer for U&I and I&I than for U&U in the right hemisphere and for U&I than for U&U and I&I in the left. In dynamic face perception (S2 onset), the strength (moment) of the occipitotemporal area's activity, which was related to dynamic face perception, was significantly larger for I&I than for U&U and U&I in the right hemisphere, but not the left. These results can be summarized as follows: (1) in static face perception, the activity of the right fusiform area was more affected by the inversion of features while that of the left fusiform area was more affected by the disruption of the spatial relation between the contour and features, and (2) in dynamic face perception, the activity of the right occipitotemporal area was affected by the inversion of the facial contour.

Magnetoencephalographic study of the neural responses in body perception

A fundamental trait of human beings is the ability to discern information communicated by others. The human body is one of the important sources of such information. To date, several researchers have reported two body-selective regions in the brain-the extrastriate body area (EBA) and fusiform body area. As compared to the number of studies on spatial distribution, studies on the temporal processing of body perception are few. The electroencephalography (EEG) findings of a recent study indicate that observation of the human body induces a remarkable response leading to the generation of event-related-potentials that peak at 190 ms. However, source localization by using EEG has limitations. The advantage of magnetoencephalography (MEG) is that it enables localization of cortical activities and has excellent temporal resolution. In this study, we used MEG to measure the neural responses underlying the perception of the human body. Our results suggest that cortical activation induced by body images was observed in the bilateral EBA region with a latency of 190 ms and right-hemispheric dominance. Our study revealed the regions involved and the latency differences between these regions in body perception. Further, our results show the usefulness of MEG for body perception studies and suggest that like the face, the body plays a unique role in the human recognition process.

Selecting forward models for MEG source-reconstruction using model-evidence

We investigated four key aspects of forward models for distributed solutions to the MEG inverse problem: 1) the nature of the cortical mesh constraining sources (derived from an individual's MRI, or inverse-normalised from a template mesh); 2) the use of single-sphere, overlapping spheres, or Boundary Element Model (BEM) head-models; 3) the density of the cortical mesh (3000 vs. 7000 vertices); and 4) whether source orientations were constrained to be normal to that mesh. These were compared within the context of two types of spatial prior on the sources: a single prior corresponding to a standard L2-minimum-norm (MNM) inversion, or multiple sparse priors (MSP). The resulting generative models were compared using a free-energy approximation to the Bayesian model-evidence after fitting multiple epochs of responses to faces or scrambled faces. Statistical tests of the free-energy, across nine participants, showed clear superiority of MSP over MNM models; with the former reconstructing deeper sources. Furthermore, there was 1) no evidence that an individually-defined cortical mesh was superior to an inverse-normalised canonical mesh, but 2) clear evidence that a BEM was superior to spherical head-models, provided individually-defined inner skull and scalp meshes were used. Finally, for MSP models, there was evidence that the combination of 3) higher density cortical meshes and 4) dipoles constrained to be normal to the mesh was superior to lower-density or freely-oriented sources (in contrast to the MNM models, in which free-orientation was optimal). These results have practical implications for MEG source reconstruction, particularly in the context of group studies.

MEG and EEG data fusion: simultaneous localisation of face-evoked responses

We present an empirical Bayesian scheme for distributed multimodal inversion of electromagnetic forward models of EEG and MEG signals. We used a generative model with common source activity and separate error components for each modality. Under this scheme, the weightings of error for each modality, relative to source components, are estimated automatically from the data, by optimising the model-evidence. This obviates the need for arbitrary user-defined weightings. To evaluate the scheme, we acquired three types of data simultaneously from twelve participants: total magnetic flux (as recorded by 102 magnetometers), orthogonal in-plane gradients of the magnetic field (as recorded by 204 planar gradiometers) and voltage differences in the electrical field (recorded by 70 electrodes). We assessed the relative precision of each sensor-type in terms of signal-to-noise ratio (SNR); using empirical sample variances and optimised estimators from the generative model. We then compared the localisation of face-evoked responses, using each modality separately, with that obtained by their "fusion" under the common generative model. Finally, we quantified the conditional precisions of the source estimates using their posterior covariance, confirming that EEG can improve MEG-based source reconstructions.

Early (M170) activation of face-specific cortex by face-like objects

The tendency to perceive faces in random patterns exhibiting configural properties of faces is an example of pareidolia. Perception of 'real' faces has been associated with a cortical response signal arising at approximately 170 ms after stimulus onset, but what happens when nonface objects are perceived as faces? Using magnetoencephalography, we found that objects incidentally perceived as faces evoked an early (165 ms) activation in the ventral fusiform cortex, at a time and location similar to that evoked by faces, whereas common objects did not evoke such activation. An earlier peak at 130 ms was also seen for images of real faces only. Our findings suggest that face perception evoked by face-like objects is a relatively early process, and not a late reinterpretation cognitive phenomenon.

Event-related potentials of self-face recognition in children with pervasive developmental disorders

Patients with pervasive developmental disorders (PDD) often have difficulty reading facial expressions and deciphering their implied meaning. We focused on semantic encoding related to face cognition to investigate event-related potentials (ERPs) to the subject's own face and familiar faces in children with and without PDD. Eight children with PDD (seven boys and one girl; aged 10.8+/-2.9 years; one left-handed) and nine age-matched typically developing children (four boys and five girls; aged 11.3+/-2.3 years; one left-handed) participated in this study. The stimuli consisted of three face images (self, familiar, and unfamiliar faces), one scrambled face image, and one object image (e.g., cup) with gray scale. We confirmed three major components: N170 and early posterior negativity (EPN) in the occipito-temporal regions (T5 and T6) and P300 in the parietal region (Pz). An enhanced N170 was observed as a face-specific response in all subjects. However, semantic encoding of each face might be unrelated to N170 because the amplitude and latency were not significantly different among the face conditions. On the other hand, an additional component after N170, EPN which was calculated in each subtracted waveform (self vs. familiar and familiar vs. unfamiliar), indicated self-awareness and familiarity with respect to face cognition in the control adults and children. Furthermore, the P300 amplitude in the control adults was significantly greater in the self-face condition than in the familiar-face condition. However, no significant differences in the EPN and P300 components were observed among the self-, familiar-, and unfamiliar-face conditions in the PDD children. The results suggest a deficit of semantic encoding of faces in children with PDD, which may be implicated in their delay in social communication.

Effect of configural distortion on a face-related ERP evoked by random dots blinking

Using random dots blinking (RDB), which reflects the activity of the higher visual area related to face perception, the following stimuli were presented. (1) Upright: a schematic face; (2) Inverted: the Upright stimulus inverted; and (3) Scrambled: the same contour and features as in Upright but with the spatial relation distorted. Clear negative components (N-ERP250) were identified at approximately 250 ms after stimulus onset. At the T5 and T6 electrodes, the peak latency was significantly longer for Inverted and Scrambled than for Upright. At the P4 electrode, the maximum amplitude was significantly larger for Scrambled than for Upright and Inverted. These results indicate that the delayed latency for Inverted and Scrambled reflects the involvement of the additional analytic processing caused by the configural distortion, and that the increase in amplitude for Scrambled indicates the existence of further processing caused by the distortion of the spatial relationship between the contour and features.

Configurational factors in the perception of faces and non-facial objects: an ERP study

During the viewing of human faces, it is easier to recognize detailed features if the face is presented in an unusual configuration; for example, a split face. The present study used electroencephalography to investigate the brain activity elicited in response to a neutral, inverted, and split face and compared this to the activity produced in response to a non-facial stimulus (a clock). Results showed that the N170 response amplitude was larger and its latency longer during recognition of split and inverted faces as compared to a normal face, whereas no amplitude change was seen for the different clock configurations. However, for the P300 component, larger amplitudes were observed in both the face and the object condition. Taken together, the results suggest that unusual human face presentations are detected at earlier stages than unusual object presentations.

Population-level inferences for distributed MEG source localization under multiple constraints: Application to face-evoked fields

We address some key issues entailed by population inference about responses evoked in distributed brain systems using magnetoencephalography (MEG). In particular, we look at model selection issues at the within-subject level and feature selection issues at the between-subject level, using responses evoked by intact and scrambled faces around 170 ms (M170). We compared the face validity of subject-specific forward models and their summary statistics in terms of how estimated responses reproduced over subjects. At the within-subject level, we focused on the use of multiple constraints, or priors, for inverting distributed source models. We used restricted maximum likelihood (ReML) estimates of prior covariance components (in both sensor and source space) and show that their relative importance is conserved over subjects. At the between-subject level, we used standard anatomical normalization methods to create posterior probability maps that furnish inference about regionally specific population responses. We used these to compare different summary statistics, namely; (i) whether to test for differences between condition-specific source estimates, or whether to test the source estimate of differences between conditions, and (ii) whether to accommodate differences in source orientation by using signed or unsigned (absolute) estimates of source activity.

Effects of face contour and features on early occipitotemporal activity when viewing eye movement

We investigated whether early activity in the occipitotemporal region, corresponding to human MT/V5, is influenced by a face contour and/or features such as the mouth using magnetoencephalography (MEG). We used apparent motion as visual stimuli and compared four conditions as follows: (1) CDL: A schematic face consisting of a face Contour, two Dots and a horizontal Line (2) CD: The Contour and two Dots (3) DL: Two Dots and a horizontal Line and (4) D: Two Dots only. Subjects described a simple movement of dots for D, but eye movement for CDL, DL and CD, though movement modalities were the same through all conditions. We used a single equivalent current dipole (ECD) model within 145-220 ms after stimulus onset and estimated the location, dipole moment (strength) and peak latency. There were no significant differences in the peak latency of the estimated dipoles between each condition, but the activity was significantly stronger for CDL than for CD (p<0.05), DL (p<0.01), and D (p<0.01) in the right hemisphere, and DL and D (p<0.01) in the left. These results indicated that there is specific information processing for eye movements in the occipitotemporal region, the human MT/V5 homologue, and this activity was significantly influenced by whether movements appeared with the face contour and/or features, in other words, whether the eyes moved or not, even if the movement itself was the same.

Differential priming effects of color-opponent subliminal stimulation on visual magnetic responses

We investigated the effects of subliminal stimulation on visible stimulation to demonstrate the priority of facial discrimination processing, using a unique, indiscernible, color-opponent subliminal (COS) stimulation. We recorded event-related magnetic cortical fields (ERF) by magnetoencephalography (MEG) after the presentation of a face or flower stimulus with COS conditioning using a face, flower, random pattern, and blank. The COS stimulation enhanced the response to visible stimulation when the figure in the COS stimulation was identical to the target visible stimulus, but more so for the face than for the flower stimulus. The ERF component modulated by the COS stimulation was estimated to be located in the ventral temporal cortex. We speculated that the enhancement was caused by an interaction of the responses after subthreshold stimulation by the COS stimulation and the suprathreshold stimulation after target stimulation, such as in the processing for categorization or discrimination. We also speculated that the face was processed with priority at the level of the ventral temporal cortex during visual processing outside of consciousness.