Altered dynamic network interactions in children with ASD during face recognition revealed by time-varying EEG networks

Although the electrophysiological event-related potential in face processing (e.g. N170) is widely accepted as a face-sensitivity biomarker that is deficient in children with autism spectrum disorders, the time-varying brain networks during face recognition are still awaiting further investigation. To explore the social deficits in autism spectrum disorder, especially the time-varying brain networks during face recognition, the current study analyzed the N170, cortical activity, and time-varying networks under 3 tasks (face-upright, face-inverted, and house-upright) in autism spectrum disorder and typically developing children. The results revealed a smaller N170 amplitude in autism spectrum disorder compared with typically developing, along with decreased cortical activity mainly in occipitotemporal areas. Concerning the time-varying networks, the atypically stronger information flow and brain network connections across frontal, parietal, and temporal regions in autism spectrum disorder were reported, which reveals greater effort was exerted by autism spectrum disorder to obtain comparable performance to the typically developing children, although the amplitude of N170 was still smaller than that of the typically developing children. Different brain activation states and interaction patterns of brain regions during face processing were discovered between autism spectrum disorder and typically developing. These findings shed light on the face-processing mechanisms in children with autism spectrum disorder and provide new insight for understanding the social dysfunction of autism spectrum disorder.

Recognizing distant faces

As an 'early alerting' sense, one of the primary tasks for the human visual system is to recognize distant objects. In the specific context of facial identification, this ecologically important task has received surprisingly little attention. Most studies have investigated facial recognition at short, fixed distances. Under these conditions, the photometric and configural information related to the eyes, nose and mouth are typically found to be primary determinants of facial identity. Here we characterize face recognition performance as a function of viewing distance and investigate whether the primacy of the internal features continues to hold across increasing viewing distances. We find that exploring the distance dimension reveals a qualitatively different salience distribution across a face. Observers' recognition performance significantly exceeds that obtained with the internal facial physiognomy, and also exceeds the computed union of performances with internal and external features alone, suggesting that in addition to the mutual configuration of the eyes, nose and mouth, it is the relationships between these features and external head contours that are crucial for recognition. We have also conducted computational studies with convolutional neural networks trained on the task of face recognition to examine whether this representational bias could emerge spontaneously through exposure to faces. The results provide partial support for this possibility while also highlighting important differences between the human and artificial system. These findings have implications for the nature of facial representations useful for a visual system, whether human or machine, for recognition over large and varying distances.

From individual features to full faces: Combining aspects of face information

We investigated how information from face features is combined by comparing sensitivity to individual features with that for external (head shape, hairline) and internal (nose, mouth, eyes, eyebrows) feature compounds. Discrimination thresholds were measured for synthetic faces under the following conditions: (a) full-faces; (b) individual features (e.g., nose); and (c) feature compounds (either external or internal). Individual features and feature compounds were presented both in isolation and embedded within a fixed, task irrelevant face context. Relative to the full-face baseline, threshold elevations for the internal feature compound (2.41x) were comparable to those for the most sensitive individual feature (nose = 2.12x). External features demonstrated the same pattern. A model that incorporated all available feature information within a single channel in an efficient way overestimated sensitivity to feature compounds. Embedding individual features within a task-irrelevant context reduced discrimination sensitivity, relative to isolated presentation. Sensitivity to feature compounds, however, was unaffected by embedding. A loss of sensitivity when embedding features within a fixed-face context is consistent with holistic processing, which limits access to information about individual features. However, holistic combination of information across face features is not efficient: Sensitivity to feature compounds is no better than sensitivity to the best individual feature. No effect of embedding internal feature compounds within task-irrelevant external face features (or vice versa) suggests that external and internal features are processed independently.

A face is more than just the eyes, nose, and mouth: fMRI evidence that face-selective cortex represents external features

What is a face? Intuition, along with abundant behavioral and neural evidence, indicates that internal features (e.g., eyes, nose, mouth) are critical for face recognition, yet some behavioral and neural findings suggest that external features (e.g., hair, head outline, neck and shoulders) may likewise be processed as a face. Here we directly test this hypothesis by investigating how external (and internal) features are represented in the brain. Using fMRI, we found highly selective responses to external features (relative to objects and scenes) within the face processing system in particular, rivaling that observed for internal features. We then further asked how external and internal features are represented in regions of the cortical face processing system, and found a similar division of labor for both kinds of features, with the occipital face area and posterior superior temporal sulcus representing the parts of both internal and external features, and the fusiform face area representing the coherent arrangement of both internal and external features. Taken together, these results provide strong neural evidence that a "face" is composed of both internal and external features.

Position selectivity in face-sensitive visual cortex to facial and nonfacial stimuli: an fMRI study

BACKGROUND

Evidence for position sensitivity in object-selective visual areas has been building. On one hand, most of the relevant studies have utilized stimuli for which the areas are optimally selective and examine small sections of cortex. On the other hand, visual field maps established with nonspecific stimuli have been found in increasingly large areas of visual cortex, though generally not in areas primarily responsive to faces.

METHODS

fMRI was used to study the position sensitivity of the occipital face area (OFA) and the fusiform face area (FFA) to both standard rotating wedge retinotopic mapping stimuli and quadrant presentations of synthetic facial stimuli. Analysis methods utilized were both typical, that is, mean univariate BOLD signals and multivoxel pattern analysis (MVPA), and novel, that is, distribution of voxels to pattern classifiers and use of responses to nonfacial retinotopic mapping stimuli to classify responses to facial stimuli.

RESULTS

Polar angle sensitivity was exhibited to standard retinotopic mapping stimuli with a stronger contralateral bias in OFA than in FFA, a stronger bias toward the vertical meridian in FFA than in OFA, and a bias across both areas toward the inferior visual field. Contralateral hemispheric lateralization of both areas was again shown using synthetic face stimuli based on univariate BOLD signals, MVPA, and the biased contribution of voxels toward multivariate classifiers discriminating the contralateral visual field. Classifiers based on polar angle responsivity were used to classify the patterns of activation above chance levels to face stimuli in the OFA but not in the FFA.

CONCLUSIONS

Both the OFA and FFA exhibit quadrant sensitivity to face stimuli, though the OFA exhibits greater position responsivity across stimuli than the FFA and includes overlap in the response pattern to the disparate stimulus types. Such biases are consistent with varying position sensitivity along different surfaces of occipito-temporal cortex.

Identifying neural correlates of visual consciousness with ALE meta-analyses

Neural correlates of consciousness (NCC) have been a topic of study for nearly two decades. In functional imaging studies, several regions have been proposed to constitute possible candidates for NCC, but as of yet, no quantitative summary of the literature on NCC has been done. The question whether single (striate or extrastriate) regions or a network consisting of extrastriate areas that project directly to fronto-parietal regions are necessary and sufficient neural correlates for visual consciousness is still highly debated [e.g., Rees et al., 2002, Nat Rev. Neurosci 3, 261-270; Tong, 2003, Nat Rev. Neurosci 4, 219-229]. The aim of this work was to elucidate this issue and give a synopsis of the present state of the art by conducting systematic and quantitative meta-analyses across functional magnetic resonance imaging (fMRI) studies using several standard paradigms for conscious visual perception. In these paradigms, consciousness is operationalized via perceptual changes, while the visual stimulus remains invariant. An activation likelihood estimation (ALE) meta-analysis was performed, representing the best approach for voxel-wise meta-analyses to date. In addition to computing a meta-analysis across all paradigms, separate meta-analyses on bistable perception and masking paradigms were conducted to assess whether these paradigms show common or different NCC. For the overall meta-analysis, we found significant clusters of activation in inferior and middle occipital gyrus; fusiform gyrus; inferior temporal gyrus; caudate nucleus; insula; inferior, middle, and superior frontal gyri; precuneus; as well as in inferior and superior parietal lobules. These results suggest a subcortical-extrastriate-fronto-parietal network rather than a single region that constitutes the necessary NCC. The results of our exploratory paradigm-specific meta-analyses suggest that this subcortical-extrastriate-fronto-parietal network might be differentially activated as a function of the paradigms used to probe for NCC.

Curvature-processing network in macaque visual cortex

Our visual environment abounds with curved features. Thus, the goal of understanding visual processing should include the processing of curved features. Using functional magnetic resonance imaging in behaving monkeys, we demonstrated a network of cortical areas selective for the processing of curved features. This network includes three distinct hierarchically organized regions within the ventral visual pathway: a posterior curvature-biased patch (PCP) located in the near-foveal representation of dorsal V4, a middle curvature-biased patch (MCP) located on the ventral lip of the posterior superior temporal sulcus (STS) in area TEO, and an anterior curvature-biased patch (ACP) located just below the STS in anterior area TE. Our results further indicate that the processing of curvature becomes increasingly complex from PCP to ACP. The proximity of the curvature-processing network to the well-known face-processing network suggests a possible functional link between them.

Processing of configural and componential information in face-selective cortical areas

We investigated how face-selective cortical areas process configural and componential face information and how race of faces may influence these processes. Participants saw blurred (preserving configural information), scrambled (preserving componential information), and whole faces during fMRI scan, and performed a post-scan face recognition task using blurred or scrambled faces. The fusiform face area (FFA) showed stronger activation to blurred than to scrambled faces, and equivalent responses to blurred and whole faces. The occipital face area (OFA) showed stronger activation to whole than to blurred faces, which elicited similar responses to scrambled faces. Therefore, the FFA may be more tuned to process configural than componential information, whereas the OFA similarly participates in perception of both. Differences in recognizing own- and other-race blurred faces were correlated with differences in FFA activation to those faces, suggesting that configural processing within the FFA may underlie the other-race effect in face recognition.

The Muslim headscarf and face perception: "they all look the same, don't they?"

The headscarf conceals hair and other external features of a head (such as the ears). It therefore may have implications for the way in which such faces are perceived. Images of faces with hair (H) or alternatively, covered by a headscarf (HS) were used in three experiments. In Experiment 1 participants saw both H and HS faces in a yes/no recognition task in which the external features either remained the same between learning and test (Same) or switched (Switch). Performance was similar for H and HS faces in both the Same and Switch condition, but in the Switch condition it dropped substantially compared to the Same condition. This implies that the mere presence of the headscarf does not reduce performance, rather, the change between the type of external feature (hair or headscarf) causes the drop in performance. In Experiment 2, which used eye-tracking methodology, it was found that almost all fixations were to internal regions, and that there was no difference in the proportion of fixations to external features between the Same and Switch conditions, implying that the headscarf influenced processing by virtue of extrafoveal viewing. In Experiment 3, similarity ratings of the internal features of pairs of HS faces were higher than pairs of H faces, confirming that the internal and external features of a face are perceived as a whole rather than as separate components.

The context congruency effect is face specific

There is evidence that faces are processed by specialized and independent modules that treat them as global configurations, or wholes (Axelrod & Yovel, 2010; Kanwisher, McDermott, & Chun, 1997). The holistic nature of face perception has been demonstrated with several experimental paradigms designed to examine whether facial parts interact, or are accessed independently. A recently introduced paradigm (Meinhardt-Injac, Persike, & Meinhardt, 2010) measures the strength of contextual interaction among internal and external facial features in congruent and incongruent target/no-target relationships. For this paradigm it is shown that the context congruency effect is indeed face specific: A strong and asymmetric contextual interaction of the inner and the outer stimulus regions exists for faces, but is absent for watches, which represent a non-facial object class with a comparable inner/outer object structure.

Local discriminability determines the strength of holistic processing for faces in the fusiform face area

Recent evidence suggests that the Fusiform Face Area (FFA) is not exclusively dedicated to the interactive processing of face features, but also contains neurons sensitive to local features. This suggests the existence of both interactive and local processing modes, consistent with recent behavioral findings that the strength of interactive feature processing (IFP) engages most strongly when similar features need to be disambiguated. Here we address whether the engagement of the FFA into interactive versus featural representational modes is governed by local feature discriminability. We scanned human participants while they matched target features within face pairs, independently of the context of distracter features. IFP was operationalized as the failure to match the target without being distracted by distracter features. Picture-plane inversion was used to disrupt IFP while preserving input properties. We found that FFA activation was comparably strong, irrespective of whether similar target features were embedded in dissimilar contexts(i.e., inducing robust IFP) or dissimilar target features were embedded in the same context (i.e., engaging local processing). Second, inversion decreased FFA activation to faces most robustly when similar target features were embedded in dissimilar contexts, indicating that FFA engages into IFP mainly when features cannot be disambiguated at a local level. Third, by means of Spearman rank correlation tests, we show that the local processing of feature differences in the FFA is supported to a large extent by the Occipital Face Area, the Lateral Occipital Complex, and early visual cortex, suggesting that these regions encode the local aspects of face information. The present findings confirm the co-existence of holistic and featural representations in the FFA. Furthermore, they establish FFA as the main contributor to the featural/holistic representational mode switches determined by local discriminability.

The response of face-selective cortex with single face parts and part combinations

A critical issue in object recognition research is how the parts of an object are analyzed by the visual system and combined into a perceptual whole. However, most of the previous research has examined how changes to object parts influence recognition of the whole, rather than recognition of the parts themselves. This is particularly true of the research on face recognition, and especially with questions related to the neural substrates. Here, we investigated patterns of BOLD fMRI brain activation with internal face parts (features) presented singly and in different combinations. A preference for single features over combinations was found in the occipital face area (OFA) as well as a preference for the two-eyes combination stimulus over other combination stimulus types. The fusiform face area (FFA) and lateral occipital cortex (LO) showed no preferences among the single feature and combination stimulus types. The results are consistent with a growing view that the OFA represents processes involved in early, feature-based analysis.

The significance of hair for face recognition

Hair is a feature of the head that frequently changes in different situations. For this reason much research in the area of face perception has employed stimuli without hair. To investigate the effect of the presence of hair we used faces with and without hair in a recognition task. Participants took part in trials in which the state of the hair either remained consistent (Same) or switched between learning and test (Switch). It was found that in the Same trials performance did not differ for stimuli presented with and without hair. This implies that there is sufficient information in the internal features of the face for optimal performance in this task. It was also found that performance in the Switch trials was substantially lower than in the Same trials. This drop in accuracy when the stimuli were switched suggests that faces are represented in a holistic manner and that manipulation of the hair causes disruption to this, with implications for the interpretation of some previous studies.

Face Processing Changes in Normal Aging Revealed by fMRI Adaptation

We investigated the neural correlates of facial processing changes in healthy aging using fMRI and an adaptation paradigm. In the scanner, participants were successively presented with faces that varied in identity, viewpoint, both, or neither and performed a head size detection task independent of identity or viewpoint. In right fusiform face area (FFA), older adults failed to show adaptation to the same face repeatedly presented in the same view, which elicited the most adaptation in young adults. We also performed a multivariate analysis to examine correlations between whole-brain activation patterns and behavioral performance in a face-matching task tested outside the scanner. Despite poor neural adaptation in right FFA, high-performing older adults engaged the same face-processing network as high-performing young adults across conditions, except the one presenting a same facial identity across different viewpoints. Low-performing older adults used this network to a lesser extent. Additionally, high-performing older adults uniquely recruited a set of areas related to better performance across all conditions, indicating age-specific involvement of this added network. This network did not include the core ventral face-processing areas but involved the left inferior occipital gyrus, frontal, and parietal regions. Although our adaptation results show that the neuronal representations of the core face-preferring areas become less selective with age, our multivariate analysis indicates that older adults utilize a distinct network of regions associated with better face matching performance, suggesting that engaging this network may compensate for deficiencies in ventral face processing regions.

Division of labor between lateral and ventral extrastriate representations of faces, bodies, and objects

The occipito-temporal cortex is strongly implicated in carrying out the high-level computations associated with vision. In human neuroimaging studies, focal regions are consistently found within this broad region that respond strongly and selectively to faces, bodies, or objects. A notable feature of these selective regions is that they are found in pairs. In the posterior-lateral occipito-temporal cortex, focal selectivity is found for faces (occipital face area), bodies (extrastriate body area), and objects (lateral occipital). These three areas are found bilaterally and at close quarters to each other. Likewise, in the ventro-medial occipito-temporal cortex, three similar category-selective regions are found, also in proximity to each other: for faces (fusiform face area), bodies (fusiform body area), and objects (posterior fusiform). Here we review some of the extensive evidence on the functional properties of these areas with two aims. First, we seek to identify principles that distinguish the posterior-lateral and ventro-medial clusters of selective regions but that apply generally within each cluster across the three stimulus kinds. Our review identifies and elaborates several principles by which these relationships hold. In brief, the posterior-lateral representations are more primitive, local, and stimulus-driven relative to the ventro-medial representations, which in contrast are more invariant to visual features, global, and linked to the subjective percept. Second, because the evidence base of studies that compare both posterior-lateral and ventro-medial representations of faces, bodies, and objects is still relatively small, we seek to provoke more cross-talk among the research strands that are traditionally separate. We identify several promising approaches for such future work.

Personally familiar faces are perceived categorically in face-selective regions other than the fusiform face area

Neuroimaging studies of humans have provided inconsistent evidence with respect to the response properties of the fusiform face area (FFA). It has been claimed that neural populations within this region are sensitive to subtle differences between individual faces only when they are perceived as distinct identities [P. Rotshtein et al. (2005)Nature Neuroscience, 8, 107-113]. However, sensitivity to subtle changes of identity was found in previous studies using unfamiliar faces, for which categorical perception is less pronounced. Using functional magnetic resonance adaptation and morph continua of personally familiar faces, we investigated sensitivity to subtle changes between faces that were located either on the same or opposite sides of a categorical perceptual boundary. We found no evidence for categorical perception within the FFA, which exhibited reliable sensitivity to subtle changes of face identity whether these were perceived as distinct identities, or not. On the contrary, both the posterior superior temporal sulcus and prefrontal cortex exhibited categorical perception, as subtle changes between faces perceived as different identities yielded larger release from adaptation than those perceived as the same identity. These observations suggest that, whereas the FFA discriminates subtle physical changes of personally familiar faces, other regions encode faces in a categorical fashion.

Decoding of faces and face components in face-sensitive human visual cortex

A great challenge to the field of visual neuroscience is to understand how faces are encoded and represented within the human brain. Here we show evidence from functional magnetic resonance imaging (fMRI) for spatially distributed processing of the whole face and its components in face-sensitive human visual cortex. We used multi-class linear pattern classifiers constructed with a leave-one-scan-out verification procedure to discriminate brain activation patterns elicited by whole faces, the internal features alone, and the external head outline alone. Furthermore, our results suggest that whole faces are represented disproportionately in the fusiform cortex (FFA) whereas the building blocks of faces are represented disproportionately in occipitotemporal cortex (OFA). Faces and face components may therefore be organized with functional clustering within both the FFA and OFA, but with specialization for face components in the OFA and the whole face in the FFA.

Internal and external features of the face are represented holistically in face-selective regions of visual cortex

The perception and recognition of familiar faces depends critically on an analysis of the internal features of the face (eyes, nose, mouth). We therefore contrasted how information about the internal and external (hair, chin, face outline) features of familiar and unfamiliar faces is represented in face-selective regions. There was a significant response to both the internal and external features of the face when presented in isolation. However, the response to the internal features was greater than the response to the external features. There was significant adaptation to repeated images of either the internal or external features of the face in the fusiform face area (FFA). However, the magnitude of this adaptation was greater for the internal features of familiar faces. Next, we asked whether the internal features of the face are represented independently from the external features. There was a release from adaptation in the FFA to composite images in which the internal features were varied but the external features were unchanged, or when the internal features were unchanged but the external features varied, demonstrating a holistic response. Finally, we asked whether the holistic response to faces could be influenced by the context in which the face was presented. We found that adaptation was still evident to composite images in which the face was unchanged but body features were varied. Together, these findings show that although internal features are important in the neural representation of familiar faces, the face's internal and external features are represented holistically in face-selective regions of the human brain.