Inversion effects in face-selective cortex with combinations of face parts

The face inversion effect does not provide a pure measure of holistic face processing

It is widely held that upright faces are processed more holistically than inverted faces and that this difference is reflected in the face inversion effect. It is not clear, however, how the inversion effect can best be measured, whether it is task specific, or even whether it specifically correlates with processing of upright faces. We examined these questions in a large sample (N = 420) who provided data on processing of upright and inverted stimuli in two different tasks with faces and one with objects. We find that the inversion effects are task dependent, and that they do not correlate better among face processing tasks than they do across face and object processing tasks. These findings were obtained regardless of whether inversion effects were measured by means of difference scores or regression. In comparison, only inversion effects based on regression predicted performance with upright faces in tasks other than those the inversion effects were derived from. Critically, however, inversion effects based on regression also predicted performance with inverted faces to a similar degree as they predicted performance with upright faces. Consequently, and contrary to what is commonly assumed, inversion effects do not seem to capture effects specific to holistic processing of upright faces. While the present findings do not bring us closer to an understanding of which changes in cognitive processing are induced by inversion, they do suggest that inversion effects do not reflect a unitary construct; an implicit assumption that seems to characterize much of the research regarding face processing.

Is increased activation in the fusiform face area to Greebles a result of appropriate expertise training or caused by Greebles' face likeness?

Background

In 2011, Brants et al. trained eight individuals to become Greeble experts and found neuronal inversion effects [NIEs; i.e., higher fusiform face area (FFA) activity for upright, rather than inverted Greebles]. These effects were also found for faces, both before and after training. By claiming to have replicated the seminal Greeble training study by Gauthier and colleagues in 1999, Brants et al. interpreted these results as participants viewing Greebles as faces throughout training, contrary to the original argument of subjects becoming Greeble experts only after training. However, Brants et al.'s claim presents two issues. First, their behavioral training results did not replicate those of Gauthier and Tarr conducted in 1997 and 1998, raising concerns of whether the right training regime had been adopted. Second, both a literature review and meta-analysis of NIEs in the FFA suggest its impotency as an index of the face(-like) processing.

Objectives

To empirically evaluate these issues, the present study compared two documented training paradigms Gauthier and colleagues in 1997 and 1998, and compared their impact on the brain.

Methods

Sixteen NCKU undergraduate and graduate students (nine girls) were recruited. Sixty Greeble exemplars were categorized by two genders, five families, and six individual levels. The participants were randomly divided into two groups (one for Greeble classification at all three levels and the other for gender- and individual-level training). Several fMRI tasks were administered at various time points, specifically, before training (1st), during training (2nd), and typically no <24 h after reaching expertise criterion (3rd).

Results

The ROI analysis results showed significant increases in the FFA for Greebles, and a clear neural "adaptation," both only in the Gauthier97 group and only after training, reflecting clear modulation of extensive experiences following an "appropriate" training regime. In both groups, no clear NIEs for faces nor Greebles were found, which was also in line with the review of extant studies bearing this comparison.

Conclusion

Collectively, these results invalidate the assumptions behind Brants et al.'s findings.

The face inversion effect or the face upright effect?

The face inversion effect (FIE) refers to the observation that presenting stimuli upside-down impairs the processing of faces disproportionally more than other mono-oriented objects. This has been taken as evidence that processing of faces and objects differ qualitatively. However, nearly all FIE studies are based on comparing individuation of upright faces, which most people are rather good at, with individuation of objects most people are much less familiar with individuating (e.g., radios and airplanes). Consequently, the FIE may mainly reflect differences between categories in how they are processed prior to inversion, with within-category discrimination of upright faces being a much more familiar task than within-category discrimination among members belonging to other object classes. We tested this hypothesis by comparing inversion effects for faces and objects using object recognition tasks that do not require within-category discrimination (object decision and old/new recognition memory tasks). In all tasks (seven with objects and two with faces) we find credible inversion effects, but in no instance were these effects significantly larger for faces than for objects. This suggests that the FIE can be a product of familiarity with the type of identification process required in the upright conditions rather than some process that is selectively affected for faces when stimuli are inverted.

fMRI evidence that hyper-caricatured faces activate object-selective cortex

Many brain imaging studies have looked at the cortical responses to object categories and faces. A popular way to manipulate face stimuli is by using a "face space," a high dimensional representation of individual face images, with the average face located at the origin. However, how the brain responds to faces that deviate substantially from average has not been much explored. Increasing the distance from the average (leading to increased caricaturing) could increase neural responses in face-selective regions, an idea supported by results from non-human primates. Here, we used a face space based on principal component analysis (PCA) to generate faces ranging from average to heavily caricatured. Using functional magnetic resonance imaging (fMRI), we first independently defined face-, object- and scene-selective areas with a localiser scan and then measured responses to parametrically caricatured faces. We also included conditions in which the images of faces were inverted. Interestingly in the right fusiform face area (FFA), we found that the patterns of fMRI response were more consistent as caricaturing increased. However, we found no consistent effect of either caricature level or facial inversion on the average fMRI response in the FFA or face-selective regions more broadly. In contrast, object-selective regions showed an increase in both the consistency of response pattern and the average fMRI response with increasing caricature level. This shows that caricatured faces recruit processing from regions typically defined as object-selective, possibly through enhancing low-level properties that are characteristic of objects.

Cortical Pathways or Mechanism in the Face Inversion Effect in Patients with First-Episode Schizophrenia

OBJECTIVE

Impaired face perception is considered as a hallmark of social disability in schizophrenia. It is widely believed that inverted faces and upright faces are processed by distinct mechanisms. Previous studies have identified that individuals with schizophrenia display poorer face processing than controls. However, the mechanisms underlying the face inversion effect (FIE) in patients with first-episode schizophrenia (FSZ) remain unclear.

METHODS

We designed an fMRI task to investigate the FIE mechanism in patients with schizophrenia. Thirty-four patients with FSZ and thirty-five healthy controls (CON) underwent task-related fMRI scanning, clinical assessment, anhedonia experience examination, and social function and cognitive function evaluation.

RESULTS

The patients with FSZ exhibited distinct functional activity regarding upright and inverted face processing within the cortical face and non-face network. These results suggest that the differences in quantitative processing might mediate the FIE in schizophrenia. Compared with controls, affected patients showed impairments in processing both upright and inverted faces; and for these patients with FSZ, upright face processing was associated with more severe and broader impairment than inverted face processing. Reduced response in the left middle occipital gyrus for upright face processing was related to poorer performance of social function outcomes evaluated using the Personal and Social Performance Scale.

CONCLUSION

Our data suggested that patients with FSZ exhibited similar performance in processing inverted faces and upright faces, but were less efficient than controls; and for these patients, inverted faces are processed less efficiently than upright faces. We also provided a clue that the mechanism under abnormal FIE might be related to an aberrant activation of non-face-selective areas instead of abnormal activation of face-specific areas in patients with schizophrenia. Finally, our study indicated that the neural pathway for upright recognition might be relevant in determining the functional outcomes of this devastating disorder.

Does right hemisphere superiority sufficiently explain the left visual field advantage in face recognition?

The tendency to perceive the identity of the left half of a centrally viewed face more strongly than that of the right half is associated with visual processing of faces in the right hemisphere (RH). Here we investigate conditions under which this well-known left visual field (LVF) half-face advantage fails to occur. Our findings challenge the sufficiency of its explanation as a function of RH specialization for face processing coupled with LVF-RH correspondence. In two experiments we show that the LVF half-face advantage occurs for normal faces and chimeric faces composed of different half-face identities. In a third experiment, we show that face inversion disrupts the LVF half-face advantage. In two additional experiments we show that half-faces viewed in isolation or paired with inverted half-faces fail to show the LVF advantage. Consistent with previous explanations of the LVF half-face advantage, our findings suggest that the LVF half-face advantage reflects RH superiority for processing faces and direct transfer of LVF face information to visual cortex in the RH. Critically, however, our findings also suggest the operation of a third factor, which involves the prioritization of face-processing resources to the LVF, but only when two upright face-halves compete for these resources. We therefore conclude that RH superiority alone does not suffice to explain the LVF advantage in face recognition. We also discuss the implications of our findings for specialized visual processing of faces by the right hemisphere, and we distinguish LVF advantages for faces viewed centrally and peripherally in divided field studies.

A face is more than just the eyes, nose, and mouth: fMRI evidence that face-selective cortex represents external features

What is a face? Intuition, along with abundant behavioral and neural evidence, indicates that internal features (e.g., eyes, nose, mouth) are critical for face recognition, yet some behavioral and neural findings suggest that external features (e.g., hair, head outline, neck and shoulders) may likewise be processed as a face. Here we directly test this hypothesis by investigating how external (and internal) features are represented in the brain. Using fMRI, we found highly selective responses to external features (relative to objects and scenes) within the face processing system in particular, rivaling that observed for internal features. We then further asked how external and internal features are represented in regions of the cortical face processing system, and found a similar division of labor for both kinds of features, with the occipital face area and posterior superior temporal sulcus representing the parts of both internal and external features, and the fusiform face area representing the coherent arrangement of both internal and external features. Taken together, these results provide strong neural evidence that a "face" is composed of both internal and external features.

Stimulus Dependent Dynamic Reorganization of the Human Face Processing Network

Using the "face inversion effect", a hallmark of face perception, we examined network mechanisms supporting face representation by tracking functional magnetic resonance imaging (fMRI) stimulus-dependent dynamic functional connectivity within and between brain networks associated with the processing of upright and inverted faces. We developed a novel approach adapting the general linear model (GLM) framework classically used for univariate fMRI analysis to capture stimulus-dependent fMRI dynamic connectivity of the face network. We show that under the face inversion manipulation, the face and non-face networks have complementary roles that are evident in their stimulus-dependent dynamic connectivity patterns as assessed by network decomposition into components or communities. Moreover, we show that connectivity patterns are associated with the behavioral face inversion effect. Thus, we establish "a network-level signature" of the face inversion effect and demonstrate how a simple physical transformation of the face stimulus induces a dramatic functional reorganization across related brain networks. Finally, we suggest that the dynamic GLM network analysis approach, developed here for the face network, provides a general framework for modeling the dynamics of blocked stimulus-dependent connectivity experimental designs and hence can be applied to a host of neuroimaging studies.

Intracranial markers of conscious face perception in humans

Investigations of the neural basis of consciousness have greatly benefited from protocols that involve the presentation of stimuli at perceptual threshold, enabling the assessment of the patterns of brain activity that correlate with conscious perception, independently of any changes in sensory input. However, the comparison between perceived and unperceived trials would be expected to reveal not only the core neural substrate of a particular conscious perception, but also aspects of brain activity that facilitate, hinder or tend to follow conscious perception. We take a step towards the resolution of these confounds by combining an analysis of neural responses observed during the presentation of faces partially masked by Continuous Flash Suppression, and those responses observed during the unmasked presentation of faces and other images in the same subjects. We employed multidimensional classifiers to decode physical properties of stimuli or perceptual states from spectrotemporal representations of electrocorticographic signals (1071 channels in 5 subjects). Neural activity in certain face responsive areas located in both the fusiform gyrus and in the lateral-temporal/inferior-parietal cortex discriminated seen vs. unseen faces in the masked paradigm and upright faces vs. other categories in the unmasked paradigm. However, only the former discriminated upright vs. inverted faces in the unmasked paradigm. Our results suggest a prominent role for the fusiform gyrus in the configural perception of faces, and possibly other objects that are holistically processed. More generally, we advocate comparative analysis of neural recordings obtained during different, but related, experimental protocols as a promising direction towards elucidating the functional specificities of the patterns of neural activation that accompany our conscious experiences.

Altered topology of neural circuits in congenital prosopagnosia

Using a novel, fMRI-based inter-subject functional correlation (ISFC) approach, which isolates stimulus-locked inter-regional correlation patterns, we compared the cortical topology of the neural circuit for face processing in participants with an impairment in face recognition, congenital prosopagnosia (CP), and matched controls. Whereas the anterior temporal lobe served as the major network hub for face processing in controls, this was not the case for the CPs. Instead, this group evinced hyper-connectivity in posterior regions of the visual cortex, mostly associated with the lateral occipital and the inferior temporal cortices. Moreover, the extent of this hyper-connectivity was correlated with the face recognition deficit. These results offer new insights into the perturbed cortical topology in CP, which may serve as the underlying neural basis of the behavioral deficits typical of this disorder. The approach adopted here has the potential to uncover altered topologies in other neurodevelopmental disorders, as well.

DOI:http://dx.doi.org/10.7554/eLife.25069.001

Human babies prefer to look at faces and pictures of faces over any other object or pattern. A recent study found that even fetuses in the womb will turn their heads towards dots of light shone through the mother’s skin if the dots broadly resemble a face. Brain imaging studies show that face recognition depends on the coordinated activity of multiple brain regions. A core set of areas towards the back of the brain processes the visual features of faces, while regions elsewhere process more variable features such as emotional expressions.

Around 2% of people are born with difficulties in recognizing faces, a condition known as congenital prosopagnosia. These individuals have no obvious anatomical abnormalities in the brain, and brain scans reveal normal activity in core regions of the face processing network. So why do these people have difficulty with face recognition?

One possibility is that the condition reflects differences in the number of connections (or “connectivity”) between brain regions within the face processing network. To test this idea, Rosenthal et al. compared connectivity in individuals with congenital prosopagnosia with that in healthy volunteers. In the healthy volunteers, an area of the network called the anterior temporal cortex was highly connected to many other face processing regions: that is, it acted as a face processing hub. In individuals with congenital prosopagnosia, this hub-like connectivity was missing. Instead, a number of core regions involved in processing the basic visual features of faces, were more highly connected to one another. The greater this “hyperconnectivity”, the better the individual’s face processing abilities.

The findings of Rosenthal et al. pave the way for developing imaging-based tools to diagnose congenital prosopagnosia. The same approach could then be used to investigate the basis of other neurodevelopmental disorders that are thought to involve abnormal communication within brain networks, such as developmental dyslexia.

DOI:http://dx.doi.org/10.7554/eLife.25069.002

Position selectivity in face-sensitive visual cortex to facial and nonfacial stimuli: an fMRI study

BACKGROUND

Evidence for position sensitivity in object-selective visual areas has been building. On one hand, most of the relevant studies have utilized stimuli for which the areas are optimally selective and examine small sections of cortex. On the other hand, visual field maps established with nonspecific stimuli have been found in increasingly large areas of visual cortex, though generally not in areas primarily responsive to faces.

METHODS

fMRI was used to study the position sensitivity of the occipital face area (OFA) and the fusiform face area (FFA) to both standard rotating wedge retinotopic mapping stimuli and quadrant presentations of synthetic facial stimuli. Analysis methods utilized were both typical, that is, mean univariate BOLD signals and multivoxel pattern analysis (MVPA), and novel, that is, distribution of voxels to pattern classifiers and use of responses to nonfacial retinotopic mapping stimuli to classify responses to facial stimuli.

RESULTS

Polar angle sensitivity was exhibited to standard retinotopic mapping stimuli with a stronger contralateral bias in OFA than in FFA, a stronger bias toward the vertical meridian in FFA than in OFA, and a bias across both areas toward the inferior visual field. Contralateral hemispheric lateralization of both areas was again shown using synthetic face stimuli based on univariate BOLD signals, MVPA, and the biased contribution of voxels toward multivariate classifiers discriminating the contralateral visual field. Classifiers based on polar angle responsivity were used to classify the patterns of activation above chance levels to face stimuli in the OFA but not in the FFA.

CONCLUSIONS

Both the OFA and FFA exhibit quadrant sensitivity to face stimuli, though the OFA exhibits greater position responsivity across stimuli than the FFA and includes overlap in the response pattern to the disparate stimulus types. Such biases are consistent with varying position sensitivity along different surfaces of occipito-temporal cortex.

The effect of face inversion for neurons inside and outside fMRI-defined face-selective cortical regions

It is widely believed that face processing in the primate brain occurs in a network of category-selective cortical regions. Combined functional MRI (fMRI)-single-cell recording studies in macaques have identified high concentrations of neurons that respond more to faces than objects within face-selective patches. However, cells with a preference for faces over objects are also found scattered throughout inferior temporal (IT) cortex, raising the question whether face-selective cells inside and outside of the face patches differ functionally. Here, we compare the properties of face-selective cells inside and outside of face-selective patches in the IT cortex by means of an image manipulation that reliably disrupts behavior toward face processing: inversion. We recorded IT neurons from two fMRI-defined face-patches (ML and AL) and a region outside of the face patches (herein labeled OUT) during upright and inverted face stimulation. Overall, turning faces upside down reduced the firing rate of face-selective cells. However, there were differences among the recording regions. First, the reduced neuronal response for inverted faces was independent of stimulus position, relative to fixation, in the face-selective patches (ML and AL) only. Additionally, the effect of inversion for face-selective cells in ML, but not those in AL or OUT, was impervious to whether the neurons were initially searched for using upright or inverted stimuli. Collectively, these results show that face-selective cells differ in their functional characteristics depending on their anatomicofunctional location, suggesting that upright faces are preferably coded by face-selective cells inside but not outside of the fMRI-defined face-selective regions of the posterior IT cortex.

Processing of configural and componential information in face-selective cortical areas

We investigated how face-selective cortical areas process configural and componential face information and how race of faces may influence these processes. Participants saw blurred (preserving configural information), scrambled (preserving componential information), and whole faces during fMRI scan, and performed a post-scan face recognition task using blurred or scrambled faces. The fusiform face area (FFA) showed stronger activation to blurred than to scrambled faces, and equivalent responses to blurred and whole faces. The occipital face area (OFA) showed stronger activation to whole than to blurred faces, which elicited similar responses to scrambled faces. Therefore, the FFA may be more tuned to process configural than componential information, whereas the OFA similarly participates in perception of both. Differences in recognizing own- and other-race blurred faces were correlated with differences in FFA activation to those faces, suggesting that configural processing within the FFA may underlie the other-race effect in face recognition.

Representational demands modulate involvement of perirhinal cortex in face processing

The classic view holds that the medial temporal lobes (MTL) are dedicated to declarative memory functioning. Recent evidence, however, suggests that perirhinal cortex (PrC), a structure within the anterior MTL, may also play a role in perceptual discriminations when representations of complex conjunctions of features, or of gestalt-characteristics of objects must be generated. Interestingly, neuroimaging and electrophysiological recordings in nonhuman primates have also revealed a face patch in the anterior collateral sulcus with preferential responses to face stimuli in various task contexts. In the present fMRI study, we investigated the representational demands that influence PrC involvement in different types of judgments on human faces. Holding stimulus complexity constant, we independently manipulated the nature of the task and the orientation of the stimuli presented (through face inversion). Aspects of right PrC showed increased responses in a forced-choice recognition-memory and a perceptual-oddity task, as compared to a feature-search task that was included to probe visual detection of an isolated face feature. Effects of stimulus orientation in right PrC were observed when the recognition-memory condition for upright faces was compared with all other experimental conditions, including recognition-memory for inverted faces-a result that can be related to past work on the role of PrC in object unitization. Notably, both effects in right PrC paralleled activity patterns in broader networks of regions that also included the right fusiform gyrus and the amygdala, regions frequently implicated in face processing in prior research. As such, the current findings do not support the view that reference to a prior study episode clearly distinguishes the role of PrC from that of more posterior ventral visual pathway regions. They add to a growing body of evidence suggesting that the functional role of specific MTL structures may be best understood in terms of the representations that are required by the task and the stimuli at hand.