Whole not hole: Expert face recognition requires holistic perception

Face identity aftereffects increase monotonically with adaptor extremity over, but not beyond, the range of natural faces

Face identity aftereffects have been used to test theories of the neural coding underlying expert face recognition. Previous studies reported larger aftereffects for adaptors that are morphed further from the average face than for adaptors closer to the average, which appeared to support opponent coding along face-identity dimensions. However, only two levels were tested and it is not clear where they were located relative to the range of naturally occurring faces. This range is of interest given the functional need of the visual system both to produce good discrimination of real everyday faces and to process novel kinds of faces that we may encounter. Here, Experiment 1 establishes the boundary of faces judged as being able to occur in everyday life. Experiment 2 then shows that aftereffects increase with adaptor extremity up to this natural-range boundary, drop significantly immediately outside the boundary, and then remain stable with no drop towards zero even for highly distorted adaptors far beyond the boundary. Computational modelling shows that this unexpected pattern cannot be explained either by a simple opponent or by a classic multichannel model. However, its qualitative features can be captured either by a combination of opponent and multichannel coding (raising the possibility that not all identity-related face dimensions are opponent coded), or by a 3-pool model containing two S-shaped-response channels and a central bell-shaped channel around the average face (raising the possibility of unexpected similarities with coding of eye and head direction).

Does the size-illusion effect on prehensile movements depend on preview duration for visuomotor process?

Given that visual estimation of an object's size is affected by an illusory figure, the present study investigates the Ebbinghaus size-illusion effect on visuomotor performance within different preview durations for viewing an object (no preview, 300, 700, 1500, and 3000 ms) before initiating the movement. Twenty participants performed the following actions: (a) grasping the object and (b) matching the perceived object size with the finger aperture configuration as in the grasping task. The illusion affected the grasping aperture size only in the no- and 300-ms preview durations, while the matching aperture was affected across all preview conditions. These results suggest that the preview duration influences the size illusion to affect the grasping performance, and subjects adopt different visuomotor processes, depending on preview duration.

Visual Word Expertise: A Study of Inversion and the Word-Length Effect, with Perceptual Transforms

The word-length effect may indicate whether reading is proceeding in an efficient whole-word fashion or by serial letter processing. If it is an index of an orientation-dependent expert reading mechanism, then it should show an inversion effect, with a large difference between upright and upside-down text that is specific for normally configured text. We measured response time of healthy subjects reading 3- to 9-letter words presented in normal configuration, in mirror reflection or spelt backward, in either upright or inverted orientation. The word-length effect showed an inversion effect specific for normal text, as it was not seen for either backward or mirrored text, a result that differed from that for simple mean response times. Also, the word-length effect was smaller for backward than for mirrored text, suggesting that reading of transformed text uses primarily local letters rather than global word forms. We conclude that the word-length effect is a suitable index of expert reading, and reveals that reading under perceptually difficult conditions relies on a sublexical letter-based strategy.

Mapping Face Recognition Information Use across Cultures

Face recognition is not rooted in a universal eye movement information-gathering strategy. Western observers favor a local facial feature sampling strategy, whereas Eastern observers prefer sampling face information from a global, central fixation strategy. Yet, the precise qualitative (the diagnostic) and quantitative (the amount) information underlying these cultural perceptual biases in face recognition remains undetermined. To this end, we monitored the eye movements of Western and Eastern observers during a face recognition task, with a novel gaze-contingent technique: the Expanding Spotlight. We used 2° Gaussian apertures centered on the observers' fixations expanding dynamically at a rate of 1° every 25 ms at each fixation - the longer the fixation duration, the larger the aperture size. Identity-specific face information was only displayed within the Gaussian aperture; outside the aperture, an average face template was displayed to facilitate saccade planning. Thus, the Expanding Spotlight simultaneously maps out the facial information span at each fixation location. Data obtained with the Expanding Spotlight technique confirmed that Westerners extract more information from the eye region, whereas Easterners extract more information from the nose region. Interestingly, this quantitative difference was paired with a qualitative disparity. Retinal filters based on spatial-frequency decomposition built from the fixations maps revealed that Westerners used local high-spatial-frequency information sampling, covering all the features critical for effective face recognition (the eyes and the mouth). In contrast, Easterners achieved a similar result by using global low-spatial-frequency information from those facial features. Our data show that the face system flexibly engages into local or global eye movement strategies across cultures, by relying on distinct facial information span and culturally tuned spatially filtered information. Overall, our findings challenge the view of a unique putative process for face recognition.

Visual scan paths and recognition of facial identity in autism spectrum disorder and typical development

Background

Previous research suggests that many individuals with autism spectrum disorder (ASD) have impaired facial identity recognition, and also exhibit abnormal visual scanning of faces. Here, two hypotheses accounting for an association between these observations were tested: i) better facial identity recognition is associated with increased gaze time on the Eye region; ii) better facial identity recognition is associated with increased eye-movements around the face.

Methodology and Principal Findings

Eye-movements of 11 children with ASD and 11 age-matched typically developing (TD) controls were recorded whilst they viewed a series of faces, and then completed a two alternative forced-choice recognition memory test for the faces. Scores on the memory task were standardized according to age. In both groups, there was no evidence of an association between the proportion of time spent looking at the Eye region of faces and age-standardized recognition performance, thus the first hypothesis was rejected. However, the ‘Dynamic Scanning Index’ – which was incremented each time the participant saccaded into and out of one of the core-feature interest areas – was strongly associated with age-standardized face recognition scores in both groups, even after controlling for various other potential predictors of performance.

Conclusions and Significance

In support of the second hypothesis, results suggested that increased saccading between core-features was associated with more accurate face recognition ability, both in typical development and ASD. Causal directions of this relationship remain undetermined.

Early deafness increases the face inversion effect but does not modulate the composite face effect

Early deprivation in audition can have striking effects on the development of visual processing. Here we investigated whether early deafness induces changes in holistic/configural face processing. To this end, we compared the results of a group of early deaf participants to those of a group of hearing participants in an inversion-matching task (Experiment 1) and a composite face task (Experiment 2). We hypothesized that deaf individuals would show an enhanced inversion effect and/or an increased composite face effect compared to hearing controls in case of enhanced holistic/configural face processing. Conversely, these effects would be reduced if they rely more on facial features than hearing controls. As a result, we found that deaf individuals showed an increased inversion effect for faces, but not for non-face objects. They were also significantly slower than hearing controls to match inverted faces. However, the two populations did not differ regarding the overall size of their composite face effect. Altogether these results suggest that early deafness does not enhance or reduce the amount of holistic/configural processing devoted to faces but may increase the dependency on this mode of processing.

Stable modality-specific activity flows as reflected by the neuroenergetic approach to the FMRI weighted maps

This article uses the ideas of neuroenergetic and neural field theories to detect stimulation-driven energy flows in the brain during face and auditory word processing. In this analysis, energy flows are thought to create the stable gradients of the fMRI weighted summary images. The sources, from which activity spreads in the brain during face processing, were detected in the occipital cortex. The following direction of energy flows in the frontal cortex was described: the right inferior frontal = >the left inferior frontal = >the triangular part of the left inferior frontal cortex = >the left operculum. In the left operculum, a localized circuit was described. For auditory word processing, the sources of activity flows were detected bilaterally in the middle superior temporal regions, they were also detected in the left posterior superior temporal cortex. Thus, neuroenergetic assumptions may give a novel perspective for the analysis of neuroimaging data.

Recognising Faces Seen Alone or with Others: Why are Two Heads Worse Than One?

The ability to identify an unfamiliar target face from an identity lineup declines when the target is accompanied by a second face during visual encoding. This two-face disadvantage is still little studied and its basis remains poorly understood. We investigated several possible explanations for this phenomenon. Experiments 1 and 2 varied the number of potential targets (1 or 2) and the number of faces in a lineup (5 or 10) to explore if this effect arises from the number of identity comparisons that need to be made to detect a target in a lineup. We also explored if this effect arises from an uncertainty concerning which is the to-be-identified target in two-face displays, by cueing the relevant face during encoding. In experiment 3 we then examined whether the two-face disadvantage reflects the depth of face encoding or a memory effect. The results show that this effect arises from the additional comparisons that are necessary to compare two potential targets to an identity lineup when memory demands are minimised (experiment 1), but it reflects a difficulty in remembering several faces when targets and lineups cannot be viewed simultaneously (experiments 2 and 3). However, in both cases the two-face disadvantage could not be eliminated fully by cueing the target. This hints at a further possible locus for this effect, which might reflect perceptual interference during the initial encoding of the target. The implications of these findings are discussed.

Experience influences elemental and configural perception of certain binary odour mixtures in newborn rabbits

Elemental and configural olfactory perception allows interaction with the environment from very early in life. To evaluate how newborn rabbits can extract and respond to information from the highly complex chemical surroundings, and how experience acts on this sensory, cognitive and behavioural capability, we ran a study in four steps including a total of eight experiments. We mainly used a binary AB mixture comprising ethyl isobutyrate (component A) and ethyl maltol (component B), previously shown as a bearer of blending properties; in rabbit pups (as in human adults), the mixture elicits a weak configural perception, i.e. the perception of a configural odour different from the odours of the components. First, a repeated exposure to one component of AB led to a more elemental perception of this mixture; conversely, a repeated exposure to AB facilitated its configural processing. Second, similar impact of experience did not appear with a non-blending AC mixture (ethyl isobutyrate-guaïacol). Third, repeated exposure to AB impacted not only the perception of AB, but also and in the same way the perception of the AC mixture sharing one component, and reciprocally. However, facilitation to perceive one mixture in one mode (configural/elemental) was not generalized to a mixture sharing no components with the experienced mixture [AB versus DE (damascenone and vanillin)]. Thus, experience contributes to the neonatal perception of odour mixtures and adds plasticity to the perceptual system. However, this impact remains dependent on the chemical composition of the mixtures.

Early (n170/m170) face-sensitivity despite right lateral occipital brain damage in acquired prosopagnosia

Compared to objects, pictures of faces elicit a larger early electromagnetic response at occipito-temporal sites on the human scalp, with an onset of 130 ms and a peak at about 170 ms. This N170 face effect is larger in the right than the left hemisphere and has been associated with the early categorization of the stimulus as a face. Here we tested whether this effect can be observed in the absence of some of the visual areas showing a preferential response to faces as typically identified in neuroimaging. Event-related potentials were recorded in response to faces, cars, and their phase-scrambled versions in a well-known brain-damaged case of prosopagnosia (PS). Despite the patient’s right inferior occipital gyrus lesion encompassing the most posterior cortical area showing preferential response to faces (“occipital face area”), we identified an early face-sensitive component over the right occipito-temporal hemisphere of the patient that was identified as the N170. A second experiment supported this conclusion, showing the typical N170 increase of latency and amplitude in response to inverted faces. In contrast, there was no N170 in the left hemisphere, where PS has a lesion to the middle fusiform gyrus and shows no evidence of face-preferential response in neuroimaging (no left “fusiform face area”). These results were replicated by a magnetoencephalographic investigation of the patient, disclosing a M170 component only in the right hemisphere. These observations indicate that face-preferential activation in the inferior occipital cortex is not necessary to elicit early visual responses associated with face perception (N170/M170) on the human scalp. These results further suggest that when the right inferior occipital cortex is damaged, the integrity of the middle fusiform gyrus and/or the superior temporal sulcus – two areas showing face-preferential responses in the patient’s right hemisphere – might be necessary to generate the N170 effect.

Exploring the time course of face matching: temporal constraints impair unfamiliar face identification under temporally unconstrained viewing

The identification of unfamiliar faces has been studied extensively with matching tasks, in which observers decide if pairs of photographs depict the same person (identity matches) or different people (mismatches). In experimental studies in this field, performance is usually self-paced under the assumption that this will encourage best-possible accuracy. Here, we examined the temporal characteristics of this task by limiting display times and tracking observers' eye movements. Observers were required to make match/mismatch decisions to pairs of faces shown for 200, 500, 1000, or 2000ms, or for an unlimited duration. Peak accuracy was reached within 2000ms and two fixations to each face. However, intermixing exposure conditions produced a context effect that generally reduced accuracy on identity mismatch trials, even when unlimited viewing of faces was possible. These findings indicate that less than 2s are required for face matching when exposure times are variable, but temporal constraints should be avoided altogether if accuracy is truly paramount. The implications of these findings are discussed.

Implicit attitudes in prosopagnosia

We studied a male with acquired prosopagnosia using a battery of Implicit Association Tests (IATs) to investigate whether observing faces varying by social category would activate the patient's implicit social biases. We also asked him to categorize faces explicitly by race, gender, and political party. The patient, G.B., was marginally slower to categorize black compared to white faces. He showed congruency effects in the race and celebrity IATs, but not in the gender or political IATs. These results indicate that G.B. possesses an implicit social sensitivity to certain facial stimuli despite an inability to overtly recognize familiar faces. The results demonstrate that social biases can be retrieved based on facial stimuli via pathways bypassing the fusiform gyri. Thus the IAT effect can be added to the list of covert recognition effects found in prosopagnosia.

Can changes in eye movement scanning alter the age-related deficit in recognition memory?

Older adults typically exhibit poorer face recognition compared to younger adults. These recognition differences may be due to underlying age-related changes in eye movement scanning. We examined whether older adults' recognition could be improved by yoking their eye movements to those of younger adults. Participants studied younger and older faces, under free viewing conditions (bases), through a gaze-contingent moving window (own), or a moving window which replayed the eye movements of a base participant (yoked). During the recognition test, participants freely viewed the faces with no viewing restrictions. Own-age recognition biases were observed for older adults in all viewing conditions, suggesting that this effect occurs independently of scanning. Participants in the bases condition had the highest recognition accuracy, and participants in the yoked condition were more accurate than participants in the own condition. Among yoked participants, recognition did not depend on age of the base participant. These results suggest that successful encoding for all participants requires the bottom-up contribution of peripheral information, regardless of the locus of control of the viewer. Although altering the pattern of eye movements did not increase recognition, the amount of sampling of the face during encoding predicted subsequent recognition accuracy for all participants. Increased sampling may confer some advantages for subsequent recognition, particularly for people who have declining memory abilities.

Disorder of higher visual function

PURPOSE OF REVIEW

Both monkey and human neuroimaging studies show that visual processing beyond the striate cortex involves a highly complex network of regions with modular functions. Lesions within this network lead to specific clinical syndromes. In this review we discuss studies on blindsight, which is the ability of remaining regions to support vision in the absence of striate cortex or visual awareness, recent work on 'ventral stream' syndromes such as object agnosia, alexia, prosopagnosia, and topographagnosia, which follow damage to medial occipitotemporal structures, and simultanagnosia, the classic 'dorsal stream' deficit related to bilateral occipitoparietal lesions.

RECENT FINDINGS

We highlight work on the anatomic basis of blindsight, the recent description of the new disorder developmental topographic disorientation, and studies contrasting global and local perception in simultanagnosia.

SUMMARY

These studies advance our understanding of the mechanisms of complex visual processing and provide an important neuropsychological complement to our expanding knowledge about vision from functional neuroimaging.

Holistic face categorization in higher order visual areas of the normal and prosopagnosic brain: toward a non-hierarchical view of face perception

How a visual stimulus is initially categorized as a face in a network of human brain areas remains largely unclear. Hierarchical neuro-computational models of face perception assume that the visual stimulus is first decomposed in local parts in lower order visual areas. These parts would then be combined into a global representation in higher order face-sensitive areas of the occipito-temporal cortex. Here we tested this view in fMRI with visual stimuli that are categorized as faces based on their global configuration rather than their local parts (two-tones Mooney figures and Arcimboldo's facelike paintings). Compared to the same inverted visual stimuli that are not categorized as faces, these stimuli activated the right middle fusiform gyrus (“Fusiform face area”) and superior temporal sulcus (pSTS), with no significant activation in the posteriorly located inferior occipital gyrus (i.e., no “occipital face area”). This observation is strengthened by behavioral and neural evidence for normal face categorization of these stimuli in a brain-damaged prosopagnosic patient whose intact right middle fusiform gyrus and superior temporal sulcus are devoid of any potential face-sensitive inputs from the lesioned right inferior occipital cortex. Together, these observations indicate that face-preferential activation may emerge in higher order visual areas of the right hemisphere without any face-preferential inputs from lower order visual areas, supporting a non-hierarchical view of face perception in the visual cortex.

Holistic perception of the individual face is specific and necessary: evidence from an extensive case study of acquired prosopagnosia

We present an extensive investigation (24 experiments) of a new case of prosopagnosia following right unilateral damage, GG, with the aim of addressing two classical issues: (1) Can a visual recognition impairment truly be specific to faces? (2) What is the nature of acquired prosopagnosia? We show that GG recognizes nonface objects perfectly and quickly, even when it requires fine-grained analysis to individualize these objects. He is also capable of perceiving objects and faces as integrated wholes, as indicated by normal Navon effect, 3D-figures perception and perception of Mooney and Arcimboldo face stimuli. However, the patient could not perceive individual faces holistically, showing no inversion, composite, or whole-part advantage effects for faces. We conclude that an occipito-temporal right hemisphere lesion may lead to a specific impairment of holistic perception of individual items, a function that appears critical for normal face recognition but not for object recognition.

Fixation patterns during recognition of personally familiar and unfamiliar faces

Previous studies recording eye gaze during face perception have rendered somewhat inconclusive findings with respect to fixation differences between familiar and unfamiliar faces. This can be attributed to a number of factors that differ across studies: the type and extent of familiarity with the faces presented, the definition of areas of interest subject to analyses, as well as a lack of consideration for the time course of scan patterns. Here we sought to address these issues by recording fixations in a recognition task with personally familiar and unfamiliar faces. After a first common fixation on a central superior location of the face in between features, suggesting initial holistic encoding, and a subsequent left eye bias, local features were focused and explored more for familiar than unfamiliar faces. Although the number of fixations did not differ for un-/familiar faces, the locations of fixations began to differ before familiarity decisions were provided. This suggests that in the context of familiarity decisions without time constraints, differences in processing familiar and unfamiliar faces arise relatively early – immediately upon initiation of the first fixation to identity-specific information – and that the local features of familiar faces are processed more than those of unfamiliar faces.