Information Processing during Face Recognition: The Effects of Familiarity, Inversion, and Morphing on Scanning Fixations

Holistic gaze strategy to categorize facial expression of varying intensities

Using faces representing exaggerated emotional expressions, recent behaviour and eye-tracking studies have suggested a dominant role of individual facial features in transmitting diagnostic cues for decoding facial expressions. Considering that in everyday life we frequently view low-intensity expressive faces in which local facial cues are more ambiguous, we probably need to combine expressive cues from more than one facial feature to reliably decode naturalistic facial affects. In this study we applied a morphing technique to systematically vary intensities of six basic facial expressions of emotion, and employed a self-paced expression categorization task to measure participants' categorization performance and associated gaze patterns. The analysis of pooled data from all expressions showed that increasing expression intensity would improve categorization accuracy, shorten reaction time and reduce number of fixations directed at faces. The proportion of fixations and viewing time directed at internal facial features (eyes, nose and mouth region), however, was not affected by varying levels of intensity. Further comparison between individual facial expressions revealed that although proportional gaze allocation at individual facial features was quantitatively modulated by the viewed expressions, the overall gaze distribution in face viewing was qualitatively similar across different facial expressions and different intensities. It seems that we adopt a holistic viewing strategy to extract expressive cues from all internal facial features in processing of naturalistic facial expressions.

Start position strongly influences fixation patterns during face processing: difficulties with eye movements as a measure of information use

Fixation patterns are thought to reflect cognitive processing and, thus, index the most informative stimulus features for task performance. During face recognition, initial fixations to the center of the nose have been taken to indicate this location is optimal for information extraction. However, the use of fixations as a marker for information use rests on the assumption that fixation patterns are predominantly determined by stimulus and task, despite the fact that fixations are also influenced by visuo-motor factors. Here, we tested the effect of starting position on fixation patterns during a face recognition task with upright and inverted faces. While we observed differences in fixations between upright and inverted faces, likely reflecting differences in cognitive processing, there was also a strong effect of start position. Over the first five saccades, fixation patterns across start positions were only coarsely similar, with most fixations around the eyes. Importantly, however, the precise fixation pattern was highly dependent on start position with a strong tendency toward facial features furthest from the start position. For example, the often-reported tendency toward the left over right eye was reversed for the left starting position. Further, delayed initial saccades for central versus peripheral start positions suggest greater information processing prior to the initial saccade, highlighting the experimental bias introduced by the commonly used center start position. Finally, the precise effect of face inversion on fixation patterns was also dependent on start position. These results demonstrate the importance of a non-stimulus, non-task factor in determining fixation patterns. The patterns observed likely reflect a complex combination of visuo-motor effects and simple sampling strategies as well as cognitive factors. These different factors are very difficult to tease apart and therefore great caution must be applied when interpreting absolute fixation locations as indicative of information use, particularly at a fine spatial scale.

Gaze patterns during identity and emotion judgments in hearing adults and deaf users of American Sign Language

Deaf individuals rely on facial expressions for emotional, social, and linguistic cues. In order to test the hypothesis that specialized experience with faces can alter typically observed gaze patterns, twelve hearing adults and twelve deaf, early-users of American Sign Language judged the emotion and identity of expressive faces (including whole faces, and isolated top and bottom halves), while accuracy and fixations were recorded. Both groups recognized individuals more accurately from top than bottom halves, and emotional expressions from bottom than top halves. Hearing adults directed the majority of fixations to the top halves of faces in both tasks, but fixated the bottom half slightly more often when judging emotion than identity. In contrast, deaf adults often split fixations evenly between the top and bottom halves regardless of task demands. These results suggest that deaf adults have habitual fixation patterns that may maximize their ability to gather information from expressive faces.

The role of skin texture and facial shape in representations of age and identity

Faces have both shape and skin texture, but the relative importance of the two in face representations is unclear. Our goals were first, to determine the contribution of shape versus texture to aftereffects for facial age and identity and second, to assess whether adaptation transferred between shape and texture, suggesting integration in a single representation. In our first experiment we examined age aftereffects. We obtained young and old images of two celebrities and created hybrid images, one combining the structure of the old face with the skin texture of the young face, the other combining the young structure with the old skin texture. This allowed us to create adaptation contrasts where the two adapting faces had the same facial structure but different skin texture, and vice versa. In the second experiment, we performed a similar study but this time examining identity aftereffects between two people of a similar age. We found that both skin texture and facial shape generated significant age aftereffects, but the contribution was greater from texture than from shape. Both texture and shape also generated significant identity aftereffects, but the contribution was greater from shape than from texture. In the last experiment, we used the normal and hybrid images to determine if adaptation to one property (i.e., texture) could create aftereffects in the perception of age in the other property (i.e., shape). While there was significant within-component adaptation for texture and shape, there was no evidence of cross-component adaptation. We conclude that shape and texture contribute differently to different face representations, with texture dominating for age. The lack of cross-component adaptation transfer suggests independent encoding of shape and texture, at least for age representations.

The effects of prior exposure on face processing in younger and older adults

Older adults differ from their younger counterparts in the way they view faces. We assessed whether older adults can use past experience to mitigate these typical face-processing differences; that is, we examined whether there are age-related differences in the use of memory to support current processing. Eye movements of older and younger adults were monitored as they viewed faces that varied in the type/amount of prior exposure. Prior exposure was manipulated by including famous and novel faces, and by presenting faces up to five times. We expected that older adults may have difficulty quickly establishing new representations to aid in the processing of recently presented faces, but would be able to invoke face representations that have been stored in memory long ago to aid in the processing of famous faces. Indeed, younger adults displayed effects of recent exposure with a decrease in the total fixations to the faces and a gradual increase in the proportion of fixations to the eyes. These effects of recent exposure were largely absent in older adults. In contrast, the effect of fame, revealed by a subtle increase in fixations to the inner features of famous compared to non-famous faces, was similar for younger and older adults. Our results suggest that older adults’ current processing can benefit from lifetime experience, however the full benefit of recent experience on face processing is not realized in older adults.

Exploring the time course of face matching: temporal constraints impair unfamiliar face identification under temporally unconstrained viewing

The identification of unfamiliar faces has been studied extensively with matching tasks, in which observers decide if pairs of photographs depict the same person (identity matches) or different people (mismatches). In experimental studies in this field, performance is usually self-paced under the assumption that this will encourage best-possible accuracy. Here, we examined the temporal characteristics of this task by limiting display times and tracking observers' eye movements. Observers were required to make match/mismatch decisions to pairs of faces shown for 200, 500, 1000, or 2000ms, or for an unlimited duration. Peak accuracy was reached within 2000ms and two fixations to each face. However, intermixing exposure conditions produced a context effect that generally reduced accuracy on identity mismatch trials, even when unlimited viewing of faces was possible. These findings indicate that less than 2s are required for face matching when exposure times are variable, but temporal constraints should be avoided altogether if accuracy is truly paramount. The implications of these findings are discussed.

Through the eyes of the own-race bias: eye-tracking and pupillometry during face recognition

People are generally better at remembering faces of their own race than faces of a different race, and this effect is known as the own-race bias (ORB) effect. We used eye-tracking and pupillometry to investigate whether Caucasian and Asian face stimuli elicited different-looking patterns in Caucasian participants in a face-memory task. Consistent with the ORB effect, we found better recognition performance for own-race faces than other-race faces, and shorter response times. In addition, at encoding, eye movements and pupillary responses to Asian faces (i.e., the other race) were different from those to Caucasian faces (i.e., the own race). Processing of own-race faces was characterized by more active scanning, with a larger number of shorter fixations, and more frequent saccades. Moreover, pupillary diameters were larger when viewing other-race than own-race faces, suggesting a greater cognitive effort when encoding other-race faces.

Irrelevant objects of expertise compete with faces during visual search

Prior work suggests that nonface objects of expertise can interfere with the perception of faces when the two categories are alternately presented, suggesting competition for shared perceptual resources. Here, we ask whether task-irrelevant distractors from a category of expertise compete when faces are presented in a standard visual search task. Participants searched for a target (face or sofa) in an array containing both relevant and irrelevant distractors. The number of distractors from the target category (face or sofa) remained constant, whereas the number of distractors from the irrelevant category (cars) varied. Search slopes, calculated as a function of the number of irrelevant cars, were correlated with car expertise. The effect was not due to car distractors grabbing attention, because they did not compete with sofa targets. Objects of expertise interfere with face perception even when they are task irrelevant, visually distinct, and separated in space from faces.

Deidentification of facial images using composites

PURPOSE

Maxillofacial surgeons rely on photography for education and documentation. Photographs of the face, unlike those of other body regions, are readily identifiable. Traditional methods of facial image deidentification decrease educational quality or fail to adequately conceal identity. In the present study, a method that uses blended facial composites to deidentify original facial images was developed. This method allows significant components of the original face to be visualized while concealing its identity.

MATERIALS AND METHODS

The method was used to develop 20 different composite facial images that were viewed by student subjects. Ten of these images contained at least one third of a face that was familiar to the subjects. Subjects viewed the composite faces twice--first unaware that the faces were composites, and then primed to the presence of composites. Subjects later rated the efficacy of this method for image deidentification.

RESULTS

When unaware that they were viewing composite images, no subjects recognized the familiar faces within the composites or rated them as familiar (0/120 total views, 0%). When later primed to the potential presence of familiar faces within composites, the identification rate increased significantly (74/120, 62%; P < .001). Results were similar no matter which portion of the familiar face (upper, 67%; mid, 54%; lower, 67%) was present. Subjects rated all composites as clinically realistic patient images. They also rated composites as more effective at deidentification than traditional methods.

CONCLUSION

The use of composites appears to be a promising concept for facial image deidentification. Further larger-scale studies are needed to validate these findings.

Consistent left gaze bias in processing different facial cues

While viewing faces, humans often demonstrate a natural gaze bias towards the left visual field, that is, the right side of the viewee's face is often inspected first and for longer periods. Previous studies have suggested that this gaze asymmetry is a part of the gaze pattern associated with face exploration, but its relation with perceptual processing of facial cues is unclear. In this study we recorded participants' saccadic eye movements while exploring face images under different task instructions (free viewing, judging familiarity and judging facial expression). We observed a consistent left gaze bias in face viewing irrespective of task demands. The probability of the first fixation and the proportion of overall fixations directed at the left hemiface were indistinguishable across different task instructions or across different facial expressions. It seems that the left gaze bias is an automatic reflection of hemispheric lateralisation in face processing, and is not necessarily correlated with the perceptual processing of a specific type of facial information.

Inverting faces does not abolish cultural diversity in eye movements

Face processing is widely understood to be a basic, universal visual function effortlessly achieved by people from all cultures and races. The remarkable recognition performance for faces is markedly and specifically affected by picture-plane inversion: the so-called face-inversion effect (FIE), a finding often used as evidence for face-specific mechanisms. However, it has recently been shown that culture shapes the way people deploy eye movements to extract information from faces. Interestingly, the comparable lack of experience with inverted faces across cultures offers a unique opportunity to establish the extent to which such cultural perceptual biases in eye movements are robust, but also to assess whether face-specific mechanisms are universally tuned. Here we monitored the eye movements of Western Caucasian (WC) and East Asian (EA) observers while they learned and recognised WC and EA inverted faces. Both groups of observers showed a comparable impairment in recognising inverted faces of both races. WC observers deployed a scattered inverted triangular scanpath with a bias towards the mouth, whereas EA observers uniformly extended the focus of their fixations from the centre towards the eyes. Overall, our data show that cultural perceptual differences in eye movements persist during the FIE, questioning the universality of face-processing mechanisms.

Preliminary Evidence that the Limbal Ring Influences Facial Attractiveness

The limbal ring of the eye appears as a dark annulus where the iris meets the sclera. Both width and opacity of the limbal ring are influenced by iris pigmentation and optical properties of the region. With age the limbal ring becomes less prominent, making it a probabilistic indicator of youth and health. This raises the question: Are judgments of facial attractiveness sensitive to this signal in a potentially adaptive way? Here we show that the answer is yes. For male and female observers, both male and female faces with a dark and distinct limbal ring are rated as more attractive than otherwise identical faces with no limbal ring. This result is observed not just for upright faces but also for inverted faces, suggesting that the limbal ring is processed primarily as a local feature rather than as a configural feature in the analysis of facial beauty. We also discuss directions for future research that can clarify the role of the limbal ring in the visual perception of facial attractiveness.

Human spontaneous gaze patterns in viewing of faces of different species

Clear differences in perceptual and neural processing of faces of different species have been reported, implying the contribution of visual experience to face perception. Can these differences be revealed by our eye scanning patterns while we extract salient facial information? Here, we systematically compared non-pet-owners' gaze patterns while exploring human, monkey, dog, and cat faces in a passive viewing task. Our analysis revealed that the faces of different species induced similar patterns of fixation distribution between left and right hemiface, and among key local facial features, with the eyes attracting the highest proportion of fixations and viewing times, followed by the nose, and then the mouth. Only the proportion of fixations directed at the mouth region was species-dependent and could be differentiated at the earliest stage of face viewing. Our spontaneous eye-scanning patterns associated with face exploration appear to have been mainly constrained by general facial configurations; the species affiliation of the inspected faces had limited impact on gaze allocation, at least under free-viewing conditions.

Fixation patterns during recognition of personally familiar and unfamiliar faces

Previous studies recording eye gaze during face perception have rendered somewhat inconclusive findings with respect to fixation differences between familiar and unfamiliar faces. This can be attributed to a number of factors that differ across studies: the type and extent of familiarity with the faces presented, the definition of areas of interest subject to analyses, as well as a lack of consideration for the time course of scan patterns. Here we sought to address these issues by recording fixations in a recognition task with personally familiar and unfamiliar faces. After a first common fixation on a central superior location of the face in between features, suggesting initial holistic encoding, and a subsequent left eye bias, local features were focused and explored more for familiar than unfamiliar faces. Although the number of fixations did not differ for un-/familiar faces, the locations of fixations began to differ before familiarity decisions were provided. This suggests that in the context of familiarity decisions without time constraints, differences in processing familiar and unfamiliar faces arise relatively early – immediately upon initiation of the first fixation to identity-specific information – and that the local features of familiar faces are processed more than those of unfamiliar faces.

The Thatcher illusion in humans and monkeys

Primates possess the remarkable ability to differentiate faces of group members and to extract relevant information about the individual directly from the face. Recognition of conspecific faces is achieved by means of holistic processing, i.e. the processing of the face as an unparsed, perceptual whole, rather than as the collection of independent features (part-based processing). The most striking example of holistic processing is the Thatcher illusion. Local changes in facial features are hardly noticeable when the whole face is inverted (rotated 180 degrees ), but strikingly grotesque when the face is upright. This effect can be explained by a lack of processing capabilities for locally rotated facial features when the face is turned upside down. Recently, a Thatcher illusion was described in the macaque monkey analogous to that known from human investigations. Using a habituation paradigm combined with eye tracking, we address the critical follow-up questions raised in the aforementioned study to show the Thatcher illusion as a function of the observer's species (humans and macaques), the stimulus' species (humans and macaques) and the level of perceptual expertise (novice, expert).

Featural, configural, and holistic face-processing strategies evoke different scan patterns

In two experiments we investigated the role of eye movements during face processing. In experiment 1, using modified faces with primarily featural (scrambled faces) or configural (blurred faces) information as cue stimuli, we manipulated the way participants processed subsequently presented intact faces. In a sequential same-different task, participants decided whether the identity of an intact test face matched a preceding scrambled or blurred cue face. Analysis of eye movements for test faces showed more interfeatural saccades when they followed a blurred face, and longer gaze duration within the same feature when they followed scrambled faces. In experiment 2, we used a similar paradigm except that test faces were cued by intact faces, low-level blurred stimuli, or second-order scrambled stimuli (features were cut out but maintained their first-order relations). We found that in the intact condition participants performed fewer interfeatural saccades than in low-level blurred condition and had shorter gaze duration than in second-order scrambled condition. Moreover, participants fixated the centre of the test face to grasp the information from the whole face. Our findings suggest a differentiation between featural, configural, and holistic processing strategies, which can be associated with specific patterns of eye movements.

Covert face recognition relies on affective valence in congenital prosopagnosia

Dominant accounts of covert recognition in prosopagnosia assume subthreshold activation of face representations created prior to onset of the disorder. Yet, such accounts cannot explain covert recognition in congenital prosopagnosia, where the impairment is present from birth. Alternatively, covert recognition may rely on affective valence, yet no study has explored this possibility. The current study addressed this issue in 3 individuals with congenital prosopagnosia, using measures of the scanpath to indicate recognition. Participants were asked to memorize 30 faces paired with descriptions of aggressive, nice, or neutral behaviours. In a later recognition test, eye movements were monitored while participants discriminated studied from novel faces. Sampling was reduced for studied--nice compared to studied--aggressive faces, and performance for studied--neutral and novel faces fell between these two conditions. This pattern of findings suggests that (a) positive emotion can facilitate processing in prosopagnosia, and (b) covert recognition may rely on emotional valence rather than familiarity.

Distinguishing the cause and consequence of face inversion: the perceptual field hypothesis

I published a critical review of the face inversion effect (Rossion, 2008) that triggered a few reactions and commentaries by colleagues in the field (Riesenhuber & Wolff, 2009; Yovel, in press). Here, I summarize my original paper and attempt to identify the source of both the agreements and disagreements with my colleagues, as well as other authors, regarding the nature of the face inversion effect. My view is that the major cause of the detrimental effect of inversion on an observer's performance at individual face recognition is the disruption of a perceptual process. This perceptual process is makes and observer see the multiple features of a whole individual upright face at once. It also makes the percept of a given facial feature highly dependent on the location and identity of the other features in the whole face. The perceptual process is holistic because it is driven by a holistic face representation, derived from visual experience. Hence, an inverted face cannot be perceived holistically: the perceptual field of the observer is constricted for inverted faces, each facial feature having to be processed sequentially, independently, i.e. over a smaller spatial window than the whole face. Consequently, it is particularly difficult to perceive diagnostic cues that involve several elements over a wide space on an inverted face, such as long-range relative distances between features (e.g., relative distance between eyes and mouth), or diagnostic cues that are located far away from usual gaze fixation (e.g., mouth-nose distance or mouth shape when fixating between the eyes). These difficulties are mere consequences of face inversion--the cause being a loss of holistic perception--, and it does not follow that relative distances between internal features are necessarily particularly important to recognize faces, that they should be labeled "configural", or should be given a specific status at the representational level. I argue that distinguishing the cause and consequence(s) of face inversion this way can provide a parsimonious and yet complete theoretical account of the face inversion effect.

Gaze behavior in face comparison: the roles of sex, task, and symmetry

Knowing where people look on a face provides an objective insight into the information entering the visual system and into cognitive processes involved in face perception. In the present study, we recorded eye movements of human participants while they compared two faces presented simultaneously. Observers' viewing behavior and performance was examined in two tasks of parametrically varying difficulty, using two types of face stimuli (sex morphs and identity morphs). The frequency, duration, and temporal sequence of fixations on previously defined areas of interest in the faces were analyzed. As was expected, viewing behavior and performance varied with difficulty. Interestingly, observers compared predominantly the inner halves of the face stimuli-a result inconsistent with the general left-hemiface bias reported for single faces. Furthermore, fixation patterns and performance differed between tasks, independently of stimulus type. Moreover, we found differences in male and female participants' viewing behaviors, but only when the sex of the face stimuli was task relevant.

Taking a real look at social attention

The eyes of others are important to us, and we care about where they are directed. Lab-based studies often fail to capture this intuition because the studies are so simple and controlled that the situational complexity that is critical to social attention is lost. A research approach called cognitive ethology begins its investigation at the level of naturally occurring phenomena before moving into the lab. In doing so one can maintain the link between lab research and the phenomena it seeks to understand. Instances of lab failures that are offset by a cognitive ethology approach are presented and discussed.

Two fixations suffice in face recognition

It is well known that there exist preferred landing positions for eye fixations in visual word recognition. However, the existence of preferred landing positions in face recognition is less well established. It is also unknown how many fixations are required to recognize a face. To investigate these questions, we recorded eye movements during face recognition. During an otherwise standard face-recognition task, subjects were allowed a variable number of fixations before the stimulus was masked. We found that optimal recognition performance is achieved with two fixations; performance does not improve with additional fixations. The distribution of the first fixation is just to the left of the center of the nose, and that of the second fixation is around the center of the nose. Thus, these appear to be the preferred landing positions for face recognition. Furthermore, the fixations made during face learning differ in location from those made during face recognition and are also more variable in duration; this suggests that different strategies are used for face learning and face recognition.

Humans and macaques employ similar face-processing strategies

Primates developed the ability to recognize and individuate their conspecifics by the face. Despite numerous electrophysiological studies in monkeys, little is known about the face-processing strategies that monkeys employ. In contrast, face perception in humans has been the subject of many studies providing evidence for specific face processing that evolves with perceptual expertise. Importantly, humans process faces holistically, here defined as the processing of faces as wholes, rather than as collections of independent features (part-based processing). The question remains to what extent humans and monkeys share these face-processing mechanisms. By using the same experimental design and stimuli for both monkey and human behavioral experiments, we show that face processing is influenced by the species affiliation of the observed face stimulus (human versus macaque face). Furthermore, stimulus manipulations that selectively reduced holistic and part-based information systematically altered eye-scanning patterns for human and macaque observers similarly. These results demonstrate the similar nature of face perception in humans and monkeys and pin down effects of expert face-processing versus novice face-processing strategies. These findings therefore directly contribute to one of the central discussions in the behavioral and neurosciences about how faces are perceived in primates.

The surprisingly high human efficiency at learning to recognize faces

We investigated the ability of humans to optimize face recognition performance through rapid learning of individual relevant features. We created artificial faces with discriminating visual information heavily concentrated in single features (nose, eyes, chin or mouth). In each of 2500 learning blocks a feature was randomly selected and retained over the course of four trials, during which observers identified randomly sampled, noisy face images. Observers learned the discriminating feature through indirect feedback, leading to large performance gains. Performance was compared to a learning Bayesian ideal observer, resulting in unexpectedly high learning compared to previous studies with simpler stimuli. We explore various explanations and conclude that the higher learning measured with faces cannot be driven by adaptive eye movement strategies but can be mostly accounted for by suboptimalities in human face discrimination when observers are uncertain about the discriminating feature. We show that an initial bias of humans to use specific features to perform the task even though they are informed that each of four features is equally likely to be the discriminatory feature would lead to seemingly supra-optimal learning. We also examine the possibility of inefficient human integration of visual information across the spatially distributed facial features. Together, the results suggest that humans can show large performance improvement effects in discriminating faces as they learn to identify the feature containing the discriminatory information.

The effect of varying talker identity and listening conditions on gaze behavior during audiovisual speech perception

During face-to-face conversation the face provides auditory and visual linguistic information, and also conveys information about the identity of the speaker. This study investigated behavioral strategies involved in gathering visual information while watching talking faces. The effects of varying talker identity and varying the intelligibility of speech (by adding acoustic noise) on gaze behavior were measured with an eyetracker. Varying the intelligibility of the speech by adding noise had a noticeable effect on the location and duration of fixations. When noise was present subjects adopted a vantage point that was more centralized on the face by reducing the frequency of the fixations on the eyes and mouth and lengthening the duration of their gaze fixations on the nose and mouth. Varying talker identity resulted in a more modest change in gaze behavior that was modulated by the intelligibility of the speech. Although subjects generally used similar strategies to extract visual information in both talker variability conditions, when noise was absent there were more fixations on the mouth when viewing a different talker every trial as opposed to the same talker every trial. These findings provide a useful baseline for studies examining gaze behavior during audiovisual speech perception and perception of dynamic faces.

Picture-plane inversion leads to qualitative changes of face perception

Presenting a face stimulus upside-down generally causes a larger deficit in perceiving metric distances between facial features ("configuration") than local properties of these features. This effect supports a qualitative account of face inversion: the same transformation affects the processing of different kinds of information differently. However, this view has been recently challenged by studies reporting equal inversion costs of performance for discriminating featural and configural manipulations on faces. In this paper I argue that these studies did not replicate previous results due to methodological factors rather than largely irrelevant parameters such as having equal performance for configural and featural conditions at upright orientation, or randomizing trials across conditions. I also argue that identifying similar diagnostic features (eyes and eyebrows) for discriminating individual faces at upright and inverted orientations by means of response classification methods does not dismiss at all the qualitative view of face inversion. Considering these elements as well as both behavioral and neuropsychological evidence, I propose that the generally larger effect of inversion for processing configural than featural cues is a mere consequence of the disruption of holistic face perception. That is, configural relations necessarily involve two or more distant features on the face, such that their perception is most dependent on the ability to perceive simultaneously multiple features of a face as a whole.

Sensitivity to feature displacement in familiar and unfamiliar faces: beyond the internal/external feature distinction

Previous studies of face recognition and of face matching have shown a general improvement for the processing of internal features as a face becomes more familiar to the participant. In this study, we used a psychophysical two-alternative forced-choice paradigm to investigate thresholds for the detection of a displacement of the eyes, nose, mouth, or ears for familiar and unfamiliar faces. No clear division between internal and external features was observed. Rather, for familiar (compared to unfamiliar) faces participants were more sensitive to displacements of internal features such as the eyes or the nose; yet, for our third internal feature-the mouth no such difference was observed. Despite large displacements, many subjects were unable to perform above chance when stimuli involved shifts in the position of the ears. These results are consistent with the proposal that familiarity effects may be mediated by the construction of a robust representation of a face, although the involvement of attention in the encoding of face stimuli cannot be ruled out. Furthermore, these effects are mediated by information from a spatial configuration of features, rather than by purely feature-based information.

The effects of face inversion and contrast-reversal on efficiency and internal noise

Inverted and contrast-reversed faces are identified less accurately and less rapidly than normal, upright faces. The effects of inversion and contrast-reversal may reflect different sampling strategies and/or different levels of internal noise. To test these alternative hypotheses, we used a combination of noise-masking and response-consistency techniques to measure the internal noise and high-noise efficiency associated with the identification of upright, inverted, and contrast-reversed faces. We found that both face inversion and contrast-reversal reduced efficiency, but did not change internal noise.

Factors contributing to the adaptation aftereffects of facial expression

Previous studies have demonstrated the existence of adaptation aftereffects for facial expressions. Here we investigated which aspects of facial stimuli contribute to these aftereffects. In Experiment 1, we examined the role of local adaptation to image elements such as curvature, shape and orientation, independent of expression, by using hybrid faces constructed from either the same or opposing expressions. While hybrid faces made with consistent expressions generated aftereffects as large as those with normal faces, there were no aftereffects from hybrid faces made from different expressions, despite the fact that these contained the same local image elements. In Experiment 2, we examined the role of facial features independent of the normal face configuration by contrasting adaptation with whole faces to adaptation with scrambled faces. We found that scrambled faces also generated significant aftereffects, indicating that expressive features without a normal facial configuration could generate expression aftereffects. In Experiment 3, we examined the role of facial configuration by using schematic faces made from line elements that in isolation do not carry expression-related information (e.g. curved segments and straight lines) but that convey an expression when arranged in a normal facial configuration. We obtained a significant aftereffect for facial configurations but not scrambled configurations of these line elements. We conclude that facial expression aftereffects are not due to local adaptation to image elements but due to high-level adaptation of neural representations that involve both facial features and facial configuration.

Scan patterns during the processing of facial identity in prosopagnosia

The scan patterns of ocular fixations made by prosopagnosic patients while they attempt to identify faces may provide insights into how they process the information in faces. Contrasts between their scanning of upright versus inverted faces may index the presence of a hypothesized orientation-dependent expert mechanism for processing faces, while contrasts between their scanning of familiar versus novel faces may index the influence of residual facial memories on their search for meaningful facial information. We recorded the eye movements of two prosopagnosics while they viewed faces. One patient, with acquired prosopagnosia from a right occipitotemporal lesion, showed degraded orientation effects but still with a normal distribution of fixations to more salient facial features. However, the dynamics of his global scan patterns were more chaotic for novel faces, suggesting degradation of an internal facial schema, and consistent with other evidence of impaired face configuration perception in this patient. His global scan patterns for famous faces differed from novel faces, suggesting the influence of residual facial memories, as indexed previously by his relatively good imagery for famous faces. The other patient, with a developmental prosopagnosia, showed anomalous orientation effects, abnormal distribution of fixations to less salient regions, and chaotic global scan patterns, in keeping with a more severe loss of face-expert mechanisms. The effects of fame on her scanning were weaker than those in the first subject and non-existent in her global scan patterns. We conclude that scan patterns in prosopagnosia can both reflect the loss of orientation-dependent expert mechanisms and index the covert influence of residual facial memories. In these two subjects the scanning data were consistent with other results from tests of configuration perception, imagery, and covert recognition.