The effects of spatial frequency overlap on face recognition

An update of the Benton Facial Recognition Test

The Benton Facial Recognition Test (BFRT) is a paper-and-pen task that is traditionally used to assess face perception skills in neurological, clinical and psychiatric conditions. Despite criticisms of its stimuli, the task enjoys a simple procedure and is rapid to administer. Further, it has recently been computerised (BFRT-c), allowing reliable measurement of completion times and the need for online testing. Here, in response to calls for repeat screening for the accurate detection of face processing deficits, we present the BFRT-Revised (BFRT-r): a new version of the BFRT-c that maintains the task's basic paradigm, but employs new, higher-quality stimuli that reflect recent theoretical advances in the field. An initial validation study with typical participants indicated that the BFRT-r has good internal reliability and content validity. A second investigation indicated that while younger and older participants had comparable accuracy, completion times were longer in the latter, highlighting the need for age-matched norms. Administration of the BFRT-r and BFRT-c to 32 individuals with developmental prosopagnosia resulted in improved sensitivity in diagnostic screening for the BFRT-r compared to the BFRT-c. These findings are discussed in relation to current diagnostic screening protocols for face perception deficits. The BFRT-r is stored in an open repository and is freely available to other researchers.

Face Recognition in Pictures is Affected by Perspective Transformation but Not by the Centre of Projection

Recognition of unfamiliar faces is susceptible to image differences caused by angular sizes subtended from the face to the camera. Research on perception of cubes suggests that apparent distortions of a shape due to large camera angle are correctable by placing the observer at the centre of projection, especially when visibility of the picture surface is low (Yang and Kubovy, 1999 Perception & Psychophysics61 456 – 467). To explore the implication of this finding for face perception, observers performed recognition and matching tasks where face images with reduced visibility of picture surface were shown with observers either at the centre of projection or at other viewpoints. The results show that, unlike perception of cubes, the effect of perspective transformation on face recognition is largely unaffected by the centre of projection. Furthermore, the use of perspective cues is not affected by textured surfaces. The limitation of perspective in restoring 3-D information of faces suggests a stronger role for image-based, rather than model-based, processes in recognition of unfamiliar faces.

The Language, Tone and Prosody of Emotions: Neural Substrates and Dynamics of Spoken-Word Emotion Perception

Rapid assessment of emotions is important for detecting and prioritizing salient input. Emotions are conveyed in spoken words via verbal and non-verbal channels that are mutually informative and unveil in parallel over time, but the neural dynamics and interactions of these processes are not well understood. In this paper, we review the literature on emotion perception in faces, written words, and voices, as a basis for understanding the functional organization of emotion perception in spoken words. The characteristics of visual and auditory routes to the amygdala—a subcortical center for emotion perception—are compared across these stimulus classes in terms of neural dynamics, hemispheric lateralization, and functionality. Converging results from neuroimaging, electrophysiological, and lesion studies suggest the existence of an afferent route to the amygdala and primary visual cortex for fast and subliminal processing of coarse emotional face cues. We suggest that a fast route to the amygdala may also function for brief non-verbal vocalizations (e.g., laugh, cry), in which emotional category is conveyed effectively by voice tone and intensity. However, emotional prosody which evolves on longer time scales and is conveyed by fine-grained spectral cues appears to be processed via a slower, indirect cortical route. For verbal emotional content, the bulk of current evidence, indicating predominant left lateralization of the amygdala response and timing of emotional effects attributable to speeded lexical access, is more consistent with an indirect cortical route to the amygdala. Top-down linguistic modulation may play an important role for prioritized perception of emotions in words. Understanding the neural dynamics and interactions of emotion and language perception is important for selecting potent stimuli and devising effective training and/or treatment approaches for the alleviation of emotional dysfunction across a range of neuropsychiatric states.

Learning the Moves: The Effect of Familiarity and Facial Motion on Person Recognition across Large Changes in Viewing Format

Familiarity with a face or person can support recognition in tasks that require generalization to novel viewing contexts. Using naturalistic viewing conditions requiring recognition of people from face or whole body gait stimuli, we investigated the effects of familiarity, facial motion, and direction of learning/test transfer on person recognition. Participants were familiarized with previously unknown people from gait videos and were tested on faces (experiment 1a) or were familiarized with faces and were tested with gait videos (experiment 1b). Recognition was more accurate when learning from the face and testing with the gait videos, than when learning from the gait videos and testing with the face. The repetition of a single stimulus, either the face or gait, produced strong recognition gains across transfer conditions. Also, the presentation of moving faces resulted in better performance than that of static faces. In experiment 2, we investigated the role of facial motion further by testing recognition with static profile images. Motion provided no benefit for recognition, indicating that structure-from-motion is an unlikely source of the motion advantage found in the first set of experiments.

Spatial Frequency Components of Images Modulate Neuronal Activity in Monkey Amygdala

Processing the spatial frequency components of an image is a crucial feature for visual perception, especially in recognition of faces. Here, we study the correlation between spatial frequency components of images of faces and neuronal activity in monkey amygdala while performing a visual recognition task. The frequency components of the images were analyzed using a fast Fourier transform for 40 spatial frequency ranges. We recorded 65 neurons showing statistically significant responses to at least one of the images used as a stimulus. A total of 37 of these neurons (n = 37) showed significant responses to at least three images, and in eight of them (8/37, 22%), we found a statistically significant correlation between neuron response and the modulus amplitude of at least one frequency range present in the images. Our results indicate that high spatial frequency and low spatial frequency components of images influence the activity of amygdala neurons.

The Movement Advantage in Famous and Unfamiliar Faces: A Comparison of Point-Light Displays and Shape-Normalised Avatar Stimuli

Facial movement may provide cues to identity, by supporting the extraction of face shape information via structure-from-motion, or via characteristic patterns of movement. Currently, it is unclear whether familiar and unfamiliar faces derive the same benefit from these mechanisms. This study examined the movement advantage by asking participants to match moving and static images of famous and unfamiliar faces to facial point-light displays (PLDs) or shape-normalised avatars in a same/different task (experiment 1). In experiment 2 we also used a same/different task, but participants matched from PLD to PLD or from avatar to avatar. In both experiments, unfamiliar face matching was more accurate for PLDs than for avatars, but there was no effect of stimulus type on famous faces. In experiment 1, there was no movement advantage, but in experiment 2, there was a significant movement advantage for famous and unfamiliar faces. There was no evidence that familiarity increased the movement advantage. For unfamiliar faces, results suggest that participants were relying on characteristic movement patterns to match the faces, and did not derive any extra benefit from the structure-from-motion cues in the PLDs. The results indicate that participants may use static and movement-based cues in a flexible manner when matching famous and unfamiliar faces.

Effects of band-pass spatial frequency filtering of face and object images on the amplitude of N170

Previous studies have suggested that physiological responses are greatest and face recognition performance is best when a band of middle relative spatial frequencies (SFs) is included in stimuli. Conversely, behavioural data suggest that object recognition performance shows comparatively little effect of SF variations. Here, we examine the effects of SF filtering on the amplitude of the N170 ERP component when participants are shown images of faces and objects. Our findings show that with face stimuli the amplitude of N170 exhibits a band-pass modulation function, with responses to middle SFs (around 11 cycles per face) being statistically indistinguishable from responses to full-band faces. In contrast to faces, object stimuli elicited a relatively flat function across much of the spectrum. However, for both faces and objects, middle spatial frequencies were sufficient to elicit the same N170 magnitude as full-band images. Our results with face stimuli are in accordance with previous work examining single-cell and MEG responses. Our results with objects are compatible with previous behavioural work showing a relative robustness of object recognition to SF manipulations. Our findings are novel in showing that the middle band elicits the same N170 as full-band images in both faces and objects.

From coarse to fine? Spatial and temporal dynamics of cortical face processing

Primary vision segregates information along 2 main dimensions: orientation and spatial frequency (SF). An important question is how this primary visual information is integrated to support high-level representations. It is generally assumed that the information carried by different SF is combined following a coarse-to-fine sequence. We directly addressed this assumption by investigating how the network of face-preferring cortical regions processes distinct SF over time. Face stimuli were flashed during 75, 150, or 300 ms and masked. They were filtered to preserve low SF (LSF), middle SF (MSF), or high SF (HSF). Most face-preferring regions robustly responded to coarse LSF, face information in early stages of visual processing (i.e., until 75 ms of exposure duration). LSF processing decayed as a function of exposure duration (mostly until 150 ms). In contrast, the processing of fine HSF, face information became more robust over time in the bilateral fusiform face regions and in the right occipital face area. The present evidence suggests the coarse-to-fine strategy as a plausible modus operandi in high-level visual cortex.

Effects of familiarity on spatial frequency thresholds for face matching

We examined whether familiarity with a face influences the spatial frequencies (SFs) required for face matching. Using the psychophysical method of constant stimuli and a 3AFC simultaneous matching paradigm, we obtained SF thresholds for familiar- and unfamiliar-face matching from fourteen observers, of which four were personally familiar with a subset of the faces while the remainder served as controls. SF thresholds from the lower extreme of the spectrum were approximately one octave lower for familiar than for unfamiliar faces, while SF thresholds from the upper extreme of the spectrum were approximately a third of an octave higher. These results highlight a quantitative difference between processing familiar and unfamiliar faces.

Abnormalities of visual processing and frontostriatal systems in body dysmorphic disorder

CONTEXT

Body dysmorphic disorder (BDD) is a psychiatric disorder in which individuals are preoccupied with perceived defects in their appearance, often related to their face. Little is known about its pathophysiology, although early research provides evidence of abnormal visual processing.

OBJECTIVE

To determine whether patients with BDD have abnormal patterns of brain activation when visually processing their own face with high, low, or normal spatial resolution.

DESIGN

Case-control study.

SETTING

A university hospital.

PARTICIPANTS

Seventeen right-handed medication-free subjects with BDD and 16 matched healthy control subjects. Intervention Functional magnetic resonance imaging while viewing photographs of face stimuli. Stimuli were neutral-expression photographs of the patient's own face and a familiar face (control stimuli) that were unaltered, altered to include only high spatial frequency (fine spatial resolution), or altered to include only low spatial frequency (low spatial resolution).

MAIN OUTCOME MEASURE

Blood oxygen level-dependent signal changes in the BDD and control groups during each stimulus type.

RESULTS

Subjects with BDD showed relative hyperactivity in the left orbitofrontal cortex and bilateral head of the caudate for the unaltered own-face vs familiar-face condition. They showed relative hypoactivity in the left occipital cortex for the low spatial frequency faces. Differences in activity in frontostriatal systems but not visual cortex covaried with aversiveness ratings of the faces. Severity of BDD symptoms correlated with activity in frontostriatal systems and visual cortex.

CONCLUSIONS

These results suggest abnormalities in visual processing and frontostriatal systems in BDD. Hypoactivation in the occipital cortex for low spatial frequency faces may indicate either primary visual system abnormalities for configural face elements or top-down modulation of visual processing. Frontostriatal hyperactivity may be associated both with aversion and with symptoms of obsessive thoughts and compulsive behaviors.

How spatial frequencies and visual awareness interact during face processing

In vision, high and low spatial frequencies have been dissociated at the cognitive and neural levels. Usually, high spatial frequency (HSF) is associated with slow analysis along the ventral cortical stream, and low spatial frequency (LSF) is associated with fast and automatic processing. These findings suggest a specific relation between spatial-frequency processing and visual awareness. We investigated this issue using masked-face priming with hybrid prime images of variable visibility. We found subliminal priming for both LSF and HSF information, along with a strong interaction between spatial frequency and visibility: HSF-related priming increased with stimulus visibility, whereas LSF influences remained unchanged. We argue that the results limit the validity of the coarse-to-fine model of vision and of models equating ventral-stream activity with perceptual awareness. Interpreting our results in light of the diagnostic approach suggests a close relation between awareness and diagnosticity.

Mr. Grimace or Ms. Smile: does categorization affect perceptual processing in autism?

This study compared the influence of categorization on perceptual processing in adults with autistic spectrum disorders (ASD) and normal control participants. Participants were asked to categorize hybrid faces (composed of two overlapped faces of different spatial bandwidths) by gender and emotion. Control participants exhibited a bias for low-pass information during gender categorization and a bias for high-pass information during emotion categorization. By contrast, adults with ASD showed the same low-pass bias in both tasks. This absence of a shift in processing style in the ASD group is discussed in terms of diminished top-down modulation in autism.

Newborns’ face recognition is based on spatial frequencies below 0.5 cycles per degree

A critical question in Cognitive Science concerns how knowledge of specific domains emerges during development. Here we examined how limitations of the visual system during the first days of life may shape subsequent development of face processing abilities. By manipulating the bands of spatial frequencies of face images, we investigated what is the nature of the visual information that newborn infants rely on to perform face recognition. Newborns were able to extract from a face the visual information lying from 0 to 1 cpd (Experiment 1), but only a narrower 0-0.5 cpd spatial frequency range was successful to accomplish face recognition (Experiment 2). These results provide the first empirical support of a low spatial frequency advantage in individual face recognition at birth and suggest that early in life low-level, non-specific perceptual constraints affect the development of the face processing system.

Effects of image background on spatial-frequency thresholds for face recognition

A great deal of work has been devoted to the question of which spatial frequencies, if any, are optimal for various visual tasks, such as face and object recognition. However, to date these studies have all been carried out with stimuli set against a uniform background. It is possible that this type of stimulus does not produce ecologically valid results. The natural world in which visual tasks normally take place involves a great deal of luminance variation and distracting visual structure, which may alter the spatial frequencies necessary for a task. We conducted two experiments that examined the effects of image background on the spatial-frequency thresholds (50% maximum of a low-pass or high-pass Butterworth filter) for face recognition by the psychophysical methods of adjustment and constant stimuli. In both experiments we found no significant difference in spatial-frequency thresholds between uniform-grey backgrounds and natural-scene backgrounds, and only minor differences between uniform-grey backgrounds and fractal noise backgrounds. This suggests that results obtained with uniform backgrounds are ecologically valid and that background effects, if they exist, are small.

Spatial frequency thresholds for face recognition when comparison faces are filtered and unfiltered

Previous work has shown an advantage of middle spatial frequencies (SFs) in face recognition. However, a few recent studies have suggested that this advantage is reduced when comparison and test stimuli are spatially filtered in a similar way. In the present study, we used standard psychophysical methods, in combination with a match-to-sample task, to determine the SF thresholds for face matching under conditions in which: (1) comparison stimuli were unfiltered and (2) comparison stimuli were spatially filtered in the same way as test stimuli. In two experiments, we show that SFs closer to the middle band are sought out more in the former case than in the latter. These results are compatible with the idea that a middle band of SFs will be most useful for any visual task and that the breadth of this optimal middle band will vary depending on task characteristics.

Is high-spatial frequency information used in the early stages of face detection?

The present study examined the role of high-spatial frequency information in early face processing, as indexed by the N170 face-sensitive ERP component. Participants detected 4 versions of famous faces, including full spectrum faces, and bandpass filtered faces containing predominantly high-spatial frequencies, low-spatial frequencies or both. The power spectra of all stimuli were balanced by superimposing the faces onto a visual noise background that included the spatial frequency information that was missing in filtered faces, e.g., high-spatial frequency faces were presented on a high- and low-spatial frequency background. An additional condition comprising of filtered visual noise only was also created to ensure that any observed effects were related to the processing of faces and not simply due to variations between spatial frequency information. Both behavioral and electrophysiological results replicated previous findings of a low-spatial frequency advantage for face processing. However, our results also show that faces containing both high and low-spatial frequency information are detected faster and more accurately than faces containing predominantly low-spatial frequencies. Furthermore, this advantage occurred with an enhanced amplitude of the N170. Together, these findings refute the suggestion that high-spatial frequencies are redundant in face perception.

Spatial-frequency thresholds for object categorisation at basic and subordinate levels

In an attempt to understand how low-level visual information contributes to object categorisation, previous studies have examined the effects of spatially filtering images on object recognition at different levels of abstraction. Here, the quantitative thresholds for object categorisation at the basic and subordinate levels are determined by using a combination of the method of adjustment and a match-to-sample method. Participants were asked to adjust the cut-off of either a low-pass or high-pass filter applied to a target image until they reached the threshold at which they could match the target image to one of six simultaneously presented category names. This allowed more quantitative analysis of the spatial frequencies necessary for recognition than previous studies. Results indicate that a more central range of low spatial frequencies is necessary for subordinate categorisation than basic, though the difference is small, at about 0.25 octaves. Conversely, there was no effect of categorisation level on high-pass thresholds.

Faces are "spatial"--holistic face perception is supported by low spatial frequencies

Faces are perceived holistically, a phenomenon best illustrated when the processing of a face feature is affected by the other features. Here, the authors tested the hypothesis that the holistic perception of a face mainly relies on its low spatial frequencies. Holistic face perception was tested in two classical paradigms: the whole-part advantage (Experiment 1) and the composite face effect (Experiments 2-4). Holistic effects were equally large or larger for low-pass filtered faces as compared to full-spectrum faces and significantly larger than for high-pass filtered faces. The disproportionate composite effect found for low-pass filtered faces was not observed when holistic perception was disrupted by inversion (Experiment 3). Experiment 4 showed that the composite face effect was enhanced only for low spatial frequencies, but not for intermediate spatial frequencies known be critical for face recognition. These findings indicate that holistic face perception is largely supported by low spatial frequencies. They also suggest that holistic processing precedes the analysis of local features during face perception.

Recognizing faces defined by texture gradients

Texture gradients can reveal surface orientation in a manner similar to shape from shading, and therefore provide an important cue for object recognition. In this study, we tested whether a complex 3-D object, such as a face, can be identified from texture gradients alone. The stimuli were laser-scanned faces for which the texture element was a fractal-noise pattern mapped onto the 3-D surface. An eight-alternative forced choice task was used in which participants matched a face defined by texture gradients to one of eight faces defined by shape from shading (Experiment 1) or by texture gradients (Experiment 2). On average, participants scored 24% and 18%, respectively, above chance in these experiments. Although this performance was much poorer than the performance based entirely on shape-from-shading stimuli (Experiment 3), the results suggest that texture gradient information may be used to recover surface geometry of complex objects.

The effects of face spatial frequencies on cortical processing revealed by magnetoencephalography

To study the spatial frequency (SF) effects on cortical face processing, we recorded magnetoencephalographic responses in seven healthy subjects to upright and inverted human faces. Four face types were used, including original (broad-band SF, BSF), low SF (LSF, <5 cycles/face), middle SF (MSF, 5-15 cycles/face), and high SF (HSF, >15 cycles/face) face images. Using equivalent current dipole (ECD) modeling, neuromagnetic M170 responses peaking around 160-185 ms were localized in right occipitotemporal region across subjects to BSF faces. M170 responses to LSF faces showed longer latency and smaller amplitude compared with those to BSF faces. We found no significant difference between BSF, MSF, and HSF conditions in M170 amplitude or latency. ECD locations for the four upright face conditions were close to one another, although the mean locations for MSF or HSF seemed more medial than those for BSF or LSF. Longer latencies for inverted than upright faces were observed in BSF (183.4+/-8.5 ms versus 168+/-6.9 ms, P<0.001) and LSF face conditions (223.6+/-13.1 ms versus 207.3+/-16.3 ms, P<0.01). M170 ECDs were located more medial for inverted than upright images in either BSF or LSF condition. In conclusion, the less M170 activation to LSF faces suggests that face parts information is important for early face processing. The cortical representations in right occipitotemporal region for configural and face feature processing are overlapping. Our findings on the face inversion effect suggest that inverted BSF and LSF faces may be processed as objects.

Scale invariant adaptation in fusiform face-responsive regions

Several functional neuroimaging studies have observed response adaptation in face-sensitive regions when repeating identical face stimuli. To address whether this was due to low-level stimulus properties or facial identity, we decomposed pictures of faces into pictures preserving only the lower or higher parts of the normal frequency spectrum. In an event-related functional neuroimaging study, pairs of such pictures were sequentially presented that showed the same or different persons in the same or different frequency bands. This factorial design allowed to separate effects related to repetition of personal identity from those related to identical stimulus properties. In a random effects group analysis, activation in the right fusiform region was affected by repetition of personal identity regardless of changing or constant spatial scale. Responses in the more medial and posterior fusiform and lingual regions adapted with repetition of the same frequency band. An analysis in regions of interest determined individually as face responsive showed that repetition decreases for the same faces in fusiform face-responsive regions generalized across spatial frequency bands. Our results therefore point to a role of this area in discriminating individual faces at a level of representation that is invariant to changes in low-level stimulus properties, as spatial scale. The same invariance could not be detected in more posterior occipital face-responsive regions.

Configural face encoding and spatial frequency information

Configural relations and a critical band of spatial frequencies (SFs) in the middle range are particularly important for face recognition. We report the results of four experiments in which the relationship between these two types of information was examined. In Experiments 1, 2A, and 2B, the face inversion effect (FIE) was used to probe configural face encoding. Recognition of upright and inverted faces and nonface objects was measured in four conditions: a no-filter condition and three SF conditions (low, medium, and high frequency). We found significant FIEs of comparable magnitudes for all frequency conditions. In Experiment 3, discrimination of faces on the basis of either configural or featural modifications was measured under the same four conditions. Although the ability to discriminate configural modifications was superior in the medium-frequency condition, so was the ability to discriminate featural modifications. We conclude that the band of SF that is critical for face recognition does not contribute preferentially to configural encoding.

Effects on face recognition of spatial-frequency information contained in inspection and test stimuli

Researchers often assume a critical band of spatial frequencies is required for face recognition. Also, many studies have not measured the contrast required for recognition. On Day 1, observers viewed high-pass-filtered (HP), low-pass-filtered (LP), or unfiltered (UF) faces. On Day 2, they viewed a variety of faces, some of which were LP filtered, HP filtered, and UF. Observers adjusted contrast until they achieved both detection and recognition. Observers were most accurate and sensitive when filtered faces agreed in spatial-frequency content across days. Faces differing in spatial-frequency content were least well recognized. Unfiltered faces always fell between the 2 extremes. Observers generally used less contrast to recognize unfiltered than filtered faces. Correspondence of information between inspection and testing seemed more important than any particular range of frequencies.

Distinct spatial frequency sensitivities for processing faces and emotional expressions

High and low spatial frequency information in visual images is processed by distinct neural channels. Using event-related functional magnetic resonance imaging (fMRI) in humans, we show dissociable roles of such visual channels for processing faces and emotional fearful expressions. Neural responses in fusiform cortex, and effects of repeating the same face identity upon fusiform activity, were greater with intact or high-spatial-frequency face stimuli than with low-frequency faces, regardless of emotional expression. In contrast, amygdala responses to fearful expressions were greater for intact or low-frequency faces than for high-frequency faces. An activation of pulvinar and superior colliculus by fearful expressions occurred specifically with low-frequency faces, suggesting that these subcortical pathways may provide coarse fear-related inputs to the amygdala.

Face recognition is robust with incongruent image resolution: relationship to security video images

Identifying a criminal captured on conventional security video typically requires matching poor-quality video footage against a high-quality photograph. The authors examined the consequence of such a large discrepancy in image quality. Recognition and matching performance of this incongruent-quality condition was compared with that of a congruent one, in which a high-quality photograph was reduced to a low-quality video. Recognition memory was little affected by this manipulation, whereas matching performance of the incongruent condition enjoyed occasional advantage. The results show that person identification can tolerate a large discrepancy between image qualities of matching stimuli when one of the images is of poor quality.