Factors influencing spatial frequency extraction in faces: A review

Discrepancies in perceived humanness between spatially filtered and unfiltered faces and their associations with uncanny feelings

Human and artificial features that coexist in certain types of human-like robots create a discrepancy in perceived humanness and evoke uncanny feelings in human observers. However, whether this perceptual mismatch in humanness occurs for all faces, and whether it is related to the uncanny feelings toward them, is unknown. We investigated this by examining perceived humanness for a variety of natural images of robot and human faces with different spatial frequency (SF) information: that is, faces with only low SF, middle SF, and high SF information, and intact (spatially unfiltered) faces. Uncanny feelings elicited by these faces were also measured. The results showed perceptual mismatches that LSF, MSF, and HSF faces were perceived as more human than intact faces. This was particularly true for intact robot faces that looked slightly human, which tended to evoke strong uncanny feelings. Importantly, the mismatch in perceived humanness between the intact and spatially filtered faces was positively correlated with uncanny feelings toward intact faces. Given that the human visual system performs SF analysis when processing faces, the perceptual mismatches observed in this study likely occur in real life for all faces, and as such might be a ubiquitous source of uncanny feelings in real-life situations.

Distinct patterns of monocular advantage for facial emotions in social anxiety

Individuals with social anxiety often exhibit atypical processing of facial expressions. Previous research in social anxiety has primarily emphasized cognitive bias associated with face processing and the corresponding abnormalities in cortico-limbic circuitry, yet whether social anxiety influences early perceptual processing of emotional faces remains largely unknown. We used a psychophysical method to investigate the monocular advantage for face perception (i.e., face stimuli are better recognized when presented to the same eye compared to different eyes), an effect that is indicative of early, subcortical processing of face stimuli. We compared the monocular advantage for different emotional expressions (neutral, angry and sad) in three groups (N = 24 per group): individuals clinically diagnosed with social anxiety disorder (SAD), individuals with high social anxiety in subclinical populations (SSA), and a healthy control (HC) group of individuals matched for age and gender. Compared to SSA and HC groups, we found that individuals with SAD exhibited a greater monocular advantage when processing neutral and sad faces. While the magnitudes of monocular advantages were similar across three groups when processing angry faces, individuals with SAD performed better in this condition when the faces were presented to different eye. The former findings suggest that social anxiety leads to an enhanced role of subcortical structures in processing nonthreatening expressions. The latter findings, on the other hand, likely reflect an enhanced cortical processing of threatening expressions in SAD group. These distinct patterns of monocular advantage indicate that social anxiety altered representation of emotional faces at various stages of information processing, starting at an early stage of the visual system.

Face motion form at learning influences the time course of face spatial frequency processing during test

Studies that use static faces suggest that facial processing follows a coarse-to-fine sequence; i.e., holistic precedes featural processing, due to low and high spatial frequencies (LSF, HSF) transmitting holistic/global and featural/local information respectively. Although recent studies have focused on the role of facial movement in holistic facial processing, it is unclear whether moving faces have the same processing mechanism as static ones, especially in the time course of processing. The current study uses the event-related potential technique to investigate this issue by manipulating the facial format at study and face spatial frequency during the test. ERP results showed that the P1 amplitude was increased by LSF faces relative to HSF ones, using both moving and static study faces, with the former larger than the latter. The N170 amplitude was more sensitive to HSF than LSF faces when only static study faces were used, while the P2 amplitude was more sensitive to LSF faces regardless of the facial study format. The above results were not modulated by the race of the faces. These results favor the view that regardless of face race, moving study faces promote holistic processing during the earliest stage of face recognition. Furthermore, holistic processing is observed to be the same for both static and moving study faces at a later stage associated with more in-depth processing. It is evident that facial motion should be factored into further studies of face recognition, given the distinctions between holistic and featural processing for moving and static study faces.

Effects of spatially filtered fearful faces and awareness on amygdala activity in adults with autism spectrum disorder: A magnetoencephalography study

BACKGROUND

The amygdala is pivotal in emotional face processing. Spatial frequencies (SFs) of visual images are divided and processed via two visual pathways: low spatial frequency (LSF) information is conveyed by the magnocellular pathway, while the parvocellular pathway carries high spatial frequency information. We hypothesized that altered amygdala activity might underlie atypical social communication caused by changes in both conscious and non-conscious emotional face processing in the brain in individuals with autism spectrum disorder (ASD).

METHOD

Eighteen adults with ASD and 18 typically developing (TD) peers participated in this study. Spatially filtered fearful- and neutral-expression faces and object stimuli were presented under supraliminal or subliminal conditions, and neuromagnetic responses in the amygdala were measured using 306-channel whole-head magnetoencephalography.

RESULTS

The latency of the evoked responses at approximately 200 ms to unfiltered neutral face stimuli and object stimuli in the ASD group was shorter than that in the TD group in the unaware condition. Regarding emotional face processing, the evoked responses in the ASD group were larger than those in the TD group under the aware condition. The later positive shift during 200-500 ms (ARV) was larger than that in the TD group, regardless of awareness. Moreover, ARV to HSF face stimuli was larger than that to the other spatial filtered face stimuli in the aware condition.

CONCLUSION

Regardless of awareness, ARV might reflect atypical face information processing in the ASD brain.

The Time Sequence of Face Spatial Frequency Differs During Working Memory Encoding and Retrieval Stages

Previous studies have found that P1 and P2 components were more sensitive to configural and featural face processing, respectively, when attentional resources were sufficient, suggesting that face processing follows a coarse-to-fine sequence. However, the role of working memory (WM) load in the time course of configural and featural face processing is poorly understood, especially whether it differs during encoding and retrieval stages. This study employed a delayed recognition task with varying WM load and face spatial frequency (SF). Our behavioral and ERP results showed that WM load modulated face SF processing. Specifically, for the encoding stage, P1 and P2 were more sensitive to broadband SF (BSF) faces, while N170 was more sensitive to low SF (LSF) and BSF faces. For the retrieval stage, P1 on the right hemisphere was more sensitive to BSF faces relative to HSF faces, N170 was more sensitive to LSF faces than HSF faces, especially under the load 1 condition, while P2 was more sensitive to high SF (HSF) faces than HSF faces, especially under load 3 condition. These results indicate that faces are perceived less finely during the encoding stage, whereas face perception follows a coarse-to-fine sequence during the retrieval stage, which is influenced by WM load. The coarse and fine information were processed especially under the low and high load conditions, respectively.

The Predictive Role of Low Spatial Frequencies in Automatic Face Processing: A Visual Mismatch Negativity Investigation

Visual processing is thought to function in a coarse-to-fine manner. Low spatial frequencies (LSF), conveying coarse information, would be processed early to generate predictions. These LSF-based predictions would facilitate the further integration of high spatial frequencies (HSF), conveying fine details. The predictive role of LSF might be crucial in automatic face processing, where high performance could be explained by an accurate selection of clues in early processing. In the present study, we used a visual Mismatch Negativity (vMMN) paradigm by presenting an unfiltered face as standard stimulus, and the same face filtered in LSF or HSF as deviant, to investigate the predictive role of LSF vs. HSF during automatic face processing. If LSF are critical for predictions, we hypothesize that LSF deviants would elicit less prediction error (i.e., reduced mismatch responses) than HSF deviants. Results show that both LSF and HSF deviants elicited a mismatch response compared with their equivalent in an equiprobable sequence. However, in line with our hypothesis, LSF deviants evoke significantly reduced mismatch responses compared to HSF deviants, particularly at later stages. The difference in mismatch between HSF and LSF conditions involves posterior areas and right fusiform gyrus. Overall, our findings suggest a predictive role of LSF during automatic face processing and a critical involvement of HSF in the fusiform during the conscious detection of changes in faces.

High spatial frequency filtered primes hastens happy faces categorization in autistic adults

Coarse information of a visual stimulus is conveyed by Low Spatial Frequencies (LSF) and is thought to be rapidly extracted to generate predictions. This may guide fast recognition with the subsequent integration of fine information, conveyed by High Spatial Frequencies (HSF). In autism, emotional face recognition is challenging, and might be related to alterations in LSF predictive processes. We analyzed the data of 27 autistic and 34 non autistic (NA) adults on an emotional Stroop task (i.e., emotional face with congruent or incongruent emotional word) with spatially filtered primes (HSF vs.LSF). We hypothesized that LSF primes would generate predictions leading to faster categorization of the target face compared to HSF primes, in the NA group but not in autism. Surprisingly, HSF primes led to faster categorization than LSF primes in both groups. Moreover, the advantage of HSF vs.LSF primes was stronger for angry than happy faces in NA, but was stronger for happy than angry faces in autistic participants. Drift diffusion modelling confirmed HSF advantage and showed a longer non-decision time (e.g., encoding) in autism. Despite LSF predictive impairments in autism was not corroborated, our analyses suggest low level processing specificities in autism.

Exploring emotional face processing in 5-month-olds: The relation with quality of parent-child interaction and spatial frequencies

It is unclear whether infants differentially process emotional faces in the brain at 5 months of age. Contradictory findings of previous research indicate that additional factors play a role in this process. The current study investigated whether five-month-old infants show differential brain activity between emotional faces. Furthermore, we explored the relation between emotional face processing and (I) stimulus characteristics, specifically the spatial frequency content, and (II) parent, child, and dyadic qualities of interaction characteristics. Face-sensitive components (i.e., N290, P400, Nc) in response to neutral and fearful faces that contained only lower or higher spatial frequencies were assessed. Quality of parent-child interaction was assessed with the Manchester Assessment of Caregiver Infant Interaction (MACI). The results show that, as a full group, none of the components differed between emotional expressions. However, when splitting the group based on median MACI scores, infants who showed high quality of interaction (i.e., more attentiveness to caregiver, positive and negative affect, and liveliness) processed emotions differently, whereas infants who showed low quality did not. These results indicate that a sub-group of infants show differential emotional face processing at 5 months of age, which seem to relate to quality of their behavior during the parent-child interaction.

Data-point-wise spatiotemporal mapping of human ventral visual areas: Use of spatial frequency/luminance-modulated chromatic faces

Visual information involving facial identity and expression is crucial for social communication. Although the influence of facial features such as spatial frequency (SF) and luminance on face processing in visual areas has been studied extensively using grayscale stimuli, the combined effects of other features in this process have not been characterized. To determine the combined effects of different SFs and color, we created chromatic stimuli with low, high or no SF components, which bring distinct SF and color information into the ventral stream simultaneously. To obtain neural activity data with high spatiotemporal resolution we recorded face-selective responses (M170) using magnetoencephalography. We used a permutation test procedure with threshold-free cluster enhancement to assess statistical significance while resolving problems related to multiple comparisons and arbitrariness found in traditional statistical methods. We found that time windows with statistically significant threshold levels were distributed differently among the stimulus conditions. Face stimuli containing any SF components evoked M170 in the fusiform gyrus (FG), whereas a significant emotional effect on M170 was only observed with the original images. Low SF faces elicited larger activation of the FG and the inferior occipital gyrus than the original images, suggesting an interaction between low and high SF information processing. Interestingly, chromatic face stimuli without SF first activated color-selective regions and then the FG, indicating that facial color was processed according to a hierarchy in the ventral stream. These findings suggest complex effects of SFs in the presence of color information, reflected in M170, and unveil the detailed spatiotemporal dynamics of face processing in the human brain.

The image features of emotional faces that predict the initial eye movement to a face

Emotional facial expressions are important visual communication signals that indicate a sender's intent and emotional state to an observer. As such, it is not surprising that reactions to different expressions are thought to be automatic and independent of awareness. What is surprising, is that studies show inconsistent results concerning such automatic reactions, particularly when using different face stimuli. We argue that automatic reactions to facial expressions can be better explained, and better understood, in terms of quantitative descriptions of their low-level image features rather than in terms of the emotional content (e.g. angry) of the expressions. Here, we focused on overall spatial frequency (SF) and localized Histograms of Oriented Gradients (HOG) features. We used machine learning classification to reveal the SF and HOG features that are sufficient for classification of the initial eye movement towards one out of two simultaneously presented faces. Interestingly, the identified features serve as better predictors than the emotional content of the expressions. We therefore propose that our modelling approach can further specify which visual features drive these and other behavioural effects related to emotional expressions, which can help solve the inconsistencies found in this line of research.

Pinpointing the optimal spatial frequency range for automatic neural facial fear processing

Faces convey an assortment of emotional information via low and high spatial frequencies (LSFs and HSFs). However, there is no consensus on the role of particular spatial frequency (SF) information during facial fear processing. Comparison across studies is hampered by the high variability in cut-off values for demarcating the SF spectrum and by differences in task demands. We investigated which SF information is minimally required to rapidly detect briefly presented fearful faces in an implicit and automatic manner, by sweeping through an entire SF range without constraints of predefined cut-offs for LSFs and HSFs. We combined fast periodic visual stimulation with electroencephalography. We presented neutral faces at 6 ​Hz, periodically interleaved every 5th image with a fearful face, allowing us to quantify an objective neural index of fear discrimination at exactly 1.2 ​Hz. We started from a stimulus containing either only very low or very high SFs and gradually increased the SF content by adding higher or lower SF information, respectively, to reach the full SF spectrum over the course of 70 ​s. We found that faces require at least SF information higher than 5.93 cycles per image (cpi) to implicitly differentiate fearful from neutral faces. However, exclusive HSF faces, even in a restricted SF range between 94.82 and 189.63 cpi already carry the critical information to extract the emotional expression of the faces.

Charting development of ERP components on face-categorization: Results from a large longitudinal sample of infants

•

We report longitudinal ERP data of 80 infants in a face-discrimination task.

•

P1, N290, Nc are all sensitive to faces in five-month-olds.

•

P1, N290, Nc show equal face-categorization in infants tested longitudinally.

•

N290 shows less variation in face-categorization trajectories than P1 or Nc.

•

Visual ERPs increase in amplitude over infancy, but this is not face-specific.

From infancy onwards, EEG is widely used to measure face-categorization, i.e. differential brain activity to faces versus non-face stimuli. Four ERP components likely signal infants’ face-sensitivity but reflect different underlying mechanisms: the P1, N290, P400, Nc. We test whether these components reveal similar developmental patterns from early to late infancy, using a longitudinal dataset of 80 infants tested at 5 and 10 months. The P1, N290, and the Nc show face-categorization already in 5-months-olds, a pattern which did not change over time. Development is visible as increased amplitudes in all components, but similar for face and non-face stimuli. By using Markov models, we illustrate that there are differences in the distribution of individual trajectories of face-categorization components from 5 to 10 months. Whereas individual trajectories appear more varied for the Nc and the P1, the N290 reveals a more consistent pattern: a larger proportion of 5-month-olds shows the dominant group response; a larger proportion of 10-month-olds remains in this group, and larger proportions of the alternative trajectories from 5- to 10-month-olds move towards the dominant group. This is vital information when one wants to examine individual differences in infant ERPs related to face-categorization.

Size Effects in the Recognition of Blurry Faces

Spatial frequencies critical for recognition of faces are scale-dependent. Progressively coarser features of the face are utilized at smaller sizes, despite the availability of finer features. Blur removes fine details in an image, disrupting the finer features utilized for recognition at large sizes. At smaller sizes, observers utilize coarser features, and thus, recognition may be less impacted by blur. This coupling between size and critical spatial frequencies allows us to predict a regime in which observers tolerate blur better with decreasing image sizes within a range of moderate face sizes. We tested recognition of famous faces in four conditions: large-intact, small-intact, large-blurry, and small-blurry. Observers showed high recognition performance in both intact conditions. Blur significantly disrupted recognition, yet accuracy was significantly and consistently higher in the small-blurry compared to the large-blurry condition. These results are suggestive of an inherent scale-dependent mechanism that, at certain sizes, negatively impacts recognition of blurry images.

Time course of spatial frequency integration in face perception: An ERP study

Face perception is based on the processing and integration of multiple spatial frequency (SF) ranges. However, the temporal dynamics of SF integration to form an early face representation in the human brain is still a matter of debate. To address this issue, we recorded event-related potentials (ERPs) during the presentation of spatial frequency-manipulated facial images. Twenty-six participants performed a gender discrimination task on non-filtered, low-, high-, and band-pass filtered face images, corresponding, respectively, to the full range, spatial frequencies up to 8 cycles/image, above 32 cycles/image, and from 8 to 16 cycles/image. Behaviorally, the task related-performance was more accurate and faster for non-filtered (NF) and mid-range SF (MSF) than for low SF (LSF) and high SF (HSF) stimuli. At both behavioral and electrophysiological levels, response to MSF contained in faces did not differ from the responses to full spectrum non-filtered (NF) facial images. In ERPs, LSF facial images evoked the largest P1 amplitude while HSF facial images evoked the largest N170 amplitude compared with the other three conditions. Since LSFs and HSFs would transmit global and local information respectively, our observations lend further support to the "coarse-to-fine" processing theory of faces. Furthermore, they offer original evidence of the effectiveness and adequacy of the mid-range spatial frequency in face perception. Possible theoretical interpretations of our findings are discussed.