Explaining the face-inversion effect: the face-scheme incompatibility (FSI) model

Recognition of facial expressions of emotions in tactile drawings by blind children, children with low vision and sighted children

In the context of blindness, studies on the recognition of facial expressions of emotions by touch are essential to define the compensatory touch abilities and to create adapted tools on emotions. This study is the first to examine the effect of visual experience in the recognition of tactile drawings of facial expressions of emotions by children with different visual experiences. To this end, we compared the recognition rates of tactile drawings of emotions between blind children, children with low vision and sighted children aged 6-12 years. Results revealed no effect of visual experience on recognition rates. However, an effect of emotions and an interaction effect between emotions and visual experience were found. Indeed, while all children had a low average recognition rate, the drawings of fear, anger and disgust were particularly poorly recognized. Moreover, sighted children were significantly better at recognizing the drawings of surprise and sadness than the blind children who only showed high recognition rates for joy. The results of this study support the importance of developing emotion tools that can be understood by children with different visual experiences.

The effect of processing partial information in dynamic face perception

Face perception is a major topic in vision research. Most previous research has concentrated on (holistic) spatial representations of faces, often with static faces as stimuli. However, faces are highly dynamic stimuli containing important temporal information. How sensitive humans are regarding temporal information in dynamic faces is not well understood. Studies investigating temporal information in dynamic faces usually focus on the processing of emotional expressions. However, faces also contain relevant temporal information without any strong emotional expression. To investigate cues that modulate human sensitivity to temporal order, we utilized muted dynamic neutral face videos in two experiments. We varied the orientation of the faces (upright and inverted) and the presence/absence of eye blinks as partial dynamic cues. Participants viewed short, muted, monochromic videos of models vocalizing a widely known text (National Anthem). Videos were played either forward (in the correct temporal order) or backward. Participants were asked to determine the direction of the temporal order for each video, and (at the end of the experiment) whether they had understood the speech. We found that face orientation, and the presence/absence of an eye blink affected sensitivity, criterion (bias) and reaction time: Overall, sensitivity was higher for upright compared to inverted faces, and in the condition where an eye blink was present compared to the condition without an eye blink. Reaction times were mostly faster in the conditions with higher sensitivity. A bias to report inverted faces as 'backward' observed in Experiment I, where upright and inverted faces were presented randomly interleaved within each block, was absent when presenting upright and inverted faces in different blocks in Experiment II. Language comprehension results revealed that there was higher sensitivity when understanding the speech compared to not understanding the speech in both experiments. Taken together, our results showed higher sensitivity with upright compared to inverted faces, suggesting that the perception of dynamic, task-relevant information was superior with the canonical orientation of the faces. Furthermore, partial information coming from eye blinks, in addition to mouth movements, seemed to play a significant role in dynamic face perception, both when faces were presented upright and inverted. We suggest that studying the perception of facial dynamics beyond emotional expressions will help us to better understand the mechanisms underlying the temporal integration of facial information from different -partial and holistic- sources, and that our results show how different strategies, depending on the available information, are employed by human observers when judging the temporal order of faces.

Looming Angry Faces: Preliminary Evidence of Differential Electrophysiological Dynamics for Filtered Stimuli via Low and High Spatial Frequencies

Looming motion interacts with threatening emotional cues in the initial stages of visual processing. However, the underlying neural networks are unclear. The current study investigated if the interactive effect of threat elicited by angry and looming faces is favoured by rapid, magnocellular neural pathways and if exogenous or endogenous attention influences such processing. Here, EEG/ERP techniques were used to explore the early ERP responses to moving emotional faces filtered for high spatial frequencies (HSF) and low spatial frequencies (LSF). Experiment 1 applied a passive-viewing paradigm, presenting filtered angry and neutral faces in static, approaching, or receding motions on a depth-cued background. In the second experiment, broadband faces (BSF) were included, and endogenous attention was directed to the expression of faces. Our main results showed that regardless of attentional control, P1 was enhanced by BSF angry faces, but neither HSF nor LSF faces drove the effect of facial expressions. Such findings indicate that looming motion and threatening expressions are integrated rapidly at the P1 level but that this processing relies neither on LSF nor on HSF information in isolation. The N170 was enhanced for BSF angry faces regardless of attention but was enhanced for LSF angry faces during passive viewing. These results suggest the involvement of a neural pathway reliant on LSF information at the N170 level. Taken together with previous reports from the literature, this may indicate the involvement of multiple parallel neural pathways during early visual processing of approaching emotional faces.

The role of inversion and face masks on simultaneous and delayed face matching tasks

Although it is generally accepted that face recognition relies on holistic processing, it has been suggested that the simultaneous face matching task may depend on a more analytical or featural processing approach. However, empirical evidence supporting this claim is limited. In two experiments, we further explored the role of holistic and featural processing on simultaneous face matching by manipulating holistic processing through inversion and presenting faces with or without face masks. The results from Experiment 1 revealed that both inversion and face masks impaired matching performance. However, while the inversion effect was evident in both full-view and masked faces, the mask effect was only found in upright, but not inverted, faces. These results were replicated in Experiment 2 but, the inversion and mask effects were stronger in delayed face matching than in simultaneous face matching. Our findings suggest that simultaneous face matching relies on holistic processing, but to a smaller extent compared to higher memory-demanding identification tasks.

Change detection versus change localization for faces, houses, and words

Holistic processing aids in the discrimination of visually similar objects, but it may also come with a cost. Indeed holistic processing may improve the ability to detect changes to a face while impairing the ability to locate where the changes occur. We investigated the capacity to detect the occurrence of a change versus the capacity to detect the localization of a change for faces, houses, and words. Change detection was better than change localization for faces. Change localization outperformed change detection for houses. For words, there was no difference between detection and localization. We know from previous studies that words are processed holistically. However, being an object of visual expertise processed holistically, visual words are also a linguistic entity. Previously, the word composite effect was found for phonologically consistent words but not for phonologically inconsistent words. Being an object of visual expertise for which linguistic information is important, letter position information, is also crucial. Thus, the importance of localization of letters and features may augment the capacity to localize a change in words making the detection of a change and the detection of localization of a change equivalent.

Face pareidolia in male schizophrenia

Faces are valuable signals for efficient social interaction. Yet, social cognition including the sensitivity to a coarse face scheme may be deviant in schizophrenia (SZ). Tuning to faces in non-face images such as shadows, grilled toasts, or ink blots is termed face pareidolia. This phenomenon is poorly investigated in SZ. Here face tuning was assessed in 44 male participants with SZ and person-by-person matched controls by using recently created Face-n-Thing images (photographs of non-face objects to a varying degree resembling a face). The advantage of these images is that single components do not automatically trigger face processing. Participants were administered a set of images with upright and inverted (180° in the image plane) orientation. In a two-alternative forced-choice paradigm, they had to indicate whether an image resembled a face. The findings showed that: (i) With upright orientation, SZ patients exhibited deficits in face tuning: they provided much fewer face responses than controls. (ii) Inversion generally hindered face pareidolia. However, while in neurotypical males, inversion led to a drastic drop in face impression, in SZ, the impact of orientation was reduced. (iii) Finally, in accord with the signal detection theory analysis, the sensitivity index (d-prime) was lower in SZ, whereas no difference occurred in decision criterion. The outcome suggests altered face pareidolia in SZ is caused by lower face sensitivity rather than by alterations in cognitive bias. Comparison of these findings with earlier evidence confirms that tuning to social signals is lower in SZ, and warrants tailored brain imaging research.

The relationships between reading fluency and different measures of holistic word processing

Recently, paradigms in the face recognition literature have been adopted to reveal holistic processing in word recognition. It is unknown, however, whether different measures of holistic word processing share similar underlying mechanisms, and whether fluent word reading relies on holistic word processing. We measured holistic processing effects in three paradigms (composite, configural sensitivity, part-whole) as well as in reading fluency (3DM task: reading aloud high- and low-frequency words and pseudowords). Bin scores were used to combine accuracy and response time variables in the quest for a more comprehensive, reliable, and valid measure of holistic processing. Weak correlations were found between the different holistic processing measures, with only a significant correlation between the configural sensitivity effect and part-whole effect (r = .32) and a trend of a positive correlation between the word composite effect and configural sensitivity effect (r = .21). Of the three holistic processing measures, only one (part-whole effect) correlated with a lexical access measure of 3DM (r = .23). We also performed a principal component analysis (PCA) of performance in the three lists of 3DM, with the second most probably reflecting lexical access processes. There was a tendency for a positive correlation between part-whole bin measure and Component 2 of PCA. We also found a positive correlation between composite aligned in accuracy and Component 2 of PCA.Our results show that different measures of holistic word processing reflect predominantly different mechanisms, and that differences among normal readers in word reading do not seem to depend highly on holistic processing.

Comparing Inverted Faces to Upright Faces Using Similarity or Mental Rotation

A major interest of research in face recognition lies in explaining the face inversion effect, in which the recognition of an inverted face is less successful than that of an upright face. However, prior research has devoted little effort to examining how the cognitive system handles comparison between inverted and upright faces. The results of a preparatory experiment and two following experiments support the conclusion that the comparison is based more on visual similarity than on a mental rotation of the inverted face to an upright face. Visual similarity is based on certain elements mutual to the two faces, which resist the transformation of inversion. These elements are symmetric or salient components of the face, such as round eyes or thick lips.

The inversion effect in word recognition: the effect of language familiarity and handwriting

Humans have expertise with visual words and faces. One marker of this expertise is the inversion effect. This is attributed to experience with those objects being biased towards a canonical orientation, rather than some inherent property of object structure or perceptual anisotropy. To confirm the role of experience, we measured inversion effects in word matching for familiar and unfamiliar languages. Second, we examined whether there may be more demands on reading expertise with handwritten stimuli rather than computer font, given the greater variability and irregularities in the former, with the prediction of larger inversion effects for handwriting. We recruited two cohorts of subjects, one fluent in Farsi and the other in Punjabi, neither of whom were able to read the other's language. Subjects performed a match-to-sample task with words in either computer fonts or handwritings. Subjects were more accurate and faster with their familiar language, even when it was inverted. Inversion effects were present for the familiar but not the unfamiliar language. The inversion effect in accuracy for handwriting was larger than that for computer fonts in the familiar language. We conclude that the word inversion effect is generated solely by orientation-biased experience, and that demands on this expertise are greater with handwriting than computer font.

Robustness of the aging effect of smiling against vertical facial orientation

Background: Previous studies have shown that the association between smiling and youth is a misconception; smiling faces have been estimated to be older than neutral faces. Previous studies have indicated that this aging effect of smiling (AES) is due to eye wrinkles caused by the facial action of smiling. However, whether holistic processing for facial expressions is involved in AES has not been investigated. The present study aimed to clarify these issues. Methods: Participants were recruited to participate in an online experiment that had a 3 (facial expression: smiling/neutral/surprised) × 2 (facial orientation: upright/inverted) mixed design. Participants were presented with an upright or inverted face for each expression (neutral, smiling, and surprised) and were asked to estimate the individual's age. Results: In total, 104 participants were included in the analysis. The results show that smiling faces were estimated to be older than neutral faces, whereas there was no significant difference between upright and inverted faces. Conclusions: Our findings suggest that AES is not dependent on holistic processing.

Robustness of the aging effect of smiling against vertical facial orientation

Background: Previous studies have shown that the association between smiling and youth is a misconception; smiling faces have been estimated to be older than neutral faces. Previous studies have indicated that this aging effect of smiling (AES) is due to eye wrinkles caused by the facial action of smiling. However, whether holistic processing for facial expressions is involved in AES has not been investigated. The present study aimed to clarify these issues. Methods: Participants were recruited to participate in an online experiment that had a 3 (facial expression: smiling/neutral/surprised) × 2 (facial orientation: upright/inverted) mixed design. Participants were presented with an upright or inverted face for each expression (neutral, smiling, and surprised) and were asked to estimate the individual's age. Results: In total, 104 participants were included in the analysis. The results show that smiling faces were estimated to be older than neutral faces, whereas there was no significant difference between upright and inverted faces. Conclusions: Our findings suggest that AES is not dependent on holistic processing.

Robustness of the aging effect of smiling against vertical facial orientation

Background: Previous studies have shown that the association between smiling and youth is a misconception; smiling faces have been estimated to be older than neutral faces. Previous studies have indicated that this aging effect of smiling (AES) is due to eye wrinkles caused by the facial action of smiling. However, whether holistic processing for facial expressions is involved in AES has not been investigated. The present study aimed to clarify these issues. Methods: Participants were recruited to participate in an online experiment that had a 3 (facial expression: smiling/neutral/surprised) × 2 (facial orientation: upright/inverted) mixed design. Participants were presented with an upright or inverted face for each expression (neutral, smiling, and surprised) and were asked to estimate the individual's age. Results: In total, 104 participants were included in the analysis. The results show that smiling faces were estimated to be older than neutral faces, whereas there was no significant difference between upright and inverted faces. Conclusions: Our findings suggest that direct age estimation is not dependent on holistic processing.

Different measures of holistic face processing tap into distinct but partially overlapping mechanisms

Holistic processing, which includes the integration of facial features and analysis of their relations to one another, is a hallmark of what makes faces 'special'. Various experimental paradigms purport to measure holistic processing but these have often produced inconsistent results. This has led researchers to question the nature and structure of the mechanism(s) underlying holistic processing. Using an individual differences approach, researchers have examined relations between various measures of holistic processing in an attempt to resolve these questions. In keeping with this, we examined relationships between four commonly used measures of holistic face processing in a large group of participants (N = 223): (1) The Face Inversion Effect, (2) the Part Whole Effect (PWE), (3) the Composite Face Effect, and (4) the Configural Featural Detection Task (CFDT). Several novel methodological and analytical elements were introduced, including the use of factor analysis and the inclusion of control conditions to confirm the face specificity of all of the effects measured. The four indexes of holistic processing derived from each measure loaded onto two factors, one encompassing the PWE and the CFDT, and one encompassing the CE. The 16 conditions tested across the four tasks loaded onto four factors, each factor corresponding to a different measure. These results, together with those of other studies, suggest that holistic processing is a multifaceted construct and that different measures tap into distinct but partially overlapping elements of it.

Synchronous facial action binds dynamic facial features

We asked how dynamic facial features are perceptually grouped. To address this question, we varied the timing of mouth movements relative to eyebrow movements, while measuring the detectability of a small temporal misalignment between a pair of oscillating eyebrows-an eyebrow wave. We found eyebrow wave detection performance was worse for synchronous movements of the eyebrows and mouth. Subsequently, we found this effect was specific to stimuli presented to the right visual field, implicating the involvement of left lateralised visual speech areas. Adaptation has been used as a tool in low-level vision to establish the presence of separable visual channels. Adaptation to moving eyebrows and mouths with various relative timings reduced eyebrow wave detection but only when the adapting mouth and eyebrows moved asynchronously. Inverting the face led to a greater reduction in detection after adaptation particularly for asynchronous facial motion at test. We conclude that synchronous motion binds dynamic facial features whereas asynchronous motion releases them, allowing adaptation to impair eyebrow wave detection.

Face and word composite effects are similarly affected by priming of local and global processing

Holistic processing has been shown with both faces and words, but it is unclear how similar their underlying mechanisms are. In this study attention to global and local features was manipulated and the consequences for holistic word and face processing were examined. On each trial participants were presented two Navon figures and told to focus on either the global or the local level. Then they performed a composite task in which they indicated whether the target halves of two sequentially presented faces or words were the same or different, ignoring the irrelevant halves. Similar stronger global priming effects were found for faces and words, indicating that holistic processing for the two types of stimuli were susceptible to attention manipulations to similar degrees, which was confirmed with Bayesian analyses. The findings add to the investigation of the similarity and differences between holistic processing and help reveal those aspects of holistic processing that are domain general and those specific to individual categories.

Face pareidolia in the brain: Impact of gender and orientation

Research on face sensitivity is of particular relevance during the rapidly evolving Covid-19 pandemic leading to social isolation, but also calling for intact interaction and sharing. Humans possess high sensitivity even to a coarse face scheme, seeing faces in non-face images where real faces do not exist. The advantage of non-face images is that single components do not trigger face processing. Here by implementing a novel set of Face-n-Thing images, we examined (i) how face tuning alters with changing display orientation, and (ii) whether it is affected by observers’ gender. Young females and males were presented with a set of Face-n-Thing images either with canonical upright orientation or inverted 180° in the image plane. Face impression was substantially impeded by display inversion. Furthermore, whereas with upright display orientation, no gender differences were found, with inversion, Face-n-Thing images elicited face impression in females significantly more often. The outcome sheds light on the origins of the face inversion effect in general. Moreover, the findings open a way for examination of face sensitivity and underwriting brain networks in neuropsychiatric conditions related to the current pandemic (such as depression and anxiety), most of which are gender/sex-specific.

Understanding the role of configural processing in face emotion recognition in Parkinson's disease

This investigation examined whether impairment in configural processing could explain deficits in face emotion recognition in people with Parkinson's disease (PD). Stimuli from the Radboud Faces Database were used to compare recognition of four negative emotion expressions by older adults with PD (n = 16) and matched controls (n = 17). Participants were tasked with categorizing emotional expressions from upright and inverted whole faces and facial composites; it is difficult to derive configural information from these two types of stimuli so featural processing should play a larger than usual role in accurate recognition of emotional expressions. We found that the PD group were impaired relative to controls in recognizing anger, disgust and fearful expressions in upright faces. Then, consistent with a configural processing deficit, participants with PD showed no composite effect when attempting to identify facial expressions of anger, disgust and fear. A face inversion effect, however, was observed in the performance of all participants in both the whole faces and facial composites tasks. These findings can be explained in terms of a configural processing deficit if it is assumed that the disruption caused by facial composites was specific to configural processing, whereas inversion reduced performance by making it difficult to derive both featural and configural information from faces.

Why Is an Inverted Car Identified Much Better Than an Inverted Face?

The present study shows that the presentation of a pair of the same car in upright and then in inverted orientation (UI-car) or in inverted and upright orientations (IU-car) increases recognition of an inverted car more than recognition of an inverted face given the previous presentation of a UI-face or IU-face. Presentation of a UI-face or IU-face exerts no significant effect on recognition of an inverted face and upright face. The differences between the visual forms of a face and a car and reasons why it is more difficult to unify a UI-face (IU-face) than a UI-car (IU-car) are discussed.

Holistic word processing is correlated with efficiency in visual word recognition

Holistic processing of visual words (i.e., obligatory encoding of/attending to all letters of a word) could be a marker of expert word recognition. In the present study, we thus examined for the first time whether there is a direct relation between the word-composite effect (i.e., all parts of a visual word are fully processed when observers perform a task on a word part) and fast access to the orthographic lexicon by visual word experts (i.e., fluent adult readers). We adopted an individual differences approach and used the word-frequency effect (i.e., faster recognition of high- than low-frequency words) in an independent lexical decision task as a proxy of fast access to lexical orthographic representations. Fluent readers with larger word-composite effect showed smaller word-frequency effect. This correlation was mainly driven by an association between a larger composite effect and faster lexical decision on low-frequency words, probably because these lexical representations are less stable and integrated/unitized, hence allowing differentiating among fluent readers. We thus showed that holistic processing of visual words is indeed related to higher efficiency in visual word recognition by skilled readers.

The Eye-Size Illusion Is Handled by Two Separate Processes

As a visual stimulus, the face is handled by 2 separate processes: the general visual and the face specific. The present study evinces for the first time that the former process is strengthened by the latter when a geometric illusion (handled by the general visual process) is embedded in a face (handled by the face-specific process). This theoretical approach is supported by the following main finding. When a geometric illusion is implanted in a face, the eye-size illusion is generated. Change in or gradual elimination of facial information diminishes the illusion. When all facial information is removed, one obtains a geometric illusion: the rectangle-size illusion, wherein the eyes are replaced by a black rectangle and all other facial features are stripped off. The eye-size illusion is much larger than the rectangle-size illusion. Further results provide additional support for the theoretical approach adopted by the present study.

Comparison of visual perceptual organization in schizophrenia and body dysmorphic disorder

People with schizophrenia are impaired at organizing potentially ambiguous visual information into well-formed shape and object representations. This perceptual organization (PO) impairment has not been found in other psychiatric disorders. However, recent data on body dysmorphic disorder (BDD), suggest that BDD may also be characterized by reduced PO. Similarities between these groups could have implications for understanding the RDoC dimension of visual perception in psychopathology, and for modeling symptom formation across these two conditions. We compared patients with SCZ (n=24) to those with BDD (n=20), as well as control groups of obsessive-compulsive disorder (OCD) patients (n=20) and healthy controls (n=20), on two measures of PO that have been reliably associated with schizophrenia-related performance impairment. On both the contour integration and Ebbinghaus illusion tests, only the SCZ group demonstrated abnormal performance relative to controls; the BDD group performed similarly to the OCD and CON groups. In addition, on both tasks, the SCZ group performed more abnormally than the BDD group. Overall, these data suggest that PO reductions observed in SCZ are not present in BDD. Visual processing impairments in BDD may arise instead from other perceptual disturbances or attentional biases related to emotional factors.

Effects of visual expertise on a novel eye-size illusion: implications for holistic face processing

We examined the effect of visual experience on the magnitude of a novel eye-size illusion: when the size of a face's frame is increased or decreased but eye size is unchanged, observers judge the size of the eyes to be different from that in the original face frame. In the current study, we asked Chinese and Caucasian participants to judge eye size in different pairs of faces and measured the magnitude of the illusion when the faces were own- or other-age (adult vs. infant faces) and when the faces were own- or other-race (Chinese vs. Caucasian faces). We found an other-age effect and an other-race effect with the eye-size illusion: The illusion was more pronounced with own-race and own-age faces than with other-race and other-age faces. These findings taken together suggest that visual experience with faces influences the magnitude of this novel illusion. Extensive experience with certain face categories strengthens the illusion in the context of these categories, but lack of it reduces the magnitude of the illusion. Our results further imply that holistic processing may play an important role in engendering the eye-size illusion.

Developmental origins of the face inversion effect

A hallmark of adults' expertise for faces is that they are better at recognizing, discriminating, and processing upright faces compared to inverted faces. We investigate the developmental origins of "the face inversion effect" by reviewing research on infants' perception of upright and inverted faces during the first year of life. We review the effects of inversion on infants' face preference, recognition, processing (holistic and second-order configural), and scanning as well as face-related neural responses. Particular attention is paid to the developmental patterns that emerge within and across these areas of face perception. We conclude that the developmental origins of the inversion effect begin in the first few months of life and grow stronger over the first year, culminating in effects that are commonly thought to indicate adult-like expertise. We posit that by the end of the first year, infants' face-processing system has become specialized to upright faces and a foundation for adults' upright-face expertise has been established. Developmental mechanisms that may facilitate the emergence of this upright-face specialization are discussed, including the roles that physical and social development may play in upright faces' becoming more meaningful to infants during the first year.

The eye-size illusion: psychophysical characteristics, generality, and relation to holistic face processing

Rakover [(2011). In Y. H. Zhang (Ed.), Advances in face image analysis: Techniques and technologies (pp. 316-333). Hershey, PA: IGI Global] observed a novel eye-size illusion: when increasing the size of a face but keeping the size of its eyes unchanged, the eyes are perceived to be smaller than in the original face. Here, we systematically manipulated the face size and found that the magnitude of this illusion linearly changed as a function of the face frame size (experiment 1). Additionally, the same magnitude of an illusion was observed for the perception of the size of the mouth when we changed the face frame but kept the mouth size constant (experiment 2). Further, when the faces and eyes were presented upside down, the magnitude of the illusion was significantly reduced in both Chinese participants (experiment 3) and Caucasian participants (experiment 4). The results suggest that the perception of eye or mouth size occurs in the relational context of the whole face; and when the face is inverted, thereby disrupting holistic processing, the magnitude of the illusion is reduced. We therefore suggest that holistic processing is involved in producing the illusion.