The role of holistic processing in face perception: evidence from the face inversion effect

Investigating the effect of wearing the hijab: Perception of facial attractiveness by Emirati Muslim women living in their native Muslim country

The Hijab and other forms of Islamic veiling are important social, cultural, and religious symbols that are central to the identity of millions of Muslim women across the world. However, despite the large body of literature that exists on the political and socio-cultural aspects of Islamic veiling, little is known about how the appearance of women wearing the hijab is perceived by other Muslim women within their native Muslim country. To throw light on this important issue, the current study focussed on the effects of the hijab on female facial attractiveness perceived by practising Muslim Emirati women living in their native Muslim country (the United Arab Emirates) who themselves wore the hijab as everyday attire. Participants were shown frontal-head images of women in three different conditions: covered (heads fully covered by the hijab except for the face), partially covered (heads fully covered by the hijab except for the face and the hair around the forehead) and uncovered (heads with no covering). The findings showed that faces in images where heads were covered and partially covered by the hijab were rated as equally attractive but both were rated as significantly less attractive than faces in images where heads were uncovered. These findings suggest that, even for practising Muslim Emirati females living in their native Muslim country for whom wearing the hijab is a normal aspect of everyday life, perception of facial attractiveness is compromised by wearing this garment. We argue that this effect of wearing the hijab is not consistent with a preference for one's own cultural group (cultural endogamy) and may, instead, occur because wearing a hijab occludes external features, such as hair and ears, which normally contribute to the perception of human facial attractiveness. In sum, while wearing the hijab may be dominated by male attitudes towards suppressing female attractiveness towards males, the findings from this study suggest that female Muslims too perceive the negative influence of wearing the hijab on female facial attractiveness.

Neurophysiological correlates of holistic face processing in adolescents with and without autism spectrum disorder

Background

Face processing has been found to be impaired in autism spectrum disorders (ASD). One hypothesis is that individuals with ASD engage in piecemeal compared to holistic face processing strategies. To investigate the role of possible impairments in holistic face processing in individuals with autism, the current study investigated behavioral and electroencephalography (EEG) correlates of face processing (P1/N170 and gamma-band activity) in adolescents with ASD and sex-, age-, and IQ-matched neurotypical controls.

Methods

Participants were presented with upright and inverted Mooney stimuli; black and white low information faces that are only perceived as faces when processed holistically. Participants indicated behaviorally the detection of a face. EEG was collected time-locked to the presentation of the stimuli.

Results

Adolescents with ASD perceived Mooney stimuli as faces suggesting ability to use holistic processing but displayed a lower face detection rate and slower response times. ERP components suggest slowed temporal processing of Mooney stimuli in the ASD compared to control group for P1 latency but no differences between groups for P1 amplitude and at the N170. Increases in gamma-band activity was similar during the perception of the Mooney images by group, but the ASD group showed prolonged temporal elevation in activity.

Conclusion

Overall, our results suggest that adolescents with ASD were able to utilize holistic processing to perceive a face within the Mooney stimuli. Delays in early processing, marked by the P1, and elongated elevation in gamma activity indicate that the neural systems supporting holistic processing are slightly altered suggesting a less automatic and less efficient facial processing system.

Trial registration

Non-applicable.

Electronic supplementary material

The online version of this article (10.1186/s11689-018-9244-y) contains supplementary material, which is available to authorized users.

Attending to behaviorally relevant moments enhances incidental relational memory

Memory for the items one has recently encountered is sometimes enhanced in divided attention tasks: Attending to behaviorally relevant items, such as a target in a detection task, boosts memory for unrelated background items (e.g., scenes or words). However, a central feature of episodic memory is memory for the spatiotemporal relationship between items and other elements of an event (relational memory), not just the item itself. Three experiments examined whether attending to a behaviorally relevant target-item boosts memory for the relationship between that item, its features, and a background scene. Participants memorized briefly presented scenes. At the same time, they pressed a button if a second unrelated item (a figure or face) was a particular target color (Experiments 1 and 2) or target gender (Experiment 3) rather than a distractor color or gender. Target and distractor items also varied in task-irrelevant features (shape, location, or facial identity). If attending to behaviorally relevant events influences relational memory, then participants should be better able to report both target-defining and irrelevant features of items that appeared with target-paired scenes rather than distractor-paired scenes. This was the case in all experiments: memory was enhanced for the target-paired scenes as well as the association between a scene and features of the paired target-item. Attending to behaviorally relevant moments therefore has broader effects on memory encoding than previously thought. In addition to boosting memory for unrelated background items, attending to targets facilitates relational memory in these tasks.

Does Holistic Processing Require a Large Brain? Insights From Honeybees and Wasps in Fine Visual Recognition Tasks

The expertise of humans for recognizing faces is largely based on holistic processing mechanism, a sophisticated cognitive process that develops with visual experience. The various visual features of a face are thus glued together and treated by the brain as a unique stimulus, facilitating robust recognition. Holistic processing is known to facilitate fine discrimination of highly similar visual stimuli, and involves specialized brain areas in humans and other primates. Although holistic processing is most typically employed with face stimuli, subjects can also learn to apply similar image analysis mechanisms when gaining expertise in discriminating novel visual objects, like becoming experts in recognizing birds or cars. Here, we ask if holistic processing with expertise might be a mechanism employed by the comparatively miniature brains of insects. We thus test whether honeybees (Apis mellifera) and/or wasps (Vespula vulgaris) can use holistic-like processing with experience to recognize images of human faces, or Navon-like parameterized-stimuli. These insect species are excellent visual learners and have previously shown ability to discriminate human face stimuli using configural type processing. Freely flying bees and wasps were consequently confronted with classical tests for holistic processing, the part-whole effect and the composite-face effect. Both species could learn similar faces from a standard face recognition test used for humans, and their performance in transfer tests was consistent with holistic processing as defined for studies on humans. Tests with parameterized stimuli also revealed a capacity of honeybees, but not wasps, to process complex visual information in a holistic way, suggesting that such sophisticated visual processing may be far more spread within the animal kingdom than previously thought, although may depend on ecological constraints.

Spatial Mechanisms within the Dorsal Visual Pathway Contribute to the Configural Processing of Faces

Human face recognition is often attributed to configural processing; namely, processing the spatial relationships among the features of a face. If configural processing depends on fine-grained spatial information, do visuospatial mechanisms within the dorsal visual pathway contribute to this process? We explored this question in human adults using functional magnetic resonance imaging and transcranial magnetic stimulation (TMS) in a same-different face detection task. Within localized, spatial-processing regions of the posterior parietal cortex, configural face differences led to significantly stronger activation compared to featural face differences, and the magnitude of this activation correlated with behavioral performance. In addition, detection of configural relative to featural face differences led to significantly stronger functional connectivity between the right FFA and the spatial processing regions of the dorsal stream, whereas detection of featural relative to configural face differences led to stronger functional connectivity between the right FFA and left FFA. Critically, TMS centered on these parietal regions impaired performance on configural but not featural face difference detections. We conclude that spatial mechanisms within the dorsal visual pathway contribute to the configural processing of facial features and, more broadly, that the dorsal stream may contribute to the veridical perception of faces.

Sensitivity to Faces with Typical and Atypical Part Configurations within Regions of the Face-processing Network: An fMRI Study

Perception of faces has been shown to engage a domain-specific set of brain regions, including the occipital face area (OFA) and the fusiform face area (FFA). It is commonly held that the OFA is responsible for the detection of faces in the environment, whereas the FFA is responsible for processing the identity of the face. However, an alternative model posits that the FFA is responsible for face detection and subsequently recruits the OFA to analyze the face parts in the service of identification. An essential prediction of the former model is that the OFA is not sensitive to the arrangement of internal face parts. In the current fMRI study, we test the sensitivity of the OFA and FFA to the configuration of face parts. Participants were shown faces in which the internal parts were presented in a typical configuration (two eyes above a nose above a mouth) or in an atypical configuration (the locations of individual parts were shuffled within the face outline). Perception of the atypical faces evoked a significantly larger response than typical faces in the OFA and in a wide swath of the surrounding posterior occipitotemporal cortices. Surprisingly, typical faces did not evoke a significantly larger response than atypical faces anywhere in the brain, including the FFA (although some subthreshold differences were observed). We propose that face processing in the FFA results in inhibitory sculpting of activation in the OFA, which accounts for this region's weaker response to typical than to atypical configurations.

Face-specific memory deficits and changes in eye scanning patterns among patients with amnestic mild cognitive impairment

Amnestic mild cognitive impairment (aMCI) is a prodromal stage of Alzheimer's disease (AD). Previous studies have shown functional and structural degradation of the fusiform face area, which is a core region for face processing, in addition to medial temporal lobe degradation. We predicted that patients with aMCI exhibit a loss of face processing and/or face memory, accompanied by abnormal eye scanning patterns, since patients who have deficits in face perception (i.e. prosopagnosia) exhibit such tendencies. Eighteen patients with aMCI and age-matched healthy controls were tested for perception and short-term memory of visually presented faces and houses while their gaze was recorded. Patients with aMCI showed a decline in memory, compared with control observers, for faces, but not for houses. Patients looked more at the mouth of faces, compared with control observers. We demonstrate here the loss of short-term face memory in aMCI with abnormal scanning patterns that might reflect the cerebral abnormality found in patients with aMCI.

Face Pareidolia in the Rhesus Monkey

Face perception in humans and nonhuman primates is rapid and accurate [1-4]. In the human brain, a network of visual-processing regions is specialized for faces [5-7]. Although face processing is a priority of the primate visual system, face detection is not infallible. Face pareidolia is the compelling illusion of perceiving facial features on inanimate objects, such as the illusory face on the surface of the moon. Although face pareidolia is commonly experienced by humans, its presence in other species is unknown. Here we provide evidence for face pareidolia in a species known to possess a complex face-processing system [8-10]: the rhesus monkey (Macaca mulatta). In a visual preference task [11, 12], monkeys looked longer at photographs of objects that elicited face pareidolia in human observers than at photographs of similar objects that did not elicit illusory faces. Examination of eye movements revealed that monkeys fixated the illusory internal facial features in a pattern consistent with how they view photographs of faces [13]. Although the specialized response to faces observed in humans [1, 3, 5-7, 14] is often argued to be continuous across primates [4, 15], it was previously unclear whether face pareidolia arose from a uniquely human capacity. For example, pareidolia could be a product of the human aptitude for perceptual abstraction or result from frequent exposure to cartoons and illustrations that anthropomorphize inanimate objects. Instead, our results indicate that the perception of illusory facial features on inanimate objects is driven by a broadly tuned face-detection mechanism that we share with other species.

Interocularly merged face percepts eliminate binocular rivalry

Faces are important visual objects for humans and other social animals. A complex network of specialized brain areas is involved in the recognition and interpretation of faces. This network needs to strike a balance between being sensitive enough to distinguish between different faces with similar features, and being tolerant of low-level visual changes so that a given face is stably perceived as a particular individual. Such stability may require feedback from higher brain regions down to the level where details are represented. Here, we describe a phenomenon in which interocular competition between face features is stabilized and eliminated when observers attend high-level face characteristics. Two different face images presented to the individual eyes do not cause the perceptual fluctuations that are typically observed in binocular rivalry. Instead, they merge into a stable percept of an intermediate face that combines features from both eyes’ images. The stability of the intermediate face percept depends on the observer attending holistic face properties such as identity or gender. It disappears when observers explicitly attend facial features, suggesting a crucial role of top-down stabilizing feedback from high-level areas that represent holistic faces back to lower processing levels where detailed face features compete for conscious representation.

3D faces are recognized more accurately and faster than 2D faces, but with similar inversion effects

Recognition of faces typically occurs via holistic processing where individual features are combined to provide an overall facial representation. However, when faces are inverted, there is greater reliance on featural processing where faces are recognized based on their individual features. These findings are based on a substantial number of studies using 2-dimensional (2D) faces and it is unknown whether these results can be extended to 3-dimensional (3D) faces, which have more depth information that is absent in the typical 2D stimuli used in face recognition literature. The current study used the face inversion paradigm as a means to investigate how holistic and featural processing are differentially influenced by 2D and 3D faces. Twenty-five participants completed a delayed face-matching task consisting of upright and inverted faces that were presented as both 2D and 3D stereoscopic images. Recognition accuracy was significantly higher for 3D upright faces compared to 2D upright faces, providing support that the enriched visual information in 3D stereoscopic images facilitates holistic processing that is essential for the recognition of upright faces. Typical face inversion effects were also obtained, regardless of whether the faces were presented in 2D or 3D. Moreover, recognition performances for 2D inverted and 3D inverted faces did not differ. Taken together, these results demonstrated that 3D stereoscopic effects influence face recognition during holistic processing but not during featural processing. Our findings therefore provide a novel perspective that furthers our understanding of face recognition mechanisms, shedding light on how the integration of stereoscopic information in 3D faces influences face recognition processes.

Face Context Influences Local Part Processing: An ERP Study

Perception of face parts on the basis of features is thought to be different from perception of whole faces, which is more based on configural information. Face context is also suggested to play an important role in face processing. To investigate how face context influences the early-stage perception of facial local parts, we used an oddball paradigm that tested perceptual stages of face processing rather than recognition. We recorded the event-related potentials (ERPs) elicited by whole faces and face parts presented in four conditions (upright-normal, upright-thatcherised, inverted-normal and inverted-thatcherised), as well as the ERPs elicited by non-face objects (whole houses and house parts) with corresponding conditions. The results showed that face context significantly affected the N170 with increased amplitudes and earlier peak latency for upright normal faces. Removing face context delayed the P1 latency but did not affect the P1 amplitude prominently for both upright and inverted normal faces. Across all conditions, neither the N170 nor the P1 was modulated by house context. The significant changes on the N170 and P1 components revealed that face context influences local part processing at the early stage of face processing and this context effect might be specific for face perception. We further suggested that perceptions of whole faces and face parts are functionally distinguished.

Mnemonic discrimination of similar face stimuli and a potential mechanism for the "other race" effect

Face recognition is an important component of successful social interactions in humans. A large literature in social psychology has focused on the phenomenon termed the "other race" (ORE) effect, the tendency to be more proficient with face recognition within one's own ethnic group compared with other ethnic groups. Several potential hypotheses have been proposed for this effect, including perceptual expertise, social grouping, and holistic face processing. Recent work on mnemonic discrimination (i.e., the ability to resolve mnemonic interference among similar experiences) may provide a mechanistic account for the ORE. In the current study, we examined how discrimination and generalization in the presence of mnemonic interference may contribute to the ORE. We developed a database of computerized faces divided evenly among ethnic origins (Black, Caucasian, East Asian, South Asian), as well as morphed face stimuli that varied in the amount of similarity to the original stimuli (30%, 40%, 50%, and 60% morphs). Participants first examined the original unmorphed stimuli during study, then during test were asked to judge the prior occurrence of repetitions (targets), morphed stimuli (lures), and new stimuli (foils). We examined participants' ability to correctly reject similar morphed lures and found that it increased linearly as a function of face dissimilarity. We additionally found that Caucasian participants' mnemonic discrimination-generalization functions were sharply tuned for Caucasian faces but considerably less tuned for East Asian and Black faces. These results suggest that expertise plays an important role in resolving mnemonic interference, which may offer a mechanistic account for the ORE.

The cerebral correlates of subliminal emotions: an eleoencephalographic study with emotional hybrid faces

In a high-resolution electroencephalographic study, participants evaluated the friendliness level of upright and inverted 'hybrid faces', i.e. facial photos containing a subliminal emotional core in the low spatial frequencies (< 6 cycles/image), superimposed on a neutral expression in the rest of the spatial frequencies. Upright happy and angry faces were judged as more friendly or less friendly than neutral faces, respectively. We observed the time course of cerebral correlates of these stimuli with event-related potentials (ERPs), confirming that hybrid faces elicited the posterior emotion-related and face-related components (P1, N170 and P2), previously shown to be engaged by non-subliminal emotional stimuli. In addition, these components were stronger in the right hemisphere and were both enhanced and delayed by face inversion. A frontal positivity (210-300 ms) was stronger for emotional than for neutral faces, and for upright than for inverted faces. Hence, hybrid faces represent an original approach in the study of subliminal emotions, which appears promising for investigating their electrophysiological correlates.

Comparison of visual perceptual organization in schizophrenia and body dysmorphic disorder

People with schizophrenia are impaired at organizing potentially ambiguous visual information into well-formed shape and object representations. This perceptual organization (PO) impairment has not been found in other psychiatric disorders. However, recent data on body dysmorphic disorder (BDD), suggest that BDD may also be characterized by reduced PO. Similarities between these groups could have implications for understanding the RDoC dimension of visual perception in psychopathology, and for modeling symptom formation across these two conditions. We compared patients with SCZ (n=24) to those with BDD (n=20), as well as control groups of obsessive-compulsive disorder (OCD) patients (n=20) and healthy controls (n=20), on two measures of PO that have been reliably associated with schizophrenia-related performance impairment. On both the contour integration and Ebbinghaus illusion tests, only the SCZ group demonstrated abnormal performance relative to controls; the BDD group performed similarly to the OCD and CON groups. In addition, on both tasks, the SCZ group performed more abnormally than the BDD group. Overall, these data suggest that PO reductions observed in SCZ are not present in BDD. Visual processing impairments in BDD may arise instead from other perceptual disturbances or attentional biases related to emotional factors.

Disrupting perceptual grouping of face parts impairs holistic face processing

Face perception is widely believed to involve integration of facial features into a holistic perceptual unit, but the mechanisms underlying this integration are relatively unknown. We examined whether perceptual grouping cues influence a classic marker of holistic face perception, the “composite-face effect.” Participants made same–different judgments about a cued part of sequentially presented chimeric faces, and holistic processing was indexed as the degree to which the task-irrelevant face halves impacted performance. Grouping was encouraged or discouraged by adjusting the backgrounds behind the face halves: Although the face halves were always aligned, their respective backgrounds could be misaligned and of different colors. Holistic processing of face, but not of nonface, stimuli was significantly reduced when the backgrounds were misaligned and of different colors, cues that discouraged grouping of the face halves into a cohesive unit (Exp. 1). This effect was sensitive to stimulus orientation at short (200 ms) but not at long (2,500 ms) encoding durations, consistent with the previously documented temporal properties of the holistic processing of upright and inverted faces (Exps. 2 and 3). These results suggest that grouping mechanisms, typically involved in the perception of objecthood more generally, might contribute in important ways to the holistic perception of faces.

Novel Inversions in Auditory Sequences Provide Evidence for Spontaneous Subtraction of Time and Number

Animals, including fish, birds, rodents, non-human primates, and pre-verbal infants are able to discriminate the duration and number of events without the use of language. In this paper, we present the results of six experiments exploring the capability of adult rats to count 2–6 sequentially presented white-noise stimuli. The investigation focuses on the animal’s ability to exhibit spontaneous subtraction following the presentation of novel stimulus inversions in the auditory signals being counted. Results suggest that a subtraction operation between two opposite sensory representations may be a general processing strategy used for the comparison of stimulus magnitudes. These findings are discussed within the context of a mode-control model of timing and counting that relies on an analog temporal-integration process for the addition and subtraction of sequential events.

Cognitive processing of scrambled faces

We propose a model, face-checking, that is based on first-order relational properties and discriminates between different faces (regular and scrambled) and predicts their similarity choice in 2 orientations: upright and inverted. The results of an experiment confirmed the model's predictive ability. An alternative hypothesis, which is based on facial configural information, failed to provide an efficient explanation for these findings, as the proposed model does. The model shows how components of first-order relational properties are mathematically integrated and involved in face processing.

Stimulus requirements for face perception: an analysis based on "totem poles"

The stimulus requirements for perceiving a face are not well defined but are presumably simple, for vivid faces can often by seen in random or natural images such as cloud or rock formations. To characterize these requirements, we measured where observers reported the impression of faces in images defined by symmetric 1/f noise. This allowed us to examine the prominence and properties of different features and their necessary configurations. In these stimuli many faces can be perceived along the vertical midline, and appear stacked at multiple scales, reminiscent of "totem poles." In addition to symmetry, the faces in noise are invariably upright and thus reveal the inversion effects that are thought to be a defining property of configural face processing. To a large extent, seeing a face required seeing eyes, and these were largely restricted to dark regions in the images. Other features were more subordinate and showed relatively little bias in polarity. Moreover, the prominence of eyes depended primarily on their luminance contrast and showed little influence of chromatic contrast. Notably, most faces were rated as clearly defined with highly distinctive attributes, suggesting that once an image area is coded as a face it is perceptually completed consistent with this interpretation. This suggests that the requisite trigger features are sufficient to holistically "capture" the surrounding noise structure to form the facial representation. Yet despite these well articulated percepts, we show in further experiments that while a pair of dark spots added to noise images appears face-like, these impressions fail to elicit other signatures of face processing, and in particular, fail to elicit an N170 or fixation patterns typical for images of actual faces. These results suggest that very simple stimulus configurations are sufficient to invoke many aspects of holistic and configural face perception while nevertheless failing to fully engage the neural machinery of face coding, implying that that different signatures of face processing may have different stimulus requirements.

Strabismic amblyopia affects relational but not featural and Gestalt processing of faces

The ability to identify faces is of critical importance for normal social interactions. Previous evidence suggests that early visual deprivation may impair certain aspects of face recognition. The effects of strabismic amblyopia on face processing have not been investigated previously. In this study, a group of individuals with amblyopia were administered two tasks known to selectively measure face detection based on a Gestalt representation of a face (Mooney faces task) and featural and relational processing of faces (Jane faces task). Our data show that--when relying on their amblyopic eye only - strabismic amblyopes perform as well as normally sighted individuals in face detection and recognition on the basis of their single features. However, they are significantly impaired in discriminating among different faces on the basis of the spacing of their single features (i.e., configural processing of relational information). Our findings are the first to demonstrate that strabismic amblyopia may cause specific deficits in face recognition, and add to previous reports characterizing visual perceptual deficits associated in amblyopia as high-level and not only as low-level processing.

The Composite-Face Effect Survives Asymmetric Face Distortions

In two experiments, we investigated whether adults use holistic processing even for faces that are grossly distorted because their eyes have been moved asymmetrically to violate the common layout of a face (distorting its first-order relations). To this end we used a compelling demonstration that faces are processed as wholes, the composite-face effect. Specifically, adults judged the similarity of sequentially presented top halves of normal (original condition) and distorted faces with one eye (one-eye condition) or two eyes (two-eyes condition) shifted up by an abnormal amount. Trials were either blocked by type of distortion (experiment 1) or intermixed within the experiment (experiment 2). In both experiments, participants demonstrated a composite-face effect of the same magnitude in the three conditions, a pattern suggesting that they processed holistically even faces whose first-order relations were violated.