Greater reliance on the eye region predicts better face recognition ability

A Biological Approach to Periorbital Aesthetics in Caucasian Females: A Review of the Literature

The face plays an important role in human interactions, and the periorbital region is particularly important for recognition and attractiveness. There are several studies on the beauty of the periorbital region with a variety of methodologies, but few articles consider the attractiveness factors derived from evolutionary psychology such as symmetry, dimorphism, age and average, neoteny, and facial expression. The aim of this study was to identify periorbital attractiveness criteria in Caucasian females based on experimental studies and to interpret them in the light of studies on biological attractiveness factors. A review of literature was conducted with PubMed, the Cochrane Library Database, and EMBASE. Studies published after March 20, 2022, were analyzed, and no date limit was applied to reference papers. The search strategy was focused on 3 main concepts: attractiveness AND evaluation AND facial feature of periorbital region. A total of 780 articles were identified with this search strategy; 534 were excluded based on title and abstract and another 110 after full text assessment. Eighteen articles were finally included in the analysis. The main factors identified were an ascending intercanthal axis and eyebrow axis, a regular pretarsal plate and a well-defined upper lid crease, no scleral show, and an open eye fissure exposing the upper half of the iris. In our review we were able to identify objective anthropometric characteristics linked to biological attractiveness criteria that may be helpful in preoperative planning and assessment of the periorbital region of patients.

LEVEL OF EVIDENCE: 3

Differences in eye movements for face recognition between Canadian and Chinese participants are not modulated by social orientation

Face recognition strategies do not generalize across individuals. Many studies have reported robust cultural differences between West Europeans/North Americans and East Asians in eye movement strategies during face recognition. The social orientation hypothesis posits that individualistic vs. collectivistic (IND/COL) value systems, respectively defining West European/North American and East Asian societies, would be at the root of many cultural differences in visual perception. Whether social orientation is also responsible for such cultural contrast in face recognition remains to be clarified. To this aim, we conducted two experiments with West European/North American and Chinese observers. In Experiment 1, we probed the existence of a link between IND/COL social values and eye movements during face recognition, by using an IND/COL priming paradigm. In Experiment 2, we dissected the latter relationship in greater depth, by using two IND/COL questionnaires, including subdimensions to those concepts. In both studies, cultural differences in fixation patterns were revealed between West European/North American and East Asian observers. Priming IND/COL values did not modulate eye movement visual sampling strategies, and only specific subdimensions of the IND/COL questionnaires were associated with distinct eye-movement patterns. Altogether, we show that the typical contrast between IND/COL cannot fully account for cultural differences in eye movement strategies for face recognition. Cultural differences in eye movements for faces might originate from mechanisms distinct from social orientation.

High familiar faces have both eye recognition and holistic processing advantages

People recognize familiar faces better than unfamiliar faces. However, it remains unknown whether familiarity affects part-based and/or holistic processing. Wang et al., Frontiers in Psychology, 6, 559 (2015), Vision Research, 157, 89-96 (2019) found both enhanced part-based and holistic processing in eye relative to mouth regions (i.e., in a region-selective manner) for own-race and own-species faces, i.e., faces with more experience. Here, we examined the role of face familiarity in eyes (part-based, region-selective) and holistic processing. Face familiarity was tested at three levels: high-familiar (faces of students from the same department and the same class who attended almost all courses together), low-familiar (faces of students from the same department but different classes who attended some courses together), and unfamiliar (faces of schoolmates from different departments who seldom attended the same courses). Using the old/new task in Experiment 1, we found that participants recognized eyes of high-familiar faces better than low-familiar and unfamiliar ones, while similar performance was observed for mouths, indicating a region-selective, eyes familiarity effect. Using the "Perceptual field" paradigm in Experiment 2, we observed a stronger inversion effect for high-familiar faces, a weaker inversion effect for low-familiar faces, but a non-significant inversion effect for unfamiliar faces, indicating that face familiarity plays a role in holistic processing. Taken together, our results suggest that familiarity, like other experience-based variables (e.g., race and species), can improve both eye processing and holistic processing.

Stable eye versus mouth preference in a live speech-processing task

Looking at the mouth region is thought to be a useful strategy for speech-perception tasks. The tendency to look at the eyes versus the mouth of another person during speech processing has thus far mainly been studied using screen-based paradigms. In this study, we estimated the eye-mouth-index (EMI) of 38 adult participants in a live setting. Participants were seated across the table from an experimenter, who read sentences out loud for the participant to remember in both a familiar (English) and unfamiliar (Finnish) language. No statistically significant difference in the EMI between the familiar and the unfamiliar languages was observed. Total relative looking time at the mouth also did not predict the number of correctly identified sentences. Instead, we found that the EMI was higher during an instruction phase than during the speech-processing task. Moreover, we observed high intra-individual correlations in the EMI across the languages and different phases of the experiment. We conclude that there are stable individual differences in looking at the eyes versus the mouth of another person. Furthermore, this behavior appears to be flexible and dependent on the requirements of the situation (speech processing or not).

The development of processing second-order spatial relations of faces in Chinese preschoolers

Second-order relational information processing is the perception of the relative distance between facial features. Previous studies ignored the effect of different spatial manipulations on second-order sensitivity in face processing, and little is known about its developmental trajectory in East Asian populations, who have stronger holistic face processing than Western populations. We addressed these gaps in the literature through an experiment with four groups of Chinese preschool children (aged 3-6 years; n = 157) and a group of adults (n = 25). The participants were presented with face pairs displaying features with various spatial distance manipulations (Change 1: changes in the spacing between eyes; Change 2: nose-mouth spacing changes; Change 3: a combination of Changes 1 and 2) using a simultaneous two-alternative forced-choice task. Second-order sensitivity was already present in 3-year-old children across all manipulations and became more pronounced in 4-year-old children. Second-order sensitivity to the spatial distance between the eyes (i.e., Changes 1 and 3) among 4-year-olds was higher than that of 3-year-olds and was similar to that of adults, suggesting a key increase of this sensitivity from 3 to 4 years of age. Regarding the Change 2 condition, preschoolers aged 5 and 6 years had higher sensitivity than 3-year-olds; however, all preschoolers' sensitivity was inferior to that of adults. These findings show that the development of Chinese preschoolers' sensitivity for detecting spatial relations between the eyes might be faster than that for detecting nose-mouth spacing, supporting the importance of eyes in face processing.

Differences Between East Asians and Westerners in the Mental Representations and Visual Information Extraction Involved in the Decoding of Pain Facial Expression Intensity

Effectively communicating pain is crucial for human beings. Facial expressions are one of the most specific forms of behavior associated with pain, but the way culture shapes expectations about the intensity with which pain is typically facially conveyed, and the visual strategies deployed to decode pain intensity in facial expressions, is poorly understood. The present study used a data-driven approach to compare two cultures, namely East Asians and Westerners, with respect to their mental representations of pain facial expressions (experiment 1, N=60; experiment 2, N=74) and their visual information utilization during the discrimination of facial expressions of pain of different intensities (experiment 3; N=60). Results reveal that compared to Westerners, East Asians expect more intense pain expressions (experiments 1 and 2), need more signal, and do not rely as much as Westerners on core facial features of pain expressions to discriminate between pain intensities (experiment 3). Together, those findings suggest that cultural norms regarding socially accepted pain behaviors shape the expectations about pain facial expressions and decoding visual strategies. Furthermore, they highlight the complexity of emotional facial expressions and the importance of studying pain communication in multicultural settings.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-023-00186-1.

Analysing the brain networks corresponding to the facial contrast-chimeras

How humans recognise faces and objects effortlessly, has become a great point of interest. To understand the underlying process, one of the approaches is to study the facial features, in particular ordinal contrast relations around the eye region, which plays a crucial role in face recognition and perception. Recently the graph-theoretic approaches to electroencephalogram (EEG) analysis are found to be effective in understating the underlying process of human brain while performing various tasks. We have explored this approach in face recognition and perception to know the importance of contrast features around the eye region. We studied functional brain networks, formed using EEG responses, corresponding to four types of visual stimuli with varying contrast relationships: Positive faces, chimeric faces (photo-negated faces, preserving the polarity of contrast relationships around eyes), photo-negated faces and only eyes. We observed the variations in brain networks of each type of stimuli by finding the distribution of graph distances across brain networks of all subjects. Moreover, our statistical analysis shows that positive and chimeric faces are equally easy to recognise in contrast to difficult recognition of negative faces and only eyes.

Face masks are less effective than sunglasses in masking face identity

The effect of covering faces on face identification is recently garnering interest amid the COVID-19 pandemic. Here, we investigated how face identification performance was affected by two types of face disguise: sunglasses and face masks. Observers studied a series of faces; then judged whether a series of test faces, comprising studied and novel faces, had been studied before or not. Face stimuli were presented either without coverings (full faces), wearing sunglasses covering the upper region (eyes, eyebrows), or wearing surgical masks covering the lower region (nose, mouth, chin). We found that sunglasses led to larger reductions in sensitivity (d') to face identity than face masks did, while both disguises increased the tendency to report faces as studied before, a bias that was absent for full faces. In addition, faces disguised during either study or test only (i.e. study disguised faces, test with full faces; and vice versa) led to further reductions in sensitivity from both studying and testing with disguised faces, suggesting that congruence between study and test is crucial for memory retrieval. These findings implied that the upper region of the face, including the eye-region features, is more diagnostic for holistic face-identity processing than the lower face region.

Looking at the upper facial half enlarges the range of holistic face processing

Previous studies suggested that upper and lower facial halves might be involved in the human holistic face processing differently. In this study, we replicated and extended the finding above. In Experiment 1, we used the standard composite-face task to measure holistic face processing when participants made judgements on the upper and lower facial halves separately. Results showed that the composite-face effect was stronger for the upper facial half compared to the lower half. In Experiment 2, we investigated how facial information was integrated when participants focused on different features, using the perceptual field paradigm. Results showed that: (1) more "peripheral faces" were chosen when participants fixated at the eyes than when they fixated at the mouth; (2) less "peripheral faces" were chosen for inverted faces regardless of the fixated features. Findings from both experiments together indicate that more peripheral facial information were integrated when participants focused on the upper facial half, highlighting the significance of focusing on the upper facial half in face processing.

The mechanisms supporting holistic perception of words and faces are not independent

The question of whether word and face recognition rely on overlapping or dissociable neural and cognitive mechanisms received considerable attention in the literature. In the present work, we presented words (aligned or misaligned) superimposed on faces (aligned or misaligned) and tested the interference from the unattended stimulus category on holistic processing of the attended category. In Experiment 1, we found that holistic face processing is reduced when a face was overlaid with an unattended, aligned word (processed holistically). In Experiment 2, we found a similar reduction of holistic processing for words when a word was superimposed on an unattended, aligned face (processed holistically). This reciprocal interference effect indicates a trade-off in holistic processing of the two stimuli, consistent with the idea that word and face recognition may rely on non-independent, overlapping mechanisms.

Recognition of Masked Faces in the Era of the Pandemic: No Improvement Despite Extensive Natural Exposure

Face masks, which became prevalent across the globe during the COVID-19 pandemic, have had a negative impact on face recognition despite the availability of critical information from uncovered face parts, especially the eyes. An outstanding question is whether face-mask effects would be attenuated following extended natural exposure. This question also pertains, more generally, to face-recognition training protocols. We used the Cambridge Face Memory Test in a cross-sectional study ( N = 1,732 adults) at six different time points over a 20-month period, alongside a 12-month longitudinal study ( N = 208). The results of the experiments revealed persistent deficits in recognition of masked faces and no sign of improvement across time points. Additional experiments verified that the amount of individual experience with masked faces was not correlated with the mask effect. These findings provide compelling evidence that the face-processing system does not easily adapt to visual changes in face stimuli, even following prolonged real-life exposure.

Face-Information Sampling in Super-Recognizers

Perceptual processes underlying individual differences in face-recognition ability remain poorly understood. We compared visual sampling of 37 adult super-recognizers-individuals with superior face-recognition ability-with that of 68 typical adult viewers by measuring gaze position as they learned and recognized unfamiliar faces. In both phases, participants viewed faces through "spotlight" apertures that varied in size, with face information restricted in real time around their point of fixation. We found higher accuracy in super-recognizers at all aperture sizes-showing that their superiority does not rely on global sampling of face information but is also evident when they are forced to adopt piecemeal sampling. Additionally, super-recognizers made more fixations, focused less on eye region, and distributed their gaze more than typical viewers. These differences were most apparent when learning faces and were consistent with trends we observed across the broader ability spectrum, suggesting that they are reflective of factors that vary dimensionally in the broader population.

Detection & recognition of veiled and unveiled human face on the basis of eyes using transfer learning

Face detection and recognition are the most substantial research areas in computer vision and transfer learning due to the inspiring nature of faces as an object. In this paper, we show that we can obtain promising results on the standard face databanks when the features are extracted merely from the eye. The contributions of this work are divided into three parts, specifically face detection, eyes detection and recognition for individual identification. The key features for face recognition, used in this study are the eyes, nostrils, and mouth. The key features for eyes recognition are center of left eye, center of right eye, midpoint of eyes and extraction of eyebrows. Extracted Local Binary Pattern Histogram (LBPH) method is used to extract the facial features of face images whose computational complexity is very low and these features contain simple pixel values. Furthermore, neighborhood pixels are calculated to extract effective facial feature to realize eyes recognition and person verification. This study is able to identify an individual on the basis of even a single eye. The algorithm finds the brighter eye from the face and then, on the basis of that eye, the person is identified and the name of person is provided. The experimental results of this study show that faces are recognized accurately and LBPH method has achieved 98.2% accuracy.

Normal gaze processing in developmental prosopagnosia

Two key functions in human face perception are gaze discrimination and identity recognition. Here we examine whether gaze discrimination can be intact when identity recognition is impaired in developmental prosopagnosia (DP). We ran a large sample of developmental prosopagnosics (DPs) with a series of gaze discrimination tasks that assess various mechanisms in gaze processing. Experiment 1 (N = 101 DP participants) investigates spatial processing using an abnormal eye gaze detection task and a Wollaston illusion task that measures perceptual integration of eye and head direction. Experiment 2 (N = 45 DP participants) investigates temporal processing using an adaptation task and a serial dependence task. Despite their deficits with identity recognition, DPs performed in the normal range across both experiments. These results demonstrate that gaze discrimination can be normal in DP, and that various mechanisms of gaze processing can be spared when identity recognition is impaired. Our findings clarify the highly selective nature of impairments in DP and provide support for neurocognitive models of face perception with distinct mechanisms for gaze and identity processing.

Emotion Facial Processing in Children With Autism Spectrum Disorder: A Pilot Study of the Impact of Service Dogs

Processing and recognizing facial expressions are key factors in human social interaction. Past research suggests that individuals with autism spectrum disorder (ASD) present difficulties to decode facial expressions. Those difficulties are notably attributed to altered strategies in the visual scanning of expressive faces. Numerous studies have demonstrated the multiple benefits of exposure to pet dogs and service dogs on the interaction skills and psychosocial development of children with ASD. However, no study has investigated if those benefits also extend to the processing of facial expressions. The aim of this study was to investigate if having a service dog had an influence on facial expression processing skills of children with ASD. Two groups of 15 children with ASD, with and without a service dog, were compared using a facial expression recognition computer task while their ocular movements were measured using an eye-tracker. While the two groups did not differ in their accuracy and reaction time, results highlighted that children with ASD owning a service dog directed less attention toward areas that were not relevant to facial expression processing. They also displayed a more differentiated scanning of relevant facial features according to the displayed emotion (i.e., they spent more time on the mouth for joy than for anger, and vice versa for the eyes area). Results from the present study suggest that having a service dog and interacting with it on a daily basis may promote the development of specific visual exploration strategies for the processing of human faces.

Language statistics as a window into mental representations

Large-scale linguistic data is nowadays available in abundance. Using this source of data, previous research has identified redundancies between the statistical structure of natural language and properties of the (physical) world we live in. For example, it has been shown that we can gauge city sizes by analyzing their respective word frequencies in corpora. However, since natural language is always produced by human speakers, we point out that such redundancies can only come about indirectly and should necessarily be restricted cases where human representations largely retain characteristics of the physical world. To demonstrate this, we examine the statistical occurrence of words referring to body parts in very different languages, covering nearly 4 billions of native speakers. This is because the convergence between language and physical properties of the stimuli clearly breaks down for the human body (i.e., more relevant and functional body parts are not necessarily larger in size). Our findings indicate that the human body as extracted from language does not retain its actual physical proportions; instead, it resembles the distorted human-like figure known as the sensory homunculus, whose form depicts the amount of cortical area dedicated to sensorimotor functions of each body part (and, thus, their relative functional relevance). This demonstrates that the surface-level statistical structure of language opens a window into how humans represent the world they live in, rather than into the world itself.

EXPRESS: Distinct abilities associated with matching same identity faces vs. discriminating different faces: Evidence from individual differences in prosopagnosics and controls

Previous face matching studies provide evidence that matching same identity faces (match trials) and discriminating different face identities (non-match trials) rely on distinct processes. For example, instructional studies geared towards improving face matching in applied settings have often found selective improvements in match or non-match trials only. Additionally, a small study found that developmental prosopagnosics (DPs) have specific deficits in making match but not non-match judgments. In the current study, we sought to replicate this finding in DPs and examine how individual differences across DPs and controls in match vs. non-match performance relate to featural vs. holistic processing abilities. 43 DPs and 27 controls matched face images shown from similar front views or with varied lighting or viewpoint. Participants also performed tasks measuring featural (eyes/mouth) and holistic processing (part-whole task). We found that DPs showed worse overall matching performance than controls and that their relative match vs. non-match deficit depended on image variation condition, indicating that DPs do not consistently show match- or non-match-specific deficits. When examining the association between holistic and featural processing abilities and match vs. non-match trials in the entire group of DPs and controls, we found a very clear dissociation: Match trials significantly correlated with eye processing ability (r=.48) but not holistic processing (r=.11), whereas non-match trials significantly correlated with holistic processing (r=.32) but not eye processing (r=.03). This suggests that matching same identity faces relies more on eye processing while discriminating different faces relies more on holistic processing.

Effects of face masks on the appearance of emotional expressions and invariant characteristics

Faces convey a lot of information about a person. However, the usage of face masks occludes important parts of the face. There is already information that face masks alter the processing of variable characteristics such as emotional expressions and the identity of a person. To investigate whether masks influenced the processing of facial information, we compared ratings of full faces and those covered by face masks. 196 participants completed one of two parallel versions of the experiment. The data demonstrated varying effects of face masks on various characteristics. First, we showed that the perceived intensity of emotional expressions was reduced when the face was covered by face masks. This can be regarded as conceptual replication and extension of the impairing effects of face masks on the recognition of emotional expressions. Next, by analyzing valence and arousal ratings, the data illustrated that emotional expressions were regressed toward neutrality for masked faces relative to no-masked faces. This effect was grossly pronounced for happy facial expressions, less for neutral expressions, and absent for sad expressions. The sex of masked faces was also less accurately identified. Finally, masked faces looked older and less attractive. Post hoc correlational analyses revealed correlation coefficient differences between no-masked and masked faces. The differences occurred in some characteristic pairs (e.g., Age and Attractiveness, Age and Trustworthiness) but not in others. This suggested that the ratings for some characteristics could be influenced by the presence of face masks. Similarly, the ratings of some characteristics could also be influenced by other characteristics, irrespective of face masks. We speculate that the amount of information available on a face could drive our perception of others during social communication. Future directions for research were discussed.

Face Masks Bolsters the Characteristics From Looking at a Face Even When Facial Expressions Are Impaired

Face masks impact social interactions because emotion recognition is difficult due to face occlusion. However, is this enough to conclude that face masks negatively impact social interactions? We investigated the impact of face masks on invariant characteristics (sex, age), trait-like characteristics (trustworthiness, attractiveness, and approachability), and emotional expressions (happiness and excitability). Participants completed an online survey and rated masked and no-masked faces. The same face remained masked or no-masked throughout the survey. Results revealed that, when compared to no-masked faces, masked happy faces appeared less happy. Face masks did not negatively impact the ratings of other characteristics. Participants were better at judging the sex of masked faces. Masked faces also appeared younger, more trustworthy, more attractive, and more approachable. Therefore, face masks did not always result in unfavorable ratings. An additional post hoc modeling revealed that trustworthiness and attractiveness ratings for masked faces predicted the same trait ratings for no-masked faces. However, approachability ratings for no-masked faces predicted the same trait ratings for masked faces. This hinted that information from masked/no-masked faces, such as from the eye and eye region, could aid in the understanding of others during social interaction. Future directions were proposed to expand the research.

Social evaluation under stress: Does acute stress affect social attributions and eye gaze?

Acute stress has been found to elicit pro-social, anti-social or null responses in humans. The causes for these contradicting findings are currently poorly understood, and may rise from subjects' characteristics, such as sex or hormonal status, as well as stimuli-based traits, such as group membership. In the current study, 120 subjects performed either the Trier Social Stress Test or a control (non-stress inducing) condition, followed by ranking displayed faces according to several attributes (e.g., trustworthiness, attractiveness, dominance). Participants' eye gaze was also tracked while viewing facial stimuli. We examined how acute stress interacts with participants' sex, female participants' hormonal status (hormonal contraceptives, early-follicular phase and mid-luteal phase), and the observed faces’ social group (ethnicity-based in-group or out-groups). In general, frequentist and Bayesian analyses showed that acute stress exposure did not affect social attributions or gaze behavior, nor did it interact with subjects' sex or the group membership of the observed faces. Interestingly, sub-group analyses showed that in females, acute stress interacted with hormonal status to yield heterogenous anti-social effects, such as post-stress reductions in perceived trustworthiness in the early-follicular phase, and lower perceived attractiveness in the mid-luteal phase. Given the small sample sizes for the sub-groups, these results should be viewed as preliminary until further replicated. Our results highlight the necessity for large-scale studies, particularly in females, to further refine existing theories regarding the nature and contexts by which acute stress elicits changes in social cognition and behavior.

•

Bayesian analysis showed that stress did not broadly affect social attributions.

•

However, females' hormonal status may interact with stress to modulate attributions.

•

Attributions were highly biased against out-group members, regardless of stress.

•

Stress also did not influence patterns of eye gaze to faces.

Spatial frequency and valence interact in complex emotion perception

Research on spatial frequency contributions to facial emotion identification has largely focused on basic emotions. The present experiment characterised spatial frequency contributions to decoding complex emotions, which can be less visible and intense than basic emotions. We investigated the effects of spatial frequency, expression valence and perceptually available features (full face or eyes only) on decoding accuracy. We observed main effects of all factors, with better performance for high (relative to low) spatial frequency, for positive (relative to negative) emotions and for full face (relative to eyes only) conditions. We also observed an interaction of all factors. The high spatial frequency advantage in decoding accuracy was eliminated only for full faces expressing more positive complex emotions. These findings suggest advantages from high spatial frequency content in accurately decoding complex emotions may attenuate when positive complex emotions are decoded from the spatial frequency content of a broader constellation of features.

Holistic face recognition is an emergent phenomenon of spatial processing in face-selective regions

Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial processing in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical holistic processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. Here we show that in face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage are smaller and shifted downward in response to face inversion. From these measurements, we successfully predict the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial processing in face-selective regions may enable holistic perception. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.

Spatial Integration in Normal Face Processing and Its Breakdown in Congenital Prosopagnosia

Congenital prosopagnosia (CP), a life-long impairment in face processing that occurs in the absence of any apparent brain damage, provides a unique model in which to explore the psychological and neural bases of normal face processing. The goal of this review is to offer a theoretical and conceptual framework that may account for the underlying cognitive and neural deficits in CP. This framework may also provide a novel perspective in which to reconcile some conflicting results that permits the expansion of the research in this field in new directions. The crux of this framework lies in linking the known behavioral and neural underpinnings of face processing and their impairments in CP to a model incorporating grid cell-like activity in the entorhinal cortex. Moreover, it stresses the involvement of active, spatial scanning of the environment with eye movements and implicates their critical role in face encoding and recognition. To begin with, we describe the main behavioral and neural characteristics of CP, and then lay down the building blocks of our proposed model, referring to the existing literature supporting this new framework. We then propose testable predictions and conclude with open questions for future research stemming from this model. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

The effect of face masks and sunglasses on identity and expression recognition with super-recognizers and typical observers

Face masks present a new challenge to face identification (here matching) and emotion recognition in Western cultures. Here, we present the results of three experiments that test the effect of masks, and also the effect of sunglasses (an occlusion that individuals tend to have more experienced with) on (i) familiar face matching, (ii) unfamiliar face matching and (iii) emotion categorization. Occlusion reduced accuracy in all three tasks, with most errors in the mask condition; however, there was little difference in performance for faces in masks compared with faces in sunglasses. Super-recognizers, people who are highly skilled at matching unconcealed faces, were impaired by occlusion, but at the group level, performed with higher accuracy than controls on all tasks. Results inform psychology theory with implications for everyday interactions, security and policing in a mask-wearing society.

Optimum Feature Selection with Particle Swarm Optimization to Face Recognition System Using Gabor Wavelet Transform and Deep Learning

In this study, Gabor wavelet transform on the strength of deep learning which is a new approach for the symmetry face database is presented. A proposed face recognition system was developed to be used for different purposes. We used Gabor wavelet transform for feature extraction of symmetry face training data, and then, we used the deep learning method for recognition. We implemented and evaluated the proposed method on ORL and YALE databases with MATLAB 2020a. Moreover, the same experiments were conducted applying particle swarm optimization (PSO) for the feature selection approach. The implementation of Gabor wavelet feature extraction with a high number of training image samples has proved to be more effective than other methods in our study. The recognition rate when implementing the PSO methods on the ORL database is 85.42% while it is 92% with the three methods on the YALE database. However, the use of the PSO algorithm has increased the accuracy rate to 96.22% for the ORL database and 94.66% for the YALE database.

Face individual identity recognition: a potential endophenotype in autism

BACKGROUND

Face individual identity recognition skill is heritable and independent of intellectual ability. Difficulties in face individual identity recognition are present in autistic individuals and their family members and are possibly linked to oxytocin polymorphisms in families with an autistic child. While it is reported that developmental prosopagnosia (i.e., impaired face identity recognition) occurs in 2-3% of the general population, no prosopagnosia prevalence estimate is available for autism. Furthermore, an autism within-group approach has not been reported towards characterizing impaired face memory and to investigate its possible links to social and communication difficulties.

METHODS

The present study estimated the prevalence of prosopagnosia in 80 autistic adults with no intellectual disability, investigated its cognitive characteristics and links to autism symptoms' severity, personality traits, and mental state understanding from the eye region by using standardized tests and questionnaires.

RESULTS

More than one third of autistic participants showed prosopagnosia. Their face memory skill was not associated with their symptom's severity, empathy, alexithymia, or general intelligence. Face identity recognition was instead linked to mental state recognition from the eye region only in autistic individuals who had prosopagnosia, and this relationship did not depend on participants' basic face perception skills. Importantly, we found that autistic participants were not aware of their face memory skills.

LIMITATIONS

We did not test an epidemiological sample, and additional work is necessary to establish whether these results generalize to the entire autism spectrum.

CONCLUSIONS

Impaired face individual identity recognition meets the criteria to be a potential endophenotype in autism. In the future, testing for face memory could be used to stratify autistic individuals into genetically meaningful subgroups and be translatable to autism animal models.

Birthweight predicts individual differences in adult face recognition ability

It has long been known that premature birth and/or low birthweight can lead to general difficulties in cognitive and emotional functioning throughout childhood. However, the influence of these factors on more specific processes has seldom been addressed, despite their potential to account for wide individual differences in performance that often appear innate. Here, we examined the influence of gestation and birthweight on adults' face perception and face memory skills. Performance on both sub-processes was predicted by birthweight and birthweight-for-gestation, but not gestation alone. Evidence was also found for the domain-specificity of these effects: No perinatal measure correlated with performance on object perception or memory tasks, but they were related to the size of the face inversion effect on the perceptual test. This evidence indicates a novel, very early influence on individual differences in face recognition ability, which persists into adulthood, influences face-processing strategy itself, and may be domain-specific.

Abnormal visual representations associated with confusion of perceived facial expression in schizophrenia with social anxiety disorder

Deficits in social functioning are especially severe amongst schizophrenia individuals with the prevalent comorbidity of social anxiety disorder (SZ&SAD). Yet, the mechanisms underlying the recognition of facial expression of emotions-a hallmark of social cognition-are practically unexplored in SZ&SAD. Here, we aim to reveal the visual representations SZ&SAD (n = 16) and controls (n = 14) rely on for facial expression recognition. We ran a total of 30,000 trials of a facial expression categorization task with Bubbles, a data-driven technique. Results showed that SZ&SAD's ability to categorize facial expression was impared compared to controls. More severe negative symptoms (flat affect, apathy, reduced social drive) was associated with more impaired emotion recognition ability, and with more biases in attributing neutral affect to faces. Higher social anxiety symptoms, on the other hand, was found to enhance the reaction speed to neutral and angry faces. Most importantly, Bubbles showed that these abnormalities could be explained by inefficient visual representations of emotions: compared to controls, SZ&SAD subjects relied less on fine facial cues (high spatial frequencies) and more on coarse facial cues (low spatial frequencies). SZ&SAD participants also never relied on the eye regions (only on the mouth) to categorize facial expressions. We discuss how possible interactions between early (low sensitivity to coarse information) and late stages of the visual system (overreliance on these coarse features) might disrupt SZ&SAD's recognition of facial expressions. Our findings offer perceptual mechanisms through which comorbid SZ&SAD impairs crucial aspects of social cognition, as well as functional psychopathology.

How does gaze to faces support face-to-face interaction? A review and perspective

Gaze-where one looks, how long, and when-plays an essential part in human social behavior. While many aspects of social gaze have been reviewed, there is no comprehensive review or theoretical framework that describes how gaze to faces supports face-to-face interaction. In this review, I address the following questions: (1) When does gaze need to be allocated to a particular region of a face in order to provide the relevant information for successful interaction; (2) How do humans look at other people, and faces in particular, regardless of whether gaze needs to be directed at a particular region to acquire the relevant visual information; (3) How does gaze support the regulation of interaction? The work reviewed spans psychophysical research, observational research, and eye-tracking research in both lab-based and interactive contexts. Based on the literature overview, I sketch a framework for future research based on dynamic systems theory. The framework holds that gaze should be investigated in relation to sub-states of the interaction, encompassing sub-states of the interactors, the content of the interaction as well as the interactive context. The relevant sub-states for understanding gaze in interaction vary over different timescales from microgenesis to ontogenesis and phylogenesis. The framework has important implications for vision science, psychopathology, developmental science, and social robotics.

Implicit race attitudes modulate visual information extraction for trustworthiness judgments

Black people are still considered to be one of the most stigmatized groups and have to face multiple prejudices that undermine their well-being. Assumptions and beliefs about other racial groups are quite pervasive and have been shown to impact basic social tasks such as face processing. For example, individuals with high racial prejudice conceptualize other-race faces as less trustworthy and more criminal. However, it is unknown if implicit racial bias could modulate even low-level perceptual mechanisms such as spatial frequency (SF) extraction when judging the level of trustworthiness of other-race faces. The present study showed that although similar facial features are used to judge the trustworthiness of White and Black faces, own-race faces are processed in lower SF (i.e. coarse information such as the contour of the face and blurred shapes as opposed to high SF representing fine-grained information such as eyelashes or fine wrinkles). This pattern was modulated by implicit race biases: higher implicit biases are associated with a significantly higher reliance on low SF with White than with Black faces.

What We Could Learn About Holistic Face Processing Only From Nonface Objects

Holistic processing is inferred from a number of effects, many of which suggest that people generally find it difficult to process face parts independently. The study of holistic processing using faces has revealed many failures of convergence across different measures, as well as very poor reliability. New tasks designed for individual-differences measurement of holistic processing are more reliable. But other challenges to the study of individual differences in holistic processing require a different approach, in particular the use of nonface objects. Observers’ experiences with faces may be so extensive that it cannot be quantified. In addition, it is difficult to manipulate experience with faces to study causes and mechanisms underlying holistic effects. Recent work has combined an individual-differences approach with a parametric manipulation of experience to reveal that holistic processing arises from domain-specific experience. Other work has revealed that learned attention to parts is sufficient to result in holistic processing, consistent with a mechanism rooted in category-specific learned attention.

Multi-Item Discriminability Pattern to Faces in Developmental Prosopagnosia Reveals Distinct Mechanisms of Face Processing

Previous studies have shown that individuals with developmental prosopagnosia (DP) show specific deficits in face processing. However, the mechanism underlying the deficits remains largely unknown. One hypothesis suggests that DP shares the same mechanism as normal population, though their faces processing is disproportionally impaired. An alternative hypothesis emphasizes a qualitatively different mechanism of DP processing faces. To test these hypotheses, we instructed DP and normal individuals to perceive faces and objects. Instead of calculating accuracy averaging across stimulus items, we used the discrimination accuracy for each item to construct a multi-item discriminability pattern. We found DP's discriminability pattern was less similar to that of normal individuals when perceiving faces than perceiving objects, suggesting that DP has qualitatively different mechanism in representing faces. A functional magnetic resonance imaging study was conducted to reveal the neural basis and found that multi-voxel activation patterns for faces in the right fusiform face area and occipital face area of DP were deviated away from the mean activation pattern of normal individuals. Further, the face representation was more heterogeneous in DP, suggesting that deficits of DP may come from multiple sources. In short, our study provides the first direct evidence that DP processes faces qualitatively different from normal population.

A relationship between Autism-Spectrum Quotient and face viewing behavior in 98 participants

Faces are one of the most important stimuli that we encounter, but humans vary dramatically in their behavior when viewing a face: some individuals preferentially fixate the eyes, others fixate the mouth, and still others show an intermediate pattern. The determinants of these large individual differences are unknown. However, individuals with Autism Spectrum Disorder (ASD) spend less time fixating the eyes of a viewed face than controls, suggesting the hypothesis that autistic traits in healthy adults might explain individual differences in face viewing behavior. Autistic traits were measured in 98 healthy adults recruited from an academic setting using the Autism-Spectrum Quotient, a validated 50-statement questionnaire. Fixations were measured using a video-based eye tracker while participants viewed two different types of audiovisual movies: short videos of talker speaking single syllables and longer videos of talkers speaking sentences in a social context. For both types of movies, there was a positive correlation between Autism-Spectrum Quotient score and percent of time fixating the lower half of the face that explained from 4% to 10% of the variance in individual face viewing behavior. This effect suggests that in healthy adults, autistic traits are one of many factors that contribute to individual differences in face viewing behavior.

Using principal component analysis to characterize eye movement fixation patterns during face viewing

Human faces contain dozens of visual features, but viewers preferentially fixate just two of them: the eyes and the mouth. Face-viewing behavior is usually studied by manually drawing regions of interest (ROIs) on the eyes, mouth, and other facial features. ROI analyses are problematic as they require arbitrary experimenter decisions about the location and number of ROIs, and they discard data because all fixations within each ROI are treated identically and fixations outside of any ROI are ignored. We introduce a data-driven method that uses principal component analysis (PCA) to characterize human face-viewing behavior. All fixations are entered into a PCA, and the resulting eigenimages provide a quantitative measure of variability in face-viewing behavior. In fixation data from 41 participants viewing four face exemplars under three stimulus and task conditions, the first principal component (PC1) separated the eye and mouth regions of the face. PC1 scores varied widely across participants, revealing large individual differences in preference for eye or mouth fixation, and PC1 scores varied by condition, revealing the importance of behavioral task in determining fixation location. Linear mixed effects modeling of the PC1 scores demonstrated that task condition accounted for 41% of the variance, individual differences accounted for 28% of the variance, and stimulus exemplar for less than 1% of the variance. Fixation eigenimages provide a useful tool for investigating the relative importance of the different factors that drive human face-viewing behavior.

The Scanpaths of Subjects with Developmental Prosopagnosia during a Face Memory Task

The scanpaths of healthy subjects show biases towards the upper face, the eyes and the center of the face, which suggests that their fixations are guided by a feature hierarchy towards the regions most informative for face identification. However, subjects with developmental prosopagnosia have a lifelong impairment in face processing. Whether this is reflected in the loss of normal face-scanning strategies is not known. The goal of this study was to determine if subjects with developmental prosopagnosia showed anomalous scanning biases as they processed the identity of faces. We recorded the fixations of 10 subjects with developmental prosopagnosia as they performed a face memorization and recognition task, for comparison with 8 subjects with acquired prosopagnosia (four with anterior temporal lesions and four with occipitotemporal lesions) and 20 control subjects. The scanning of healthy subjects confirmed a bias to fixate the upper over the lower face, the eyes over the mouth, and the central over the peripheral face. Subjects with acquired prosopagnosia from occipitotemporal lesions had more dispersed fixations and a trend to fixate less informative facial regions. Subjects with developmental prosopagnosia did not differ from the controls. At a single-subject level, some developmental subjects performed abnormally, but none consistently across all metrics. Scanning distributions were not related to scores on perceptual or memory tests for faces. We conclude that despite lifelong difficulty with faces, subjects with developmental prosopagnosia still have an internal facial schema that guides their scanning behavior.

Eye movements and retinotopic tuning in developmental prosopagnosia

Despite extensive investigation, the causes and nature of developmental prosopagnosia (DP)-a severe face identification impairment in the absence of acquired brain injury-remain poorly understood. Drawing on previous work showing that individuals identified as being neurotypical (NT) show robust individual differences in where they fixate on faces, and recognize faces best when the faces are presented at this location, we defined and tested four novel hypotheses for how atypical face-looking behavior and/or retinotopic face encoding could impair face recognition in DP: (a) fixating regions of poor information, (b) inconsistent saccadic targeting, (c) weak retinotopic tuning, and (d) fixating locations not matched to the individual's own face tuning. We found no support for the first three hypotheses, with NTs and DPs consistently fixating similar locations and showing similar retinotopic tuning of their face perception performance. However, in testing the fourth hypothesis, we found preliminary evidence for two distinct phenotypes of DP: (a) Subjects characterized by impaired face memory, typical face perception, and a preference to look high on the face, and (b) Subjects characterized by profound impairments to both face memory and perception and a preference to look very low on the face. Further, while all NTs and upper-looking DPs performed best when faces were presented near their preferred fixation location, this was not true for lower-looking DPs. These results suggest that face recognition deficits in a substantial proportion of people with DP may arise not from aberrant face gaze or compromised retinotopic tuning, but from the suboptimal matching of gaze to tuning.