Processing of facial identity and expression: a psychophysical, physiological, and computational perspective

Effect of low luminance on face recognition in adults with central and peripheral vision loss

PURPOSE

To investigate the effect of low luminance on face recognition, specifically facial identity discrimination (FID) and facial expression recognition (FER), in adults with central vision loss (CVL) and peripheral vision loss (PVL) and to explore the association between clinical vision measures and low luminance FID and FER.

METHODS

Participants included 33 adults with CVL, 17 with PVL and 20 controls. FID and FER were assessed under photopic and low luminance conditions. For the FID task, 12 sets of three faces with neutral expressions were presented and participants asked to indicate the odd-face-out. For FER, 12 single faces were presented and participants asked to name the expression (neutral, happy or angry). Photopic and low luminance visual acuity (VA) and contrast sensitivity (CS) were recorded for all participants and for the PVL group, Humphrey Field Analyzer (HFA) 24-2 mean deviation (MD).

RESULTS

FID accuracy in CVL, and to a lesser extent PVL, was reduced under low compared with photopic luminance (mean reduction 20% and 8% respectively; p < 0.001). FER accuracy was reduced only in CVL (mean reduction 25%; p < 0.001). For both CVL and PVL, low luminance and photopic VA and CS were moderately to strongly correlated with low luminance FID (ρ = 0.61-0.77, p < 0.05). For PVL, better eye HFA 24-2 MD was moderately correlated with low luminance FID (ρ = 0.54, p = 0.02). Results were similar for low luminance FER. Together, photopic VA and CS explained 75% of the variance in low luminance FID, and photopic VA explained 61% of the variance in low luminance FER. Low luminance vision measures explained little additional variance.

CONCLUSION

Low luminance significantly reduced face recognition, particularly for adults with CVL. Worse VA and CS were associated with reduced face recognition. Clinically, photopic VA is a good predictor of face recognition under low luminance conditions.

Communicating and reading emotion with masked faces in the Covid era: A short review of the literature

Face masks have proven to be key to slowing down the SARS-Cov2 virus spread in the COVID-19 pandemic context. However, wearing face masks is not devoid of "side-effects", at both the physical and psychosocial levels. In particular, masks hinder emotion reading from facial expressions as they hide a significant part of the face. This disturbs both holistic and featural processing of facial expressions and, therefore, impairs emotion recognition, and influences many aspects of human social behavior. Communication in general is disrupted by face masks, as they modify the wearer's voice and prevent the audience from using lip reading or other non-verbal cues for speech comprehension. Individuals suffering from psychiatric conditions with impairment of communication, are at higher risk of distress because masks increase their difficulties to read emotions from faces. The identification and acknowledgement of these "side-effects" on communication are necessary because they warrant further work on adaptive solutions that will help foster the use of face masks by the greatest number.

The impact of prior and ongoing threat on the false alarm threshold for facial discrimination

BACKGROUND AND OBJECTIVES

Perceptual adaptations that facilitate rapid responses to threats can also lead to false alarms, or the failure to discriminate safe stimuli from signals of threat. We examined the impact of varying degrees of threat on false alarms in the perceptual discrimination of faces along the dimension of emotion (Experiment 1) or identity (Experiment 2).

METHODS

Participants first trained to discriminate between a target and nontarget face. Next, we tested their ability to identify the target in randomized presentations of the target, the nontarget, and nine novel stimuli morphed in 10% increments of similarity from the target to the nontarget. The task was completed under one of three randomized conditions: 1) Ongoing-Threat paired the target with an aversive outcome in both phases; 2) Prior-Threat paired the target with an aversive outcome in the training phase only; and 3) No-Threat paired the target with a neutral outcome in the training phase only.

RESULTS

In Experiment 1 (N = 90), Ongoing-Threat lowered the false alarm threshold for facial discrimination based on anger intensity compared to Prior-Threat and No-Threat. In Experiment 2 (N = 90), Ongoing-Threat and Prior-Threat each lowered the false alarm threshold for identity-based discrimination compared to No-Threat.

LIMITATIONS

The experiment did not measure generalization of threat responses.

CONCLUSION

Associating a facial expression or identity with threat leads to faster but less accurate discrimination of faces with similar features, particularly under conditions of ongoing threat. These experiments provide an avenue for examining the parameters that impact false alarms, which play a key role in anxiety disorders.

The Role of the Face Itself in the Face Effect: Sensitivity, Expressiveness, and Anticipated Feedback in Individual Compliance

Face-to-face interactions are central to many individual choices and decision-making issues, such as customer services, sales, promotions, and negotiations. While the face effect, that is, face-to-face interactions are more effective in inducing compliance than other forms of interactions, has been noted in the literature, its mechanism has rarely been explored. This research helps to fill the theoretical void and provides new insights into the face effect with two lab experiments and one field experiment. Study 1, a field experiment conducted in a beauty salon, and Study 2, a lab experiment, show that the face effect is largely attributable to anticipated facial feedback and that the face effect is stronger when individuals are sensitive to face and when the requester's face is expressive. Study 3, using video-simulated face-to-face interactions, demonstrates that anticipated facial feedback, not necessarily actual feedback, is enough to drive the face effect. In so doing, this research furthers our understanding of factors that affect individual compliance in face-to-face interactions in both the "sending" and "receiving" stages. We discuss the theoretical and empirical implications, limitations, and future avenues of research.

The influence of dynamics and speech on understanding humanoid facial expressions

Human communication relies mostly on nonverbal signals expressed through body language. Facial expressions, in particular, convey emotional information that allows people involved in social interactions to mutually judge the emotional states and to adjust its behavior appropriately. First studies aimed at investigating the recognition of facial expressions were based on static stimuli. However, facial expressions are rarely static, especially in everyday social interactions. Therefore, it has been hypothesized that the dynamics inherent in a facial expression could be fundamental in understanding its meaning. In addition, it has been demonstrated that nonlinguistic and linguistic information can contribute to reinforce the meaning of a facial expression making it easier to be recognized. Nevertheless, few studies have been performed on realistic humanoid robots. This experimental work aimed at demonstrating the human-like expressive capability of a humanoid robot by examining whether the effect of motion and vocal content influenced the perception of its facial expressions. The first part of the experiment aimed at studying the recognition capability of two kinds of stimuli related to the six basic expressions (i.e. anger, disgust, fear, happiness, sadness, and surprise): static stimuli, that is, photographs, and dynamic stimuli, that is, video recordings. The second and third parts were focused on comparing the same six basic expressions performed by a virtual avatar and by a physical robot under three different conditions: (1) muted facial expressions, (2) facial expressions with nonlinguistic vocalizations, and (3) facial expressions with an emotionally neutral verbal sentence. The results show that static stimuli performed by a human being and by the robot were more ambiguous than the corresponding dynamic stimuli on which motion and vocalization were associated. This hypothesis has been also investigated with a 3-dimensional replica of the physical robot demonstrating that even in case of a virtual avatar, dynamic and vocalization improve the emotional conveying capability.

Feminização, estigma e o gênero facializado: a construção moral do gênero feminino por meio de cirurgias de feminização facial para travestis e mulheres transexuais

Resumo Este artigo objetiva analisar os discursos de gênero construídos em torno das chamadas “cirurgias de feminização facial” tanto pelo discurso científico biomédico como por clínicas que oferecem tais procedimentos, os quais têm se tornado bastante populares entre travestis e mulheres transexuais. A partir de levantamento bibliográfico feito no website Google Acadêmico, inicialmente analiso artigos científicos do campo biomédico que buscam construir uma noção de gênero - ou uma identificação de gênero - por meio do conjunto dos traços faciais. Essa literatura biomédica fornece as bases a partir das quais profissionais e clínicas médicas construirão discursos e práticas acerca da “necessidade” e, por conseguinte, de um desejo desse tipo de intervenção para travestis e mulheres transexuais. Na sequência, analiso como tal discurso é encarnado (embodied) na descrição e na técnica dos procedimentos cirúrgicos da dita feminização facial a partir de material etnográfico de tese de doutorado sobre o Miss T Brasil, concurso de beleza voltado para travestis e mulheres transexuais. Os resultados demonstram que tanto o saber biomédico como as práticas em torno de tais cirurgias se pautam em ideais de constituição de uma “naturalidade” nos traços faciais e preservação de uma identidade reconhecida como pessoal. Constrói-se e visibiliza-se, assim, determinada feminilidade para travestis e mulheres transexuais que tanto parece corresponder a desejos pessoais de se alcançar um ideal normativo e socialmente validado de feminino, ao mesmo tempo que se promoveria o encobrimento de um estigma ao se permitir que elas “passem por” mulher em uma relação social ordinária.

Real-life experience with personally familiar faces enhances discrimination based on global information

Despite the agreement that experience with faces leads to more efficient processing, the underlying mechanisms remain largely unknown. Building on empirical evidence from unfamiliar face processing in healthy populations and neuropsychological patients, the present experiment tested the hypothesis that personal familiarity is associated with superior discrimination when identity information is derived based on global, as opposed to local facial information. Diagnosticity and availability of local and global information was manipulated through varied physical similarity and spatial resolution of morph faces created from personally familiar or unfamiliar faces. We found that discrimination of subtle changes between highly similar morph faces was unaffected by familiarity. Contrariwise, relatively more pronounced physical (i.e., identity) differences were more efficiently discriminated for personally familiar faces, indicating more efficient processing of global, as opposed to local facial information through real-life experience.

Visual scanning behavior during processing of emotional faces in older adults with major depression

OBJECTIVES

Although several reported studies have suggested that younger adults with depression display depression-related biases during the processing of emotional faces, there remains a lack of data concerning these biases in older adults. The aim of our study was to assess scanning behavior during the processing of emotional faces in depressed older adults.

METHOD

Older adults with and without depression viewed happy, neutral or sad portraits during an eye movement recording.

RESULTS

Depressed older adults spent less time with fewer fixations on emotional features than healthy older adults, but only for sad and neutral portraits, with no significant difference for happy portraits.

CONCLUSION

These results suggest disengagement from sad and neutral faces in depressed older adults, which is not consistent with standard theoretical proposals on congruence biases in depression. Also, aging and associated emotional regulation change may explain the expression of depression-related biases. Our preliminary results suggest that information processing in depression consists of a more complex phenomenon than merely a general searching for mood-congruent stimuli or general disengagement from all kinds of stimuli. These findings underline that care must be used when evaluating potential variables, such as aging, which interact with depression and selectively influence the choice of relevant stimulus dimensions.

Getting ready for an emotion: specific premotor brain activities for self-administered emotional pictures

Emotional perception has been extensively studied, but only a few studies have investigated the brain activity preceding exposure to emotional stimuli, especially when they are triggered by the subject himself. Here, we sought to investigate the emotional expectancy by means of movement related cortical potentials (MRCPs) in a self-paced task, in which the subjects begin the affective experience by pressing a key. In this experiment, participants had to alternatively press two keys to concomitantly display positive, negative, neutral, and scrambled images extracted from the International Affective Pictures System (IAPS). Each key press corresponded to a specific emotional category, and the experimenter communicated the coupling before each trial so that the subjects always knew the valence of the forthcoming picture. The main results of the present study included a bilateral positive activity in prefrontal areas during expectancy of more arousing pictures (positive and negative) and an early and sustained positivity over occipital areas, especially during negative expectancy. In addition, we observed more pronounced and anteriorly distributed Late Positive Potential (LPPs) components in the emotional conditions. In conclusion, these results show that emotional expectancy can influence brain activity in both motor preparation and stimulus perception, suggesting enhanced pre-processing in the to-be-stimulated areas. We propose that before a predictable emotional stimulus, both appetitive and defensive motivational systems act to facilitate the forthcoming processing of survival-relevant contents by means of an enhancement of attention toward more arousing pictures.

BESST (Bochum Emotional Stimulus Set)--a pilot validation study of a stimulus set containing emotional bodies and faces from frontal and averted views

This article introduces the freely available Bochum Emotional Stimulus Set (BESST), which contains pictures of bodies and faces depicting either a neutral expression or one of the six basic emotions (happiness, sadness, fear, anger, disgust, and surprise), presented from two different perspectives (0° frontal view vs. camera averted by 45° to the left). The set comprises 565 frontal view and 564 averted view pictures of real-life bodies with masked facial expressions and 560 frontal and 560 averted view faces which were synthetically created using the FaceGen 3.5 Modeller. All stimuli were validated in terms of categorization accuracy and the perceived naturalness of the expression. Additionally, each facial stimulus was morphed into three age versions (20/40/60 years). The results show high recognition of the intended facial expressions, even under speeded forced-choice conditions, as corresponds to common experimental settings. The average naturalness ratings for the stimuli range between medium and high.

Learning to recognize face shapes through serial exploration

Human observers are experts at visual face recognition due to specialized visual mechanisms for face processing that evolve with perceptual expertize. Such expertize has long been attributed to the use of configural processing, enabled by fast, parallel information encoding of the visual information in the face. Here we tested whether participants can learn to efficiently recognize faces that are serially encoded-that is, when only partial visual information about the face is available at any given time. For this, ten participants were trained in gaze-restricted face recognition in which face masks were viewed through a small aperture controlled by the participant. Tests comparing trained with untrained performance revealed (1) a marked improvement in terms of speed and accuracy, (2) a gradual development of configural processing strategies, and (3) participants' ability to rapidly learn and accurately recognize novel exemplars. This performance pattern demonstrates that participants were able to learn new strategies to compensate for the serial nature of information encoding. The results are discussed in terms of expertize acquisition and relevance for other sensory modalities relying on serial encoding.

The Chinese Facial Emotion Recognition Database (CFERD): a computer-generated 3-D paradigm to measure the recognition of facial emotional expressions at different intensities

The Chinese Facial Emotion Recognition Database (CFERD), a computer-generated three-dimensional (3D) paradigm, was developed to measure the recognition of facial emotional expressions at different intensities. The stimuli consisted of 3D colour photographic images of six basic facial emotional expressions (happiness, sadness, disgust, fear, anger and surprise) and neutral faces of the Chinese. The purpose of the present study is to describe the development and validation of CFERD with nonclinical healthy participants (N=100; 50 men; age ranging between 18 and 50 years), and to generate normative data set. The results showed that the sensitivity index d' [d'=Z(hit rate)-Z(false alarm rate), where function Z(p), p∈[0,1]], for all emotions was 0.94. The emotion was more readily detected in happiness, and less easily detected in surprise and sadness. In general, this study replicated the previous findings on the recognition accuracy of emotional expression with the Westerner faces. However, our paradigm extends the previous work by including a wider sensitivity range to differentiate subtle perception of emotion intensities. The CFERD will be a useful tool for emotion recognition assessment in affective neurosciences research, especially for the Chinese and cross-cultural studies.

Recognition of Emotion in Moving and Static Composite Faces

This paper investigates whether the greater accuracy of emotion identification for dynamic versus static expressions, as noted in previous research, can be explained through heightened levels of either component or configural processing. Using a paradigm by Young, Hellawell, and Hay (1987 ), we tested recognition performance of aligned and misaligned composite faces with six basic emotions (happiness, fear, disgust, surprise, anger, sadness). Stimuli were created using 3D computer graphics and were shown as static peak expressions (static condition) and 7 s video sequences (dynamic condition). The results revealed that, overall, moving stimuli were better recognized than static faces, although no interaction between motion and other factors was found. For happiness, sadness, and surprise, misaligned composites were better recognized than aligned composites, suggesting that aligned composites fuse to form a single expression, while the two halves of misaligned composites are perceived as two separate emotions. For anger, disgust, and fear, this was not the case. These results indicate that emotions are perceived on the basis of both configural and component-based information, with specific activation patterns for separate emotions, and that motion has a quality of its own and does not increase configural or component-based recognition separately.

Second-order relational manipulations affect both humans and monkeys

Recognition and individuation of conspecifics by their face is essential for primate social cognition. This ability is driven by a mechanism that integrates the appearance of facial features with subtle variations in their configuration (i.e., second-order relational properties) into a holistic representation. So far, there is little evidence of whether our evolutionary ancestors show sensitivity to featural spatial relations and hence holistic processing of faces as shown in humans. Here, we directly compared macaques with humans in their sensitivity to configurally altered faces in upright and inverted orientations using a habituation paradigm and eye tracking technologies. In addition, we tested for differences in processing of conspecific faces (human faces for humans, macaque faces for macaques) and non-conspecific faces, addressing aspects of perceptual expertise. In both species, we found sensitivity to second-order relational properties for conspecific (expert) faces, when presented in upright, not in inverted, orientation. This shows that macaques possess the requirements for holistic processing, and thus show similar face processing to that of humans.

The application of 3D representations in face recognition

Most current psychological theories of face recognition suggest that faces are stored as multiple 2D views. This research aims to explore the application of 3D face representations by means of a new paradigm. Participants were required to match frontal views of faces to silhouettes of the same faces. The formats of the face stimuli were modified in different experiments to make 3D representations accessible (Experiments 1 and 2) or inaccessible (Experiment 3). Multiple 2D view-based algorithms were not applicable due to the singularity of the frontal-view faces. The results disclosed the application and adaptability of 3D face representations. Participants can readily solve the tasks when the face images retain the information essential for the formation of a 3D face representations. However, the performance substantially declined when the 3D information in faces was eliminated (Experiment 3). Performance also varied between different face orientations and different participant groups.

Apparitional experiences of new faces and dissociation of self-identity during mirror gazing

Apparitional experiences during mirror gazing were studied. Under low levels of illumination, individuals gazed at their own reflected faces in a mirror for a duration of 10 min. Face illumination was relatively uniform, via a nonvisible light source. After about a minute of mirror gazing, individuals reported perceiving strange faces--archetypical and often unknown, human or animal, living or departed parents with traits changed, and fantastical monstrous beings--instead of their own faces in the mirror. During these apparitions, the observers reported feeling that a strange person was watching them from within or beyond the mirror, while the observer maintained consciousness of himself looking at the strange "other." Three psychophysical experiments were conducted on 42 naive normal individuals. At an illumination of the face around 0.8 lux, the mean frequency of apparitions was 1.8 min x (-1) and mean duration was 7 sec. per apparition. At higher illumination, the frequency of apparitional experiences decreased while the duration of mirror gazing needed for the phenomenon to occur increased. This effect may be termed "conscious dissociation of self-identity" to distinguish it from pathological unconscious dissociative identity disorder.

Featural, configural, and holistic face-processing strategies evoke different scan patterns

In two experiments we investigated the role of eye movements during face processing. In experiment 1, using modified faces with primarily featural (scrambled faces) or configural (blurred faces) information as cue stimuli, we manipulated the way participants processed subsequently presented intact faces. In a sequential same-different task, participants decided whether the identity of an intact test face matched a preceding scrambled or blurred cue face. Analysis of eye movements for test faces showed more interfeatural saccades when they followed a blurred face, and longer gaze duration within the same feature when they followed scrambled faces. In experiment 2, we used a similar paradigm except that test faces were cued by intact faces, low-level blurred stimuli, or second-order scrambled stimuli (features were cut out but maintained their first-order relations). We found that in the intact condition participants performed fewer interfeatural saccades than in low-level blurred condition and had shorter gaze duration than in second-order scrambled condition. Moreover, participants fixated the centre of the test face to grasp the information from the whole face. Our findings suggest a differentiation between featural, configural, and holistic processing strategies, which can be associated with specific patterns of eye movements.

On the interplay between familiarity and emotional expression in face perception

Traditional models of face perception (e.g. Bruce and Young 1986) stress independent routes for processing identity and emotional expression. We investigated the interplay between familiarity and emotional expression by systematically varying both factors. In contrast to earlier studies which used binary forced-choice decisions, participants had to judge the familiarity of the presented face and the emotional hedonic valence and emotional arousal of its expressed emotion (angry, happy or neutral), using rating scales. The results demonstrated symmetric, strong interactions between familiarity and expressed emotion. Thus, this study supports more recent models of face perception (Haxby et al. 2000) that were mostly based on brain imaging data. These data together with our behavioural results emphasize the interaction of emotional expression and personal identity and support approaches that propose a relative segregation of these processes, rather than completely independent coding (Calder and Young 2005).