A smile biases the recognition of eye expressions: Configural projection from a salient mouth

The other-ethnicity effect in facial recognition of Tibetan and Han individuals: Evidence from behavioural and eye-movement data

Studies have shown that facial recognition among racial groups exhibits not only an other-race effect but also an other-ethnicity effect within the same racial group. To explore differences in facial recognition and visual scanning patterns due to the other-ethnicity effect, behavioural and eye-movement data were used to investigate the other-ethnicity effect in the facial perception of Tibetan and Han Chinese individuals and whether the visual scanning patterns varied between them. Behavioural data revealed an other-ethnicity effect on facial recognition of Tibetan and Han individuals. Eye-movement data indicated that Tibetan and Han individuals fixated more on the eye and mouth regions when recognising Han faces and on the eye and nose regions when recognising Tibetan faces. The other-ethnicity effect appeared to influence facial recognition in Tibetan and Han individuals, who adopted similar visual scanning patterns when scanning the faces of individuals of their own ethnicity and those of other ethnicities.

Mass-univariate analysis of scalp ERPs reveals large effects of gaze fixation location during face processing that only weakly interact with face emotional expression

Decoding others' facial expressions is critical for social functioning. To clarify the neural correlates of expression perception depending on where we look on the face, three combined gaze-contingent ERP experiments were analyzed using robust mass-univariate statistics. Regardless of task, fixation location impacted face processing from 50 to 350 ms, maximally around 120 ms, reflecting retinotopic mapping around C2 and P1 components. Fixation location also impacted majorly the N170-P2 interval while weak effects were seen at the face-sensitive N170 peak. Results question the widespread assumption that faces are processed holistically into an indecomposable perceptual whole around the N170. Rather, face processing is a complex and view-dependent process that continues well beyond the N170. Expression and fixation location interacted weakly during the P1-N170 interval, supporting a role for the mouth and left eye in fearful and happy expression decoding. Expression effects were weakest at the N170 peak but strongest around P2, especially for fear, reflecting task-independent affective processing. Results suggest N170 reflects a transition between processes rather than the maximum of a holistic face processing stage. Focus on this peak should be replaced by data-driven analyses of the epoch using robust statistics to fully unravel the early visual processing of faces and their affective content.

Age Weakens the Other-Race Effect among Han Subjects in Recognizing Own- and Other-Ethnicity Faces

The development and change in the other-race effect (ORE) in different age groups have always been a focus of researchers. Previous studies have mainly focused on the influence of maturity of life (from infancy to early adulthood) on the ORE, while few researchers have explored the ORE in older people. Therefore, this study used behavioral and eye movement techniques to explore the influence of age on the ORE and the visual scanning pattern of Han subjects recognizing own- and other-ethnicity faces. All participants were asked to complete a study-recognition task for faces, and the behavioral results showed that the ORE of elderly Han subjects was significantly lower than that of young Han subjects. The results of eye movement showed that there were significant differences in the visual scanning pattern of young subjects in recognizing the faces of individuals of their own ethnicity and other ethnicities, which were mainly reflected in the differences in looking at the nose and mouth, while the differences were reduced in the elderly subjects. The elderly subjects used similar scanning patterns to recognize the own- and other-ethnicity faces. This indicates that as age increases, the ORE of older people in recognizing faces of those from different ethnic groups becomes weaker, and elderly subjects have more similar visual scanning patterns in recognizing faces of their own and other ethnicities.

Deconstructing Mechanisms of Abnormal Categorical Perception of Emotional Facial Expressions in Schizophrenia Patients

OBJECTIVE

The current study aims to find out the potential reasons why most schizophrenia patients have a relatively low sensitivity to the classification of emotional facial expressions.

METHODS

By using an emotional categorical perception task, eighty-three schizophrenia patients and seventy-one healthy adults are provided with morphed emotional continuums with two emotional facial expressions (a positive emotional valence: happy; a negative emotional valence: sad).

RESULTS

Through comparing the difference between schizophrenia patients and healthy adults in the processes of estimating facial expressions with ambiguous emotions, we find that the pattern of emotional categorical perception for schizophrenia patients is significantly different from that of healthy controls when they process signals on the local facial areas. Compared to healthy people, schizophrenia patients have a significantly separate classification pattern in processing emotional signals between the eyes and mouth regions. It indicates that compared to healthy adults, schizophrenia patients have larger conflicts in integrating emotional signals from different facial areas. To overcome conflicts, more cognitive resources are required. Unfortunately, the lack of cognitive resources leads to the failure of integration, which further increases the difficulty of estimating facial expressions with ambiguous emotions, and finally leads to the relatively low sensitivity of emotional facial expressions classification.

CONCLUSION

To sum up, the deficit of abnormal perceptions of emotional facial expressions in schizophrenia patients results from an integrated deficit of signals on facial areas.

Effects of face masks on the appearance of emotional expressions and invariant characteristics

Faces convey a lot of information about a person. However, the usage of face masks occludes important parts of the face. There is already information that face masks alter the processing of variable characteristics such as emotional expressions and the identity of a person. To investigate whether masks influenced the processing of facial information, we compared ratings of full faces and those covered by face masks. 196 participants completed one of two parallel versions of the experiment. The data demonstrated varying effects of face masks on various characteristics. First, we showed that the perceived intensity of emotional expressions was reduced when the face was covered by face masks. This can be regarded as conceptual replication and extension of the impairing effects of face masks on the recognition of emotional expressions. Next, by analyzing valence and arousal ratings, the data illustrated that emotional expressions were regressed toward neutrality for masked faces relative to no-masked faces. This effect was grossly pronounced for happy facial expressions, less for neutral expressions, and absent for sad expressions. The sex of masked faces was also less accurately identified. Finally, masked faces looked older and less attractive. Post hoc correlational analyses revealed correlation coefficient differences between no-masked and masked faces. The differences occurred in some characteristic pairs (e.g., Age and Attractiveness, Age and Trustworthiness) but not in others. This suggested that the ratings for some characteristics could be influenced by the presence of face masks. Similarly, the ratings of some characteristics could also be influenced by other characteristics, irrespective of face masks. We speculate that the amount of information available on a face could drive our perception of others during social communication. Future directions for research were discussed.

Perceived emotional expressions of composite faces

The eye and mouth regions serve as the primary sources of facial information regarding an individual's emotional state. The aim of this study was to provide a comprehensive assessment of the relative importance of those two information sources in the identification of different emotions. The stimuli were composite facial images, in which different expressions (Neutral, Anger, Disgust, Fear, Happiness, Contempt, and Surprise) were presented in the eyes and the mouth. Participants (21 women, 11 men, mean age 25 years) rated the expressions of 7 congruent and 42 incongruent composite faces by clicking on a point within the valence-arousal emotion space. Eye movements were also monitored. With most incongruent composite images, the perceived emotion corresponded to the expression of either the eye region or the mouth region or an average of those. The happy expression was different. Happy eyes often shifted the perceived emotion towards a slightly negative point in the valence-arousal space, not towards the location associated with a congruent happy expression. The eye-tracking data revealed significant effects of congruency, expressions and interaction on total dwell time. Our data indicate that whether a face that combines features from two emotional expressions leads to a percept based on only one of the expressions (categorical perception) or integration of the two expressions (dimensional perception), or something altogether different, strongly depends upon the expressions involved.

A False Trail to Follow: Differential Effects of the Facial Feedback Signals From the Upper and Lower Face on the Recognition of Micro-Expressions

Micro-expressions, as fleeting facial expressions, are very important for judging people’s true emotions, thus can provide an essential behavioral clue for lie and dangerous demeanor detection. From embodied accounts of cognition, we derived a novel hypothesis that facial feedback from upper and lower facial regions has differential effects on micro-expression recognition. This hypothesis was tested and supported across three studies. Specifically, the results of Study 1 showed that people became better judges of intense micro-expressions with a duration of 450 ms when the facial feedback from upper face was enhanced via a restricting gel. Additional results of Study 2 showed that the recognition accuracy of subtle micro-expressions was significantly impaired under all duration conditions (50, 150, 333, and 450 ms) when facial feedback from lower face was enhanced. In addition, the results of Study 3 also revealed that blocking the facial feedback of lower face, significantly boosted the recognition accuracy of subtle and intense micro-expressions under all duration conditions (150 and 450 ms). Together, these results highlight the role of facial feedback in judging the subtle movements of micro-expressions.

What makes a smiling face look happy? Visual saliency, distinctiveness, and affect

We investigated the relative contribution of (a) perceptual (eyes and mouth visual saliency), (b) conceptual or categorical (eye expression distinctiveness), and (c) affective (rated valence and arousal) factors, and (d) specific morphological facial features (Action Units; AUs), to the recognition of facial happiness. The face stimuli conveyed truly happy expressions with a smiling mouth and happy eyes, or blended expressions with a smile but non-happy eyes (neutral, sad, fearful, disgusted, surprised, or angry). Saliency, distinctiveness, affect, and AUs served as predictors; the probability of judging a face as happy was the criterion. Both for truly happy and for blended expressions, the probability of perceiving happiness increased mainly as a function of positive valence of the facial configuration. In addition, for blended expressions, the probability of being (wrongly) perceived as happy increased as a function of (a) delayed saliency and (b) reduced distinctiveness of the non-happy eyes, and (c) enhanced AU 6 (cheek raiser) or (d) reduced AUs 4, 5, and 9 (brow lowerer, upper lid raiser, and nose wrinkler, respectively). Importantly, the later the eyes become visually salient relative to the smiling mouth, the more likely it is that faces will look happy.

Hiding true emotions: micro-expressions in eyes retrospectively concealed by mouth movements

When we encounter someone we dislike, we may momentarily display a reflexive disgust expression, only to follow-up with a forced smile and greeting. Our daily lives are replete with a mixture of true and fake expressions. Nevertheless, are these fake expressions really effective at hiding our true emotions? Here we show that brief emotional changes in the eyes (micro-expressions, thought to reflect true emotions) can be successfully concealed by follow-up mouth movements (e.g. a smile). In the same manner as backward masking, mouth movements of a face inhibited conscious detection of all types of micro-expressions in that face, even when viewers paid full attention to the eye region. This masking works only in a backward direction, however, because no disrupting effect was observed when the mouth change preceded the eye change. These results provide scientific evidence for everyday behaviours like smiling to dissemble, and further clarify a major reason for the difficulty we face in discriminating genuine from fake emotional expressions.

Perceptual and affective mechanisms in facial expression recognition: An integrative review

Facial expressions of emotion involve a physical component of morphological changes in a face and an affective component conveying information about the expresser's internal feelings. It remains unresolved how much recognition and discrimination of expressions rely on the perception of morphological patterns or the processing of affective content. This review of research on the role of visual and emotional factors in expression recognition reached three major conclusions. First, behavioral, neurophysiological, and computational measures indicate that basic expressions are reliably recognized and discriminated from one another, albeit the effect may be inflated by the use of prototypical expression stimuli and forced-choice responses. Second, affective content along the dimensions of valence and arousal is extracted early from facial expressions, although this coarse affective representation contributes minimally to categorical recognition of specific expressions. Third, the physical configuration and visual saliency of facial features contribute significantly to expression recognition, with "emotionless" computational models being able to reproduce some of the basic phenomena demonstrated in human observers. We conclude that facial expression recognition, as it has been investigated in conventional laboratory tasks, depends to a greater extent on perceptual than affective information and mechanisms.

A smile radiates outwards and biases the eye expression

This study investigated how extrafoveally seen smiles influence the viewers' perception of non-happy eyes in a face. A smiling mouth appeared in composite faces with incongruent (angry, fearful, neutral, etc.) eyes, thus producing blended expressions, or they appeared in intact faces with genuine expressions. Overt attention to the eye region was spatially cued, foveal vision of the mouth was blocked by gaze-contingent masking, and the distance between the eyes and the mouth was varied. Participants evaluated whether the eyes were happy or not. Results indicated that the same non-happy eyes were more likely to be judged as happy, and more slowly to be judged as not happy, in presence more than in absence of a smile. As (a) the smiling mouth was highly salient regardless of type of eyes, (b) the influence on the eyes increased gradually as a function of eye-mouth proximity, and (c) the effect occurred in the absence of fixations on the mouth, we conclude that a salient smile radiates outwards to other face regions through a projection mechanism, thus making the eye expression look happy.