Development of the Korean Facial Emotion Stimuli: Korea University Facial Expression Collection 2nd Edition

A critical overview of emotion processing assessment in non-affective and affective psychoses

AIMS

Patients with affective and non-affective psychoses show impairments in both the identification and discrimination of facial affect, which can significantly reduce their quality of life. The aim of this commentary is to present the strengths and weaknesses of the available instruments for a more careful evaluation of different stages of emotion processing in clinical and experimental studies on patients with non-affective and affective psychoses.

METHODS

We reviewed the existing literature to identify different tests used to assess the ability to recognise (e.g. Ekman 60-Faces Test, Facial Emotion Identification Test and Penn Emotion Recognition Test) and to discriminate emotions (e.g. Face Emotion Discrimination Test and Emotion Differentiation Task).

RESULTS

The current literature revealed that few studies combine instruments to differentiate between different levels of emotion processing disorders. The lack of comprehensive instruments that integrate emotion recognition and discrimination assessments prevents a full understanding of patients' conditions.

CONCLUSIONS

This commentary underlines the need for a detailed evaluation of emotion processing ability in patients with non-affective and affective psychoses, to characterise the disorder at early phases from the onset of the disease and to design rehabilitation treatments.

Reactive control in suicide ideators and attempters: An examination of the congruency sequence effect in cognitive and emotional Simon tasks

Reactive control is the cognitive ability to adjust thoughts and behaviors when encountering conflict. We investigated how this ability to manage conflict and stress distinguishes suicidal from nonsuicidal individuals. The hypothesis was that suicidal individuals would show poorer reactive control when faced with conflict generated by emotional than neutral stimuli. Hence, individuals with a lifetime history of suicide ideation or attempt and nonsuicidal controls were tested in cognitive and emotional Simon tasks. We examined the congruency sequence effect (CSE) in the Simon tasks as an indication of the efficiency of reactive control in resolving conflict. Whereas controls demonstrated significant CSEs in both tasks, suicide attempters showed a significant CSE in the cognitive task but not in the emotional task. Suicide ideators, on the other hand, displayed marginally significant CSEs in both tasks. Comparing groups with pairwise comparison demonstrated that the difference in CSE was significant only in the emotional task between attempters and controls. Our findings of attempters' inefficiency in adjusting reactive control during the emotional task reflect cognitive inflexibility in coping with conflicting situations during which suicidal individuals become vulnerable to suicide attempts in states of negative emotion.

Construction of face databases for tasks to recognize facial expressions of basic emotions: a systematic review

ABSTRACT. Recognizing the other's emotions is an important skill for the social context that can be modulated by variables such as gender, age, and race. A number of studies seek to elaborate specific face databases to assess the recognition of basic emotions in different contexts. Objectives: This systematic review sought to gather these studies, describing and comparing the methodologies used in their elaboration. Methods: The databases used to select the articles were the following: PubMed, Web of Science, PsycInfo, and Scopus. The following word crossing was used: “Facial expression database OR Stimulus set AND development OR Validation.” Results: A total of 36 articles showed that most of the studies used actors to express the emotions that were elicited from specific situations to generate the most spontaneous emotion possible. The databases were mainly composed of colorful and static stimuli. In addition, most of the studies sought to establish and describe patterns to record the stimuli, such as color of the garments used and background. The psychometric properties of the databases are also described. Conclusions: The data presented in this review point to the methodological heterogeneity among the studies. Nevertheless, we describe their patterns, contributing to the planning of new research studies that seek to create databases for new contexts.

Surprise as an Emotion: A Response to Ortony

We write in response to an article published in this journal by Andrew Ortony titled “Are All ‘Basic Emotions’ Emotions? A Problem for the (Basic) Emotions Construct.” The author claimed that “for all its elevated status as a basic emotion, surprise fails to satisfy the minimal requirements that [he] proposed for something to be an emotion, and if it is not an emotion, it cannot possibly be a basic emotion.” Although we acknowledge the concerns brought forth by Ortony, we respectfully disagree with his conclusion about surprise. To make a case against the assertion that surprise is valence-free, we summarize an extensive body of work showing that surprise is indeed valenced—in a specific manner (i.e., ambiguously valenced)—and that it meets all of Ortony’s criteria for an emotion. In other words, rather than being described as neither positive nor negative, this emotion is either positive or negative. We consider the data with respect to surprise as a basic emotion, and we dispute the definitions of basic emotions as “widely divergent.” Future work is needed to continue defining an emotion, and a basic emotion, but we believe this is a worthy effort toward shaping a still evolving field.

Towards East Asian Facial Expression Recognition in the Real World: A New Database and Deep Recognition Baseline

In recent years, the focus of facial expression recognition (FER) has gradually shifted from laboratory settings to challenging natural scenes. This requires a great deal of real-world facial expression data. However, most existing real-world databases are based on European-American cultures, and only one is for Asian cultures. This is mainly because the data on European-American expressions are more readily accessed and publicly available online. Owing to the diversity of huge data, FER in European-American cultures has recently developed rapidly. In contrast, the development of FER in Asian cultures is limited by the data. To narrow this gap, we construct a challenging real-world East Asian facial expression (EAFE) database, which contains 10,000 images collected from 113 Chinese, Japanese, and Korean movies and five search engines. We apply three neural network baselines including VGG-16, ResNet-50, and Inception-V3 to classify the images in EAFE. Then, we conduct two sets of experiments to find the optimal learning rate schedule and loss function. Finally, by training with the cosine learning rate schedule and island loss, ResNet-50 can achieve the best accuracy of 80.53% on the testing set, proving that the database is challenging. In addition, we used the Microsoft Cognitive Face API to extract facial attributes in EAFE, so that the database can also be used for facial recognition and attribute analysis. The release of the EAFE can encourage more research on Asian FER in natural scenes and can also promote the development of FER in cross-cultural domains.

Effects of diagnostic regions on facial emotion recognition: The moving window technique

With regard to facial emotion recognition, previous studies found that specific facial regions were attended more in order to identify certain emotions. We investigated whether a preferential search for emotion-specific diagnostic regions could contribute toward the accurate recognition of facial emotions. Twenty-three neurotypical adults performed an emotion recognition task using six basic emotions: anger, disgust, fear, happiness, sadness, and surprise. The participants’ exploration patterns for the faces were measured using the Moving Window Technique (MWT). This technique presented a small window on a blurred face, and the participants explored the face stimuli through a mouse-controlled window in order to recognize the emotions on the face. Our results revealed that when the participants explored the diagnostic regions for each emotion more frequently, the correct recognition of the emotions occurred at a faster rate. To the best of our knowledge, this current study is the first to present evidence that an exploration of emotion-specific diagnostic regions can predict the reaction time of accurate emotion recognition among neurotypical adults. Such findings can be further applied in the evaluation and/or training (regarding emotion recognition functions) of both typically and atypically developing children with emotion recognition difficulties.

Facial expressions of Asian people exposed to constructed urban forests: Accuracy validation and variation assessment

An outcome of building sustainable urban forests is that people's well-being is improved when they are exposed to trees. Facial expressions directly represents one's inner emotions, and can be used to assess real-time perception. The emergence and change in the facial expressions of forest visitors are an implicit process. As such, the reserved character of Asians requires an instrument rating to accurately recognize expressions. In this study, a dataset was established with 2,886 randomly photographed faces from visitors at a constructed urban forest park and at a promenade during summertime in Shenyang City, Northeast China. Six experts were invited to choose 160 photos in total with 20 images representing one of eight typical expressions: angry, contempt, disgusted, happy, neutral, sad, scared, and surprised. The FireFACE ver. 3.0 software was used to test hit-ratio validation as an accuracy measurement (ac.) to match machine-recognized photos with those identified by experts. According to the Kruskal-Wallis test on the difference from averaged scores in 20 recently published papers, contempt (ac. = 0.40%, P = 0.0038) and scared (ac. = 25.23%, P = 0.0018) expressions do not pass the validation test. Both happy and sad expression scores were higher in forests than in promenades, but there were no difference in net positive response (happy minus sad) between locations. Men had a higher happy score but lower disgusted score in forests than in promenades. Men also had a higher angry score in forests. We conclude that FireFACE can be used for analyzing facial expressions in Asian people within urban forests. Women are encouraged to visit urban forests rather than promenades to elicit more positive emotions.

Are facial emotion recognition tasks adequate for assessing social cognition in older people? A review of the literature

OBJECTIVE

Facial emotion recognition (FER) is a component of social cognition and important to interpersonal relations. Therefore, tasks have been developed to assess this skill in different population. Regarding older people, even healthy individuals have a poorer performance compared to rate of correct answers commonly used to assess such tasks. Perform a systematic review to analyze studies addressing the performance of healthy older adults on FER tasks compared to the 70% correct response rate commonly used for the creation of stimulus banks.

MATERIAL AND METHODS

Searches were conducted up to May 2019 in the Pubmed, PsycInfo, Web of Science, and Scopus databases using the keywords ("faces" OR "facial") AND ("recognition" OR "expression" OR "emotional") AND ("elderly" OR "older adults").

RESULTS

Twenty-seven articles were included in the present review. In 16 studies (59.2%), older people had correct response rates on FER lower than 70% on at least one of the emotions evaluated. Among the studies that evaluated each emotion specifically, 62.5% found correct response rates lower than 70% for the emotion fear, 50% for surprise, 50% for sadness, 37.5% for anger, 21.4% for disgust, and 5.9% for happiness. Moreover, the studies that evaluated the level of intensity of the emotions demonstrated a lower rate of correct responses when the intensity of the facial expression was low.

CONCLUSION

That studies employ methods and facial stimuli that may not be adequate for measuring this skill in older people. Thus, it is important to create adequate tasks for assessing the skill in this population.

Memory facilitation for emotional faces: Visual working memory trade-offs resulting from attentional preference for emotional facial expressions

Visual working memory (VWM) for faces is facilitated when they display negative facial expressions. The present study manipulated the emotional heterogeneity of the encoding display in a change detection task to examine whether VWM is enhanced by having a separate memory store or by a bias in the allocation of limited attentional resource. When the encoding display was emotionally heterogeneous, regardless of whether happy or fearful facial expressions were presented, memory for emotional faces increased while memory for neutral faces decreased, indicating a memory trade-off. To investigate whether this occurred as a result of preferential allocation of attentional resource towards emotional expressions over neutral ones, faces were shown sequentially in different quadrants of the display. The memory trade-off between happy and neutral faces disappeared but persisted between fearful and neutral faces at trailing serial positions. When blank intervals were inserted between faces to prevent fearful faces from having prolonged processing that consumes attentional resource that should be shared with neutral faces, the memory trade-off disappeared. Findings support the argument that emotional expressions facilitate VWM due to their bias in obtaining attentional resource but the exact mechanisms through which limited resource is allocated between happy and fearful expressions may differ.

The effects of induced and trait anxiety on the sequential modulation of emotional conflict

The current study aimed to investigate whether induced anxiety, as well as trait anxiety, would lead to the failure of the regulation of emotional conflict. To measure the regulation of emotional conflict, the congruency sequence effect (CSE), which is a reduced effect of task-irrelevant distractor after incongruent trials compared to congruent trials, was observed while participants performed an emotional conflict task. In Experiment 1, participants performed the task in a safe context and a threatening context where a couple of electric shocks were given randomly on two consecutive days. In Experiment 2, participants performed the same task in either a safe or threatening context to avoid a potential carryover effect of the threat. The CSE observed in the safe context disappeared in the threatening context as well as in participants with high-trait anxiety level even without the threat. The findings imply that induced anxiety causes a failure of cognitive control that engenders the CSE in emotional congruency tasks. Moreover, such failure driven by participants' trait anxiety level might be a potential predisposing factor leading to anxiety disorders. Overall, these results suggest that induced anxiety, as well as trait anxiety, has an adverse impact on the sequential modulation of emotional conflict.

Development and Validation of the Yonsei Face Database (YFace DB)

The purposes of this study were to develop the Yonsei Face Database (YFace DB), consisting of both static and dynamic face stimuli for six basic emotions (happiness, sadness, anger, surprise, fear, and disgust), and to test its validity. The database includes selected pictures (static stimuli) and film clips (dynamic stimuli) of 74 models (50% female) aged between 19 and 40. Thousand four hundred and eighty selected pictures and film clips were assessed for the accuracy, intensity, and naturalness during the validation procedure by 221 undergraduate students. The overall accuracy of the pictures was 76%. Film clips had a higher accuracy, of 83%; the highest accuracy was observed in happiness and the lowest in fear across all conditions (static with mouth open or closed, or dynamic). The accuracy was higher in film clips across all emotions but happiness and disgust, while the naturalness was higher in the pictures than in film clips except for sadness and anger. The intensity varied the most across conditions and emotions. Significant gender effects were found in perception accuracy for both the gender of models and raters. Male raters perceived surprise more accurately in static stimuli with mouth open and in dynamic stimuli while female raters perceived fear more accurately in all conditions. Moreover, sadness and anger expressed in static stimuli with mouth open and fear expressed in dynamic stimuli were perceived more accurately when models were male. Disgust expressed in static stimuli with mouth open and dynamic stimuli, and fear expressed in static stimuli with mouth closed were perceived more accurately when models were female. The YFace DB is the largest Asian face database by far and the first to include both static and dynamic facial expression stimuli, and the current study can provide researchers with a wealth of information about the validity of each stimulus through the validation procedure.

The effect of threatening facial expressions on inhibition-induced forgetting depends on their task-relevance

Inhibition-induced forgetting refers to impaired memory for the stimuli to which responses were inhibited. The present study aimed to examine if it would be modulated by the processing of threatening facial expressions. Angry and neutral faces were presented in a go/no-go task and subsequent memory for faces was measured in a surprise recognition task. In Experiment 1, task-irrelevant angry and neutral faces appeared randomly, and participants responded to the gender of the faces during the go/no-go task. Results showed that the perception of neutral faces was possibly biased by angry faces. So, in Experiment 2, angry and neutral faces were given in separate blocks while participants still responded to the gender. Inhibition-induced forgetting was not modulated by facial expressions, as it was observed for both angry and neutral faces. Finally, in Experiment 3, where participants were assigned to respond to either angry or neutral faces, so that facial expressions were relevant, inhibition-induced forgetting was negated only in the group in whom responses to angry faces were inhibited. The findings suggest that task-relevance plays a key role in the way the processing of emotional information influences the interaction between cognitive control and memory encoding.

Multisensory Facial Stimulation Implicitly Improves Evaluations of the Goodness of Attractive Others

It has been well demonstrated that shared multisensory experiences between the self and others can influence the social perception of out-group members. Previous research has shown that the illusion of ownership over a dark-skinned rubber hand or full virtual body generated less negative implicit bias against people with dark skin. However, less is known about how perceived attractiveness difference between self and other affects social perception toward those others after shared multisensory experience. The present study assessed whether shared multisensory experience between the self and attractive others would affect the implicit evaluation of goodness of others. Seventy-three women participated in the study. After the visuotactile multisensory stimulation procedure, participants were administered the Single Category Implicit Association Test (SC-IAT), which presents two attributes (good and bad) and one concept (other). Results showed that the more attractive the faces are, the more positive their implicit evaluation becomes after the synchronous tactile stimulation. This result suggests that shared multisensory experience makes people feel more positive toward others who have positive attribute. This finding suggests that self-other blurring in social contexts might be a compelling factor in evaluating other people positively.

Classifying Schizotypy Using an Audiovisual Emotion Perception Test and Scalp Electroencephalography

Schizotypy refers to the personality trait of experiencing "psychotic" symptoms and can be regarded as a predisposition of schizophrenia-spectrum psychopathology (Raine, 1991). Cumulative evidence has revealed that individuals with schizotypy, as well as schizophrenia patients, have emotional processing deficits. In the present study, we investigated multimodal emotion perception in schizotypy and implemented the machine learning technique to find out whether a schizotypy group (ST) is distinguishable from a control group (NC), using electroencephalogram (EEG) signals. Forty-five subjects (30 ST and 15 NC) were divided into two groups based on their scores on a Schizotypal Personality Questionnaire. All participants performed an audiovisual emotion perception test while EEG was recorded. After the preprocessing stage, the discriminatory features were extracted using a mean subsampling technique. For an accurate estimation of covariance matrices, the shrinkage linear discriminant algorithm was used. The classification attained over 98% accuracy and zero rate of false-positive results. This method may have important clinical implications in discriminating those among the general population who have a subtle risk for schizotypy, requiring intervention in advance.