Background factors predicting accuracy and improvement in micro expression recognition

Effects of forward mask duration variability on the temporal dynamics of brief facial expression categorization

The Japanese and Caucasian Brief Affect Recognition Task (JACBART) has been proposed as a standardized method for measuring people's ability to accurately categorize briefly presented images of facial expressions. However, the factors that impact performance in this task are not entirely understood. The current study sought to explore the role of the forward mask's duration (i.e., fixed vs. variable) in brief affect categorization across expressions of the six basic emotions (i.e., anger, disgust, fear, happiness, sadness, and surprise) and three presentation times (i.e., 17, 67, and 500 ms). Current findings do not demonstrate evidence that a variable duration forward mask negatively impacts brief affect categorization. However, efficiency and necessity thresholds were observed to vary across the expressions of emotion. Further exploration of the temporal dynamics of facial affect categorization will therefore require a consideration of these differences.

An ingroup disadvantage in recognizing micro-expressions

Micro-expression is a fleeting facial expression of emotion that usually occurs in high-stake situations and reveals the true emotion that a person tries to conceal. Due to its unique nature, recognizing micro-expression has great applications for fields like law enforcement, medical treatment, and national security. However, the psychological mechanism of micro-expression recognition is still poorly understood. In the present research, we sought to expand upon previous research to investigate whether the group membership of the expresser influences the recognition process of micro-expressions. By conducting two behavioral studies, we found that contrary to the widespread ingroup advantage found in macro-expression recognition, there was a robust ingroup disadvantage in micro-expression recognition instead. Specifically, in Study 1A and 1B, we found that participants were more accurate at recognizing the intense and subtle micro-expressions of their racial outgroups than those micro-expressions of their racial ingroups, and neither the training experience nor the duration of micro-expressions moderated this ingroup disadvantage. In Study 2A and 2B, we further found that mere social categorization alone was sufficient to elicit the ingroup disadvantage for the recognition of intense and subtle micro-expressions, and such an effect was also unaffected by the duration of micro-expressions. These results suggest that individuals spontaneously employ the social category information of others to recognize micro-expressions, and the ingroup disadvantage in micro-expression stems partly from motivated differential processing of ingroup micro-expressions.

Transcranial Direct Current Stimulation Over the Right Temporal Parietal Junction Facilitates Spontaneous Micro-Expression Recognition

Micro-expressions are fleeting and subtle emotional expressions. As they are spontaneous and uncontrollable by one's mind, micro-expressions are considered an indicator of genuine emotions. Their accurate recognition and interpretation promote interpersonal interaction and social communication. Therefore, enhancing the ability to recognize micro-expressions has captured much attention. In the current study, we investigated the effects of training on micro-expression recognition with a Chinese version of the Micro-Expression Training Tool (METT). Our goal was to confirm whether the recognition accuracy of spontaneous micro-expressions could be improved through training and brain stimulation. Since the right temporal parietal junction (rTPJ) has been shown to be involved in the explicit process of facial emotion recognition, we hypothesized that the rTPJ would play a role in facilitating the recognition of micro-expressions. The results showed that anodal transcranial direct-current stimulation (tDCS) of the rTPJ indeed improved the recognition of spontaneous micro-expressions, especially for those associated with fear. The improved accuracy of recognizing fear spontaneous micro-expressions was positively correlated with personal distress in the anodal group but not in the sham group. Our study supports that the combined use of tDCS and METT can be a viable way to train and enhance micro-expression recognition.

Oxytocin Impairs the Recognition of Micro-Expressions of Surprise and Disgust

As fleeting facial expressions which reveal the emotion that a person tries to conceal, micro-expressions have great application potentials for fields like security, national defense and medical treatment. However, the physiological basis for the recognition of these facial expressions is poorly understood. In the present research, we utilized a double-blind, placebo-controlled, mixed-model experimental design to investigate the effects of oxytocin on the recognition of micro-expressions in three behavioral studies. Specifically, in Studies 1 and 2, participants were asked to perform a laboratory-based standardized micro-expression recognition task after self-administration of a single dose of intranasal oxytocin (40 IU) or placebo (containing all ingredients except for the neuropeptide). In Study 3, we further examined the effects of oxytocin on the recognition of natural micro-expressions. The results showed that intranasal oxytocin decreased the recognition speed for standardized intense micro-expressions of surprise (Study 1) and decreased the recognition accuracy for standardized subtle micro-expressions of disgust (Study 2). The results of Study 3 further revealed that intranasal oxytocin administration significantly reduced the recognition accuracy for natural micro-expressions of surprise and disgust. The present research is the first to investigate the effects of oxytocin on micro-expression recognition. It suggests that the oxytocin mainly plays an inhibiting role in the recognition of micro-expressions and there are fundamental differences in the neurophysiological basis for the recognition of micro-expressions and macro-expressions.

Macro- and Micro-Expressions Facial Datasets: A Survey

Automatic facial expression recognition is essential for many potential applications. Thus, having a clear overview on existing datasets that have been investigated within the framework of face expression recognition is of paramount importance in designing and evaluating effective solutions, notably for neural networks-based training. In this survey, we provide a review of more than eighty facial expression datasets, while taking into account both macro- and micro-expressions. The proposed study is mostly focused on spontaneous and in-the-wild datasets, given the common trend in the research is that of considering contexts where expressions are shown in a spontaneous way and in a real context. We have also provided instances of potential applications of the investigated datasets, while putting into evidence their pros and cons. The proposed survey can help researchers to have a better understanding of the characteristics of the existing datasets, thus facilitating the choice of the data that best suits the particular context of their application.

Training Emotion Recognition Accuracy: Results for Multimodal Expressions and Facial Micro Expressions

Nonverbal emotion recognition accuracy (ERA) is a central feature of successful communication and interaction, and is of importance for many professions. We developed and evaluated two ERA training programs-one focusing on dynamic multimodal expressions (audio, video, audio-video) and one focusing on facial micro expressions. Sixty-seven subjects were randomized to one of two experimental groups (multimodal, micro expression) or an active control group (emotional working memory task). Participants trained once weekly with a brief computerized training program for three consecutive weeks. Pre-post outcome measures consisted of a multimodal ERA task, a micro expression recognition task, and a task about patients' emotional cues. Post measurement took place approximately a week after the last training session. Non-parametric mixed analyses of variance using the Aligned Rank Transform were used to evaluate the effectiveness of the training programs. Results showed that multimodal training was significantly more effective in improving multimodal ERA compared to micro expression training or the control training; and the micro expression training was significantly more effective in improving micro expression ERA compared to the other two training conditions. Both pre-post effects can be interpreted as large. No group differences were found for the outcome measure about recognizing patients' emotion cues. There were no transfer effects of the training programs, meaning that participants only improved significantly for the specific facet of ERA that they had trained on. Further, low baseline ERA was associated with larger ERA improvements. Results are discussed with regard to methodological and conceptual aspects, and practical implications and future directions are explored.

The Establishment of Pseudorandom Ecological Microexpression Recognition Test (PREMERT) and Its Relevant Resting-State Brain Activity

The EMERT (ecological microexpression recognition test) by Zhang et al. (2017) used between-subjects Latin square block design for backgrounds; therefore, participants could not get comparable scores. The current study used within-subject pseudorandom design for backgrounds to improve EMERT to PREMERT (pseudorandom EMERT) and used eyes-closed and eyes-open resting-state functional magnetic resonance imaging to detect relevant brain activity of PREMERT for the first time. The results showed (1) two new recapitulative indexes of PREMERT were adopted, such as microexpression M and microexpression SD. Using pseudorandom design, the participants could effectively identify almost all the microexpressions, and each microexpression type had significant background effect. The PREMERT had good split-half reliability, parallel-forms reliability, criterion validity, and ecological validity. Therefore, it could stably and efficiently detect the participants' microexpression recognition abilities. Because of its pseudorandom design, all participants did the same test; their scores could be compared with each other. (2) amplitude of low-frequency fluctuations (ALFF; 0.01-0.1 Hz) in both eyes-closed and eyes-open resting states and ALFF difference could predict microexpression M, and the ALFF difference was less predictive. The relevant resting-state brain areas of microexpression M were some frontal lobes, insula, cingulate cortex, hippocampus, amygdala, fusiform gyrus, parietal lobe, caudate nucleus, precuneus, thalamus, putamen, temporal lobe, and cerebellum. (3) ALFFs in both eyes-closed and eyes-open resting states and ALFF difference could predict microexpression SD, and the ALFF difference was more predictive. The relevant resting-state brain areas of microexpression SD were some frontal lobes, central anterior gyrus, supplementary motor area, insula, hippocampus, amygdala, cuneus, occipital lobe, fusiform gyrus, parietal lobe, caudate nucleus, pallidum, putamen, thalamus, temporal lobe, and cerebellum. (4) There were many similar relevant resting-state brain areas, such as brain areas of expression recognition, microexpressions consciousness and attention, and the change from expression backgrounds to microexpression, and some different relevant resting-state brain areas, such as precuneus, insula, and pallidum, between microexpression M and SD. The ALFF difference was more sensitive to PREMERT fluctuations.

Ecological micro-expression recognition characteristics of young adults with subthreshold depression

The micro-expression (ME) processing characteristics of patients with depression has been studied but has not been investigated in people with subthreshold depression. Based on this, by adopting the ecological MEs recognition paradigm, this study aimed to explore ME recognition in people with subthreshold depression. A 4 (background expression: happy, neutral, sad and fearful) × 4 (ME: happy, neutral, sad, and fearful) study was designed; two groups of participants (experimental group with subthreshold depression vs. healthy control group, 32 participants in each group) were asked to complete the ecological ME recognition task, and the corresponding accuracy (ACC) and reaction time (RT) were analyzed. Results: (1) Under different background conditions, recognizing happy MEs had the highest ACC and shortest RT. (2) There was no significant difference in the ACC and RT between experimental and control groups. (3)In different contexts, individuals with subthreshold depression tended to misjudge neutral, sad, and fearful MEs as happy, while neutral MEs were misjudged as sad and fearful. (4) The performance of individuals with subthreshold depression in the ecological ME recognition task were influenced by the type of ME; they showed highest ACC and shortest RT when recognizing happy MEs (vs. the other MEs). Conclusions: (1) The performance of individuals’ ecological ME recognition were influenced by the background expression, and this embodied the need for ecological ME recognition. (2) Individuals with subthreshold depression showed normal ecological ME recognition ability. (3) In terms of misjudgment, individuals with subthreshold depression showed both positive and negative bias, when completing the ecological ME recognition task. (4) Compared with the other MEs, happy MEs showed an advantage recognition effect for individuals with subthreshold depression who completed the ecological ME recognition task.

The Establishment of Weak Ecological Microexpressions Recognition Test (WEMERT): An Extension on EMERT

The JACBART (Japanese and Caucasian Brief Affect Recognition Test) microexpression recognition test only examines facial expressions under the neutral expression background and the ecological validity is not high. The EMERT (Ecological MicroExpressions Recognition Test) microexpression recognition test examined six microexpressions under seven backgrounds but does not detect the intensity of expressions. In the current study, a weak ecological microexpression recognition test was established to examine the recognition features of six weak microexpressions in all seven high intensity basic expressions. The results found: (1) the test had good retest reliability, criterion validity and ecological validity; and (2) the reliability and validity tests revealed a lot of characteristics of weak microexpression recognition. There were training effects in some weak microexpression recognition. Weak microexpression recognition was generally positively related to the microexpression recognition of JACBART but were generally negatively related to approximate common expressions. The backgrounds main effects in all weak microexpressions were significant and pairwise comparisons show there were a wide range of differences between weak microexpressions under different backgrounds. The standard deviations, of the accuracy of weak microexpressions in different backgrounds, were used to define the fluctuations of the weak microexpression recognition and we found that weak microexpression recognition had many fluctuations. (3) Personality openness and its subdimensions (O1, O2, O3, and O5) were generally positively related to some weak microexpression recognition, except O1, which was significantly negatively related to surprise under neutrality. O1 was positively related to the standard deviation of the weak anger microexpression recognition accuracies and O6 was negatively related to the standard deviation of the weak happiness microexpression recognition accuracies in the first measurement.

A False Trail to Follow: Differential Effects of the Facial Feedback Signals From the Upper and Lower Face on the Recognition of Micro-Expressions

Micro-expressions, as fleeting facial expressions, are very important for judging people’s true emotions, thus can provide an essential behavioral clue for lie and dangerous demeanor detection. From embodied accounts of cognition, we derived a novel hypothesis that facial feedback from upper and lower facial regions has differential effects on micro-expression recognition. This hypothesis was tested and supported across three studies. Specifically, the results of Study 1 showed that people became better judges of intense micro-expressions with a duration of 450 ms when the facial feedback from upper face was enhanced via a restricting gel. Additional results of Study 2 showed that the recognition accuracy of subtle micro-expressions was significantly impaired under all duration conditions (50, 150, 333, and 450 ms) when facial feedback from lower face was enhanced. In addition, the results of Study 3 also revealed that blocking the facial feedback of lower face, significantly boosted the recognition accuracy of subtle and intense micro-expressions under all duration conditions (150 and 450 ms). Together, these results highlight the role of facial feedback in judging the subtle movements of micro-expressions.

Comparison of Ecological Micro-Expression Recognition in Patients with Depression and Healthy Individuals

Previous studies have focused on the characteristics of ordinary facial expressions in patients with depression, and have not investigated the processing characteristics of ecological micro-expressions (MEs, i.e., MEs that presented in different background expressions) in these patients. Based on this, adopting the ecological MEs recognition paradigm, this study aimed to comparatively evaluate facial ME recognition in depressed and healthy individuals. The findings of the study are as follows: (1) background expression: the accuracy (ACC) in the neutral background condition tended to be higher than that in the fear background condition, and the reaction time (RT) in the neutral background condition was significantly longer than that in other backgrounds. The type of ME and its interaction with the type of background expression could affect participants’ ecological MEs recognition ACC and speed. Depression type: there was no significant difference between the ecological MEs recognition ACC of patients with depression and healthy individuals, but the patients’ RT was significantly longer than that of healthy individuals; and (2) patients with depression judged happy MEs that were presented against different backgrounds as neutral and judged neutral MEs that were presented against sad backgrounds as sad. The present study suggested the following: (1) ecological MEs recognition was influenced by background expressions. The ACC of happy MEs was the highest, of neutral ME moderate and of sadness and fear the lowest. The response to the happy MEs was significantly shorter than that of identifying other MEs. It is necessary to conduct research on ecological MEs recognition; (2) the speed of patients with depression in identifying ecological MEs was slower than of healthy individuals; indicating that the patients’ cognitive function was impaired; and (3) the patients with depression showed negative bias in the ecological MEs recognition task, reflecting the lack of happy ME recognition ability and the generalized identification of sad MEs in those patients.