Facial expression recognition (FER) survey: a vision, architectural elements, and future directions

With the cutting-edge advancements in computer vision, facial expression recognition (FER) is an active research area due to its broad practical applications. It has been utilized in various fields, including education, advertising and marketing, entertainment and gaming, health, and transportation. The facial expression recognition-based systems are rapidly evolving due to new challenges, and significant research studies have been conducted on both basic and compound facial expressions of emotions; however, measuring emotions is challenging. Fueled by the recent advancements and challenges to the FER systems, in this article, we have discussed the basics of FER and architectural elements, FER applications and use-cases, FER-based global leading companies, interconnection between FER, Internet of Things (IoT) and Cloud computing, summarize open challenges in-depth to FER technologies, and future directions through utilizing Preferred Reporting Items for Systematic reviews and Meta Analyses Method (PRISMA). In the end, the conclusion and future thoughts are discussed. By overcoming the identified challenges and future directions in this research study, researchers will revolutionize the discipline of facial expression recognition in the future.

Relationships among teacher enjoyment, emotional labor, and perceived student engagement: A daily diary approach

The present daily diary study among 587 Canadian primary and secondary school teachers assessed teachers' genuine expression, faking, hiding of happiness and enthusiasm, and their daily associations with perceived student emotional and behavioral engagement. Moreover, we measured teachers' trait enjoyment before and after the diary study to examine whether teacher trait enjoyment predicted the use of emotional labor strategies that, in turn, were related to teachers' perceptions of their students' engagement. In addition, we examined whether perceived student engagement predicted future levels of teacher trait enjoyment. Results from multilevel structural equation modeling showed that, at the between-person level, teachers who had higher levels of trait enjoyment tended to spontaneously show their positive feelings to their students (β = 0.381, p < .001), which was further positively related to student engagement (β = 0.257, p < .001). In turn, teachers' perceptions of heightened student engagement led to even greater enjoyment in the future (β = 0.134, p < .05). In contrast, teacher trait enjoyment was negatively related to faking (β = -0.297, p < .001) and hiding positive emotions (β = -0.130, p < .05), but was further unrelated to student engagement or future enjoyment. At the within-person level, genuine expression of positive emotions was positively related to student engagement (β = 0.219, p < .001), faking was negatively related to student engagement (β = -0.134, p < .001), and hiding was unrelated to student engagement.

The Human Affectome

Over the last decades, theoretical perspectives in the interdisciplinary field of the affective sciences have proliferated rather than converged due to differing assumptions about what human affective phenomena are and how they work. These metaphysical and mechanistic assumptions, shaped by academic context and values, have dictated affective constructs and operationalizations. However, an assumption about the purpose of affective phenomena can guide us to a common set of metaphysical and mechanistic assumptions. In this capstone paper, we home in on a nested teleological principle for human affective phenomena in order to synthesize metaphysical and mechanistic assumptions. Under this framework, human affective phenomena can collectively be considered algorithms that either adjust based on the human comfort zone (affective concerns) or monitor those adaptive processes (affective features). This teleologically-grounded framework offers a principled agenda and launchpad for both organizing existing perspectives and generating new ones. Ultimately, we hope the Human Affectome brings us a step closer to not only an integrated understanding of human affective phenomena, but an integrated field for affective research.

Embedding Research on Emotion Duration in a Network Model

Contrary to early theorizing, emotions often last for longer periods of time. Variability in people's emotion duration contributes to psychopathologies. Therefore, emotion theories need to account for this variability. So far, reviews only list predictors of emotion duration without integrating them in a theoretical framework. Mechanisms explaining why these predictors relate to emotion duration remain unknown. I propose to embed research on emotion duration in a network model of emotions and illustrate the central ideas with simulations using a formal network model. In the network model, the components of an emotion have direct causal effects on each other. According to the model, emotions last longer (a) when the components are more strongly connected or (b) when the components have higher thresholds (i.e., they are more easily activated). High connectivity prolongs emotions because components are constantly reactivated. Higher thresholds prolong emotions because components are more easily reactivated even when connectivity is lower. Indirect evidence from research on emotion coherence and research on the relationship of predictors of emotion duration with components outside of emotional episodes supports the usefulness of the network model. I further argue and show in simulations that a common cause model, in which a latent emotion causes changes in emotion components, cannot account for research on emotion duration. Finally, I describe future directions for research on emotion duration and emotion dynamics from a network perspective.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-023-00203-3.

Older adults' emotion recognition: No auditory-visual benefit for less clear expressions

The ability to recognise emotion from faces or voices appears to decline with advancing age. However, some studies have shown that emotion recognition of auditory-visual (AV) expressions is largely unaffected by age, i.e., older adults get a larger benefit from AV presentation than younger adults resulting in similar AV recognition levels. An issue with these studies is that they used well-recognised emotional expressions that are unlikely to generalise to real-life settings. To examine if an AV emotion recognition benefit generalizes across well and less well recognised stimuli, we conducted an emotion recognition study using expressions that had clear or unclear emotion information for both modalities, or clear visual, but unclear auditory information. Older (n = 30) and younger (n = 30) participants were tested on stimuli of anger, happiness, sadness, surprise, and disgust (expressed in spoken sentences) in auditory-only (AO), visual-only (VO), or AV format. Participants were required to respond by choosing one of 5 emotion options. Younger adults were more accurate in recognising emotions than older adults except for clear VO expressions. Younger adults showed an AV benefit even when unimodal recognition was poor. No such AV benefit was found for older adults; indeed, AV was worse than VO recognition when AO recognition was poor. Analyses of confusion responses indicated that older adults generated more confusion responses that were common between AO and VO conditions, than younger adults. We propose that older adults' poorer AV performance may be due to a combination of weak auditory emotion recognition and response uncertainty that resulted in a higher cognitive load.

Old Wine in New Bags—Suri and Gross's Connectionist Theory of Emotion is Another Type of Network Theory

Suri and Gross's 2022 connectionist emotion theory can be considered as one version of a family of theories known as network theories of emotion. It presents similarities and differences with older versions of network theories. Like previous network theories and several other traditional emotion theories, however, the connectionist theory remains a reactive theory. The class of reactive theories can be meaningfully contrasted with a class of instrumental theories of which the goal-directed theory is a representative example. Although the latter theory does not deny the existence of emotion networks in memory, it does not grant them many causal powers, thereby seriously restricting their explanatory territory. Future research efforts may help disambiguate between both classes of theories.