Developing an Understanding of Emotion Categories: Lessons from Objects

A shared structure for emotion experiences from narratives, videos, and everyday life

Our knowledge of the diversity and psychological organization of emotion experiences is based primarily on studies that used a single type of stimulus with an often limited set of rating scales and analyses. Here we take a comprehensive data-driven approach. We surveyed 1,000+ participants on a diverse set of ratings of emotion experiences to a validated set of ca. 150 text narratives, a validated set of ca. 1,000 videos, and over 10,000 personal experiences sampled longitudinally in everyday life, permitting a unique comparison. All three types of emotion experiences were characterized by similar dimensional spaces that included valence and arousal, as well as dimensions related to generalizability. Emotion experiences were distributed along continuous gradients, with no clear clusters even for the so-called basic emotions. Individual differences in personality traits were associated with differences in everyday emotion experiences but not with emotions evoked by narratives or videos.

More than labels: neural representations of emotion words are widely distributed across the brain

Although emotion words such as "anger," "disgust," "happiness," or "pride" are often thought of as mere labels, increasing evidence points to language as being important for emotion perception and experience. Emotion words may be particularly important for facilitating access to the emotion concepts. Indeed, deficits in semantic processing or impaired access to emotion words interfere with emotion perception. Yet, it is unclear what these behavioral findings mean for affective neuroscience. Thus, we examined the brain areas that support processing of emotion words using representational similarity analysis of functional magnetic resonance imaging data (N = 25). In the task, participants saw 10 emotion words (e.g. "anger," "happiness") while in the scanner. Participants rated each word based on its valence on a continuous scale ranging from 0 (Pleasant/Good) to 1 (Unpleasant/Bad) scale to ensure they were processing the words. Our results revealed that a diverse range of brain areas including prefrontal, midline cortical, and sensorimotor regions contained information about emotion words. Notably, our results overlapped with many regions implicated in decoding emotion experience by prior studies. Our results raise questions about what processes are being supported by these regions during emotion experience.

A social inference model of idealization and devaluation

People often form polarized beliefs, imbuing objects (e.g., themselves or others) with unambiguously positive or negative qualities. In clinical settings, this is referred to as dichotomous thinking or "splitting" and is a feature of several psychiatric disorders. Here, we introduce a Bayesian model of splitting that parameterizes a tendency to rigidly categorize objects as either entirely "Bad" or "Good," rather than to flexibly learn dispositions along a continuous scale. Distinct from the previous descriptive theories, the model makes quantitative predictions about how dichotomous beliefs emerge and are updated in light of new information. Specifically, the model addresses how splitting is context-dependent, yet exhibits stability across time. A key model feature is that phases of devaluation and/or idealization are consolidated by rationally attributing counter-evidence to external factors. For example, when another person is idealized, their less-than-perfect behavior is attributed to unfavorable external circumstances. However, sufficient counter-evidence can trigger switches of polarity, producing bistable dynamics. We show that the model can be fitted to empirical data, to measure individual susceptibility to relational instability. For example, we find that a latent categorical belief that others are "Good" accounts for less changeable, and more certain, character impressions of benevolent as opposed to malevolent others among healthy participants. By comparison, character impressions made by participants with borderline personality disorder reveal significantly higher and more symmetric splitting. The generative framework proposed invites applications for modeling oscillatory relational and affective dynamics in psychotherapeutic contexts. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Alexithymia modulates emotion concept activation during facial expression processing

Alexithymia is characterized by difficulties in emotional information processing. However, the underlying reasons for emotional processing deficits in alexithymia are not fully understood. The present study aimed to investigate the mechanism underlying emotional deficits in alexithymia. Using the Toronto Alexithymia Scale-20, we recruited college students with high alexithymia (n = 24) or low alexithymia (n = 24) in this study. Participants judged the emotional consistency of facial expressions and contextual sentences while recording their event-related potentials. Behaviorally, the high alexithymia group showed longer response times versus the low alexithymia group in processing facial expressions. The event-related potential results showed that the high alexithymia group had more negative-going N400 amplitudes compared with the low alexithymia group in the incongruent condition. More negative N400 amplitudes are also associated with slower responses to facial expressions. Furthermore, machine learning analyses based on N400 amplitudes could distinguish the high alexithymia group from the low alexithymia group in the incongruent condition. Overall, these findings suggest worse facial emotion perception for the high alexithymia group, potentially due to difficulty in spontaneously activating emotion concepts. Our findings have important implications for the affective science and clinical intervention of alexithymia-related affective disorders.

Rethinking Norm Psychology

Norms permeate human life. Most of people's activities can be characterized by rules about what is appropriate, allowed, required, or forbidden-rules that are crucial in making people hyper-cooperative animals. In this article, I examine the current cognitive-evolutionary account of "norm psychology" and propose an alternative that is better supported by evidence and better placed to promote interdisciplinary dialogue. The incumbent theory focuses on rules and claims that humans genetically inherit cognitive and motivational mechanisms specialized for processing these rules. The cultural-evolutionary alternative defines normativity in relation to behavior-compliance, enforcement, and commentary-and suggests that it depends on implicit and explicit processes. The implicit processes are genetically inherited and domain-general; rather than being specialized for normativity, they do many jobs in many species. The explicit processes are culturally inherited and domain-specific; they are constructed from mentalizing and reasoning by social interaction in childhood. The cultural-evolutionary, or "cognitive gadget," perspective suggests that people alive today-parents, educators, elders, politicians, lawyers-have more responsibility for sustaining normativity than the nativist view implies. People's actions not only shape and transmit the rules, but they also create in each new generation mental processes that can grasp the rules and put them into action.

Category Flexibility in Emotion Learning

Learners flexibly update category boundaries to adjust to the range of experiences they encounter. However, little is known about whether the degree of flexibility is consistent across domains. We examined whether categorization of social input, specifically emotions, is afforded more flexibility as compared to other biological input. To address this question, children (6-12 years; 32 female, 37 male; 7 Hispanic or Latino, 62 not Hispanic or Latino; 8 Black or African American, 14 multiracial, 46 White, 1 selected "other") categorized faces morphed from calm to upset and animals morphed from a horse to a cow across task phases that differed in the distribution of stimuli presented. Learners flexibly adjusted both emotion and animal category boundaries according to distributional information, yet children showed more flexibility when updating their category boundaries for emotions. These results provide support for the idea that children-who must adjust to the vast and varied emotional signals of their social partners-respond to social signals dynamically in order to make predictions about the internal states and future behaviors of others.

Basic Emotions or Constructed Emotions: Insights From Taking an Evolutionary Perspective

The ongoing debate between basic emotion theories (BETs) and the theory of constructed emotion (TCE) hampers progress in the field of emotion research. Providing a new perspective, here we aim to bring the theories closer together by dissecting them according to Tinbergen's four questions to clarify a focus on their evolutionary basis. On the basis of our review of the literature, we conclude that whereas BETs focus on the evolution question of Tinbergen, the TCE is more concerned with the causation of emotion. On the survival value of emotions both theories largely agree: to provide the best reaction in specific situations. Evidence is converging on the evolutionary history of emotions but is still limited for both theories-research within both frameworks focuses heavily on the causation. We conclude that BETs and the TCE explain two different phenomena: emotion and feeling. Therefore, they seem irreconcilable but possibly supplementary for explaining and investigating the evolution of emotion-especially considering their similar answer to the question of survival value. Last, this article further highlights the importance of carefully describing what aspect of emotion is being discussed or studied. Only then can evidence be interpreted to converge toward explaining emotion.

Association Strategies of Speech Behavior of Communicators in Coding Discourse: An Interdisciplinary Approach Towards Understanding the Role of Cognitive and Linguistic Processes in Communication

In modern media, the names of fairy-tale and mythological images are used to convey certain emotions and connotations. The aim of the study is to analyze the characteristic associative strategies presented with the mythological images of a dragon, a paper tiger and a chimera in news texts of European and Chinese mass media. In this article, the method of text analysis was used to identify patterns and the most possible interpretations of lexical units. 100 articles from Chinese and European publications were selected for the analysis (About People's Daily Online, China News Service, Guardian and France 24). The required lexemes were most widely used in articles on political topics. The most used was the image of a paper tiger (4001 and 3587 units). This is due to its well-known metaphorical meaning in both cultures, while dragon differs in Chinese and European ones. Further research may focus on the search and analysis of other fairy tale and mythological images in mass media. The present study results may also be applied for further research in the field of linguistics and journalism.

Beyond Linguistic Relativity, Emotion Concepts Illustrate How Meaning is Contextually and Individually Variable

Kemmerer describes grounded accounts of cognition and, using crosslinguistic diversity across conceptual domains, argues that these accounts entail linguistic relativity. In this comment, I extend Kemmerer's position to the domain of emotion. Emotion concepts exemplify characteristics highlighted by grounded accounts of cognition and differ by culture and language. Recent research further demonstrates considerable situation- and person-specific differences. Based on this evidence, I argue that emotion concepts carry unique implications for variation in meaning and experience, entailing a relativity that is contextual and individual in addition to linguistic. I conclude by considering what such pervasive relativity means for interpersonal understanding.

Caregiver speech predicts the emergence of children's emotion vocabulary

Learning about emotions is an important part of children's social and communicative development. How does children's emotion-related vocabulary emerge over development? How may emotion-related information in caregiver input support learning of emotion labels and other emotion-related words? This investigation examined language production and input among English-speaking toddlers (16-30 months) using two datasets: Wordbank (N = 5520; 36% female, 38% male, and 26% unknown gender; 1% Asian, 4% Black, 2% Hispanic, 40% White, 2% others, and 50% unknown ethnicity; collected in North America; dates of data collection unknown) and Child Language Data Exchange System (N = 587; 46% female, 44% male, 9% unknown gender, all unknown ethnicity; collected in North America and the UK; data collection dates, were available between 1962 and 2009). First, we show that toddlers develop the vocabulary to express increasingly wide ranges of emotional information during the first 2 years of life. Computational measures of word valence showed that emotion labels are embedded in a rich network of words with related valence. Second, we show that caregivers leverage these semantic connections in ways that may scaffold children's learning of emotion and mental state labels. This research suggests that young children use the dynamics of language input to construct emotion word meanings, and provides new techniques for defining the quality of infant-directed speech.

Somatovisceral influences on emotional development

Frameworks of emotional development have tended to focus on how environmental factors shape children's emotion understanding. However, individual experiences of emotion represent a complex interplay between both external environmental inputs and internal somatovisceral signaling. Here, we discuss the importance of afferent signals and coordination between central and peripheral mechanisms in affective response processing. We propose that incorporating somatovisceral theories of emotions into frameworks of emotional development can inform how children understand emotions in themselves and others. We highlight promising directions for future research on emotional development incorporating this perspective, namely afferent cardiac processing and interoception, immune activation, physiological synchrony, and social touch.

Prenatal maternal depression predicts neural maturation and negative emotion in infants

Despite widespread acceptance that prenatal symptoms of depression in mothers are detrimental to infants' long-term emotional and cognitive development, little is known about the mechanisms that may integrate outcomes across these domains. Rooted in the integrative perspective that emotional development is grounded in developing cognitive processes, we hypothesized that prenatal symptoms of depression in mothers would be associated with delays in neural maturation that support sociocognitive function in infants, leading to more problematic behaviors. We used a prospective longitudinal study of mothers (N = 92) and their infants to test whether self-reported symptoms of depression in mothers during the second and third trimesters were associated with neural development and infant outcomes at 4 months of age. While controlling for postpartum symptoms of depression, more prenatal symptoms of depression in mothers predicted less neural maturation in the parietal region of 4-month-old infants. Less neural maturation, in turn, was associated with greater infant negativity, suggesting neural maturation as a putative mechanism linking maternal symptoms of depression with infant outcomes. Differences in neural regions and developmental timing are also discussed.

Early antecedents of emotion differentiation and regulation: Experience tunes the appraisal thresholds of emotional development in infancy

In this review, we synthesize evidence to highlight cognitive appraisal as an important developmental antecedent of individual differences in emotion differentiation and adept emotion regulation. Emotion differentiation is the degree to which emotions are experienced in a nuanced or "granular" way-as specific and separable phenomena. More extensive differentiation is related to positive wellbeing and has emerged as a correlate of emotion regulation skill among adults. We argue that the cognitive appraisal processes that underlie these facets of emotional development are instantiated early in the first year of life and tuned by environmental input and experience. Powerful socializing input in the form of caregivers' contingent and selective responding to infants' emotional signals carves and calibrates the infant's appraisal thresholds for what in their world ought to be noticed, deemed as important or personally meaningful, and responded to (whether and how). These appraisal thresholds are thus unique to the individual child despite the ubiquity of the appraisal process in emotional responding. This appraisal infrastructure, while plastic and continually informed by experience across the lifespan, likely tunes subsequent emotion differentiation, with implications for children's emotion regulatory choices and skills. We end with recommendations for future research in this area, including the urgent need for developmental emotion science to investigate the diverse sociocultural contexts in which children's cognitive appraisals, differentiation of emotions, and regulatory responses are being built across childhood.

The age bias in labeling facial expressions in children: Effects of intensity and expression

Emotion reasoning, including labeling of facial expressions, is an important building block for a child's social development. This study investigated age biases in labeling facial expressions in children and adults, focusing on the influence of intensity and expression on age bias. Children (5 to 14 years old; N = 152) and adults (19 to 25 years old; N = 30) labeled happiness, disgust or sadness at five intensity levels (0%; 25%; 50%; 75%; and 100%) in facial images of children and adults. Sensitivity was computed for each of the expression-intensity combinations, separately for the child and adult faces. Results show that children and adults have an age bias at low levels of intensity (25%). In the case of sadness, children have an age bias for all intensities. Thus, the impact of the age of the face seems largest for expressions which might be most difficult to recognise. Moreover, both adults and children label most expressions best in adult rather than child faces, leading to an other-age bias in children and an own-age bias in adults. Overall, these findings reveal that both children and adults exhibit an age bias in labeling subtle facial expressions of emotions.

Context reconsidered: Complex signal ensembles, relational meaning, and population thinking in psychological science

This article considers the status and study of "context" in psychological science through the lens of research on emotional expressions. The article begins by updating three well-trod methodological debates on the role of context in emotional expressions to reconsider several fundamental assumptions lurking within the field's dominant methodological tradition: namely, that certain expressive movements have biologically prepared, inherent emotional meanings that issue from singular, universal processes which are independent of but interact with contextual influences. The second part of this article considers the scientific opportunities that await if we set aside this traditional understanding of "context" as a moderator of signals with inherent psychological meaning and instead consider the possibility that psychological events emerge in ecosystems of signal ensembles, such that the psychological meaning of any individual signal is entirely relational. Such a fundamental shift has radical implications not only for the science of emotion but for psychological science more generally. It offers opportunities to improve the validity and trustworthiness of psychological science beyond what can be achieved with improvements to methodological rigor alone. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Furthering the language hypothesis of alexithymia: An integrated review and meta-analysis

Alexithymia, including the inability to identify and express one's own feelings, is a subclinical condition responsible for some of the socioemotional symptoms seen across a range of psychiatric conditions. The language hypothesis of alexithymia posits a language-mediated disruption in the development of discrete emotion concepts from ambiguous affective states, exacerbating the risk of developing alexithymia in language-impaired individuals. To provide a critical evaluation, a systematic review and meta-analysis of 29 empirical studies of language functioning in alexithymia was performed. A modest association was found between alexithymia and multi-domain language deficits (r = -0.14), including structural language, pragmatics, and propensity to use emotional language. A more theoretically-relevant subsample analysis comparing alexithymia levels in language-impaired and typical individuals revealed larger effects, but a limited number of studies adopted this approach. A synthesis of 11 emotional granularity studies also found an association between alexithymia and reduced emotional granularity (r = -0.10). Language impairments seem to increase the risk of alexithymia. Heterogeneous samples and methods suggest the need for studies with improved alexithymia assessments.

Acquiring Complex Communicative Systems: Statistical Learning of Language and Emotion

During the early postnatal years, most infants rapidly learn to understand two naturally evolved communication systems: language and emotion. While these two domains include different types of content knowledge, it is possible that similar learning processes subserve their acquisition. In this review, we compare the learnable statistical regularities in language and emotion input. We then consider how domain-general learning abilities may underly the acquisition of language and emotion, and how this process may be constrained in each domain. This comparative developmental approach can advance our understanding of how humans learn to communicate with others.

A new science of emotion: implications for functional neurological disorder

Functional neurological disorder reflects impairments in brain networks leading to distressing motor, sensory and/or cognitive symptoms that demonstrate positive clinical signs on examination incongruent with other conditions. A central issue in historical and contemporary formulations of functional neurological disorder has been the mechanistic and aetiological role of emotions. However, the debate has mostly omitted fundamental questions about the nature of emotions in the first place. In this perspective article, we first outline a set of relevant working principles of the brain (e.g. allostasis, predictive processing, interoception and affect), followed by a focused review of the theory of constructed emotion to introduce a new understanding of what emotions are. Building on this theoretical framework, we formulate how altered emotion category construction can be an integral component of the pathophysiology of functional neurological disorder and related functional somatic symptoms. In doing so, we address several themes for the functional neurological disorder field including: (i) how energy regulation and the process of emotion category construction relate to symptom generation, including revisiting alexithymia, 'panic attack without panic', dissociation, insecure attachment and the influential role of life experiences; (ii) re-interpret select neurobiological research findings in functional neurological disorder cohorts through the lens of the theory of constructed emotion to illustrate its potential mechanistic relevance; and (iii) discuss therapeutic implications. While we continue to support that functional neurological disorder is mechanistically and aetiologically heterogenous, consideration of how the theory of constructed emotion relates to the generation and maintenance of functional neurological and functional somatic symptoms offers an integrated viewpoint that cuts across neurology, psychiatry, psychology and cognitive-affective neuroscience.

What Do People Think Is an Emotion?

In emotion research, both conceptual analyses and empirical studies commonly rely on emotion reports. But what do people mean when they say that they are angry, afraid, joyful, etc.? Building on extant theories of emotion, this paper presents four new studies (including a preregistered replication) measuring the weight of cognitive evaluations, bodily changes, and action tendencies in people's use of emotion concepts. The results of these studies suggest that the presence or absence of cognitive evaluations has the largest impact on people's emotion attributions, and bodily changes and action tendencies are considered to depend on cognitive evaluations. Implications for theories of emotion (concepts) and the interpretation of emotion reports are discussed.

Sinful pleasures and pious woes? Using fMRI to examine evaluative and hedonic emotion knowledge

Traditionally, lust and pride have been considered pleasurable, yet sinful in the West. Conversely, guilt is often considered aversive, yet valuable. These emotions illustrate how evaluations about specific emotions and beliefs about their hedonic properties may often diverge. Evaluations about specific emotions may shape important aspects of emotional life (e.g. in emotion regulation, emotion experience and acquisition of emotion concepts). Yet these evaluations are often understudied in affective neuroscience. Prior work in emotion regulation, affective experience, evaluation/attitudes and decision-making point to anterior prefrontal areas as candidates for supporting evaluative emotion knowledge. Thus, we examined the brain areas associated with evaluative and hedonic emotion knowledge, with a focus on the anterior prefrontal cortex. Participants (N = 25) made evaluative and hedonic ratings about emotion knowledge during functional magnetic resonance imaging (fMRI). We found that greater activity in the medial prefrontal cortex (mPFC), ventromedial PFC (vmPFC) and precuneus was associated with an evaluative (vs hedonic) focus on emotion knowledge. Our results suggest that the mPFC and vmPFC, in particular, may play a role in evaluating discrete emotions.

Emerged human-like facial expression representation in a deep convolutional neural network

Recent studies found that the deep convolutional neural networks (DCNNs) trained to recognize facial identities spontaneously learned features that support facial expression recognition, and vice versa. Here, we showed that the self-emerged expression-selective units in a VGG-Face trained for facial identification were tuned to distinct basic expressions and, importantly, exhibited hallmarks of human expression recognition (i.e., facial expression confusion and categorical perception). We then investigated whether the emergence of expression-selective units is attributed to either face-specific experience or domain-general processing by conducting the same analysis on a VGG-16 trained for object classification and an untrained VGG-Face without any visual experience, both having the identical architecture with the pretrained VGG-Face. Although similar expression-selective units were found in both DCNNs, they did not exhibit reliable human-like characteristics of facial expression perception. Together, these findings revealed the necessity of domain-specific visual experience of face identity for the development of facial expression perception, highlighting the contribution of nurture to form human-like facial expression perception.

How the Emotional Environment Shapes the Emotional Life of the Child

Emotion understanding facilitates the development of healthy social interactions. To develop emotion knowledge, infants and young children must learn to make inferences about people's dynamically changing facial and vocal expressions in the context of their everyday lives. Given that emotional information varies so widely, the emotional input that children receive might particularly shape their emotion understanding over time. This review explores how variation in children's received emotional input shapes their emotion understanding and their emotional behavior over the course of development. Variation in emotional input from caregivers shapes individual differences in infants' emotion perception and understanding, as well as older children's emotional behavior. Finally, this work can inform policy and focus interventions designed to help infants and young children with social-emotional development.

Computational Models of Readers' Apperceptive Mass

Recent progress in machine-learning-based distributed semantic models (DSMs) offers new ways to simulate the apperceptive mass (AM; Kintsch, 1980) of reader groups or individual readers and to predict their performance in reading-related tasks. The AM integrates the mental lexicon with world knowledge, as for example, acquired via reading books. Following pioneering work by Denhière and Lemaire (2004), here, we computed DSMs based on a representative corpus of German children and youth literature (Jacobs et al., 2020) as null models of the part of the AM that represents distributional semantic input, for readers of different reading ages (grades 1–2, 3–4, and 5–6). After a series of DSM quality tests, we evaluated the performance of these models quantitatively in various tasks to simulate the different reader groups' hypothetical semantic and syntactic skills. In a final study, we compared the models' performance with that of human adult and children readers in two rating tasks. Overall, the results show that with increasing reading age performance in practically all tasks becomes better. The approach taken in these studies reveals the limits of DSMs for simulating human AM and their potential for applications in scientific studies of literature, research in education, or developmental science.

Emotion words link faces to emotional scenarios in early childhood

Recent theories have suggested that emotion words may facilitate the development of emotion concepts. The present study investigates whether emotion words affect children's performance on an emotion category learning task. Across two experiments, 72 three-year-old children (49 female) were asked to identify which emotional face best matched particular emotional scenarios during nine pretest and nine posttest trials. The scenarios in the present studies aligned with emotions typically learned among older age groups (annoyed, disgusted, and nervous). Between pretest and posttest, children participated in training in which a facial configuration (annoyed, disgusted, or nervous) was paired with an associated scenario while they heard the emotion labeled explicitly or heard irrelevant information (Experiment 1) or heard a broad emotion label versus irrelevant information (Experiment 2). Aside from the labels presented, all other information was kept the same across conditions, including the specific faces and scenarios heard during learning trials. In Experiment 1, children's emotion understanding increased more from pretest to posttest in the explicit label versus irrelevant condition, t(34) = 2.26, p = .030, d = .75, but in Experiment 2 the broad emotion labels did not provide an advantage over irrelevant information, t(34) = .72, p = .474, d = .24. These results suggest that emotion labels may be particularly helpful for young children learning about unfamiliar emotions, because specific labels may help children to aggregate disparate emotional information into meaningful categories. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Assessing the Power of Words to Facilitate Emotion Category Learning

Previous research suggests that labels shape the categorization of emotional stimuli such as facial configurations, yet the strongest evidence of labels' influence on category learning comes from work on object categories. In particular, Lupyan et al. (Psychol Sci 18(12):1077-1083, 2007) found that novel categories of aliens were learned faster by participants provided with nonsense labels during feedback. We summarize a series of five studies in which we examined whether this word-enhancement effect on learning would extend to novel categories of emotion. These studies were conceptual replications of the paradigm used by Lupyan et al. (Psychol Sci 18(12):1077-1083, 2007) designed so that participants would associate novel expressive behaviors with situated experiences. We hypothesized that participants would learn to categorize exemplars of novel emotion categories over the duration of the task, and that categorization would be facilitated for participants who were presented with category labels during learning. We simultaneously analyzed data from all five studies in an integrative data analysis, allowing us to test the effects of learning over time and label condition with increased statistical power. Across all five studies, we found that, while participant performance did improve over time, in no case was it facilitated by including emotion labels at feedback. These results join others in suggesting that the effect of labels on emotion categorization may be more context-dependent than previously supposed-varying by the type of category learning task as well as the specific categories being learned and their relationship to previously acquired knowledge-such that there may be multiple pathways for emotion category learning.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-021-00084-4.

AffectiveTDA: Using Topological Data Analysis to Improve Analysis and Explainability in Affective Computing

We present an approach utilizing Topological Data Analysis to study the structure of face poses used in affective computing, i.e., the process of recognizing human emotion. The approach uses a conditional comparison of different emotions, both respective and irrespective of time, with multiple topological distance metrics, dimension reduction techniques, and face subsections (e.g., eyes, nose, mouth, etc.). The results confirm that our topology-based approach captures known patterns, distinctions between emotions, and distinctions between individuals, which is an important step towards more robust and explainable emotion recognition by machines.

Infants exploring objects: A cascades perspective

Infants spend much of their time exploring objects (Herzberg et al., 2021), and object exploration is linked to learning and development in various domains (e.g., social, cognitive, motor). But how does exploration develop in the first place, and how, exactly, does exploration promote learning? One way to approach these process-oriented questions is with a developmental cascades perspective, which holds that new skills emerge from earlier-developing ones and that various interactions with people and objects accumulate over time to influence multiple domains of development (Masten & Cicchetti, 2010). In this chapter, we describe object exploration from a developmental cascades perspective. In Section 2, we describe typical and atypical trajectories of exploration behaviors, noting how these behaviors emerge from earlier-developing cognitive and motor skills. In Section 3, we discuss how object exploration opens the door for new types of learning opportunities. In Section 4, we discuss early experiences that may shape the development of object exploration. Altogether, we aim to convey that new developments in exploration skills are extensions of earlier-developing skills, and that seemingly insignificant exploratory behaviors (e.g., shaking a rattle) may result in numerous and varied consequences for the developing infant.

Rapid category selectivity for animals versus man-made objects: An N2pc study

Visual recognition occurs rapidly at multiple categorization levels, including the superordinate level (e.g., animal), basic level (e.g., cat), or exemplar level (e.g., my cat). Visual search for animals is faster than for man-made objects, even when the images from those categories have comparable gist statistics (i.e., low- or mid-level visual information), which suggests that higher-level, conceptual influences may support this search advantage for animals. However, it remains unclear whether the search advantage can be explained in part by early visual search processes via the N2pc ERP component, which emerges earlier than behavioral responses, across different categorization levels. Participants searched for 1) an exact image (e.g., a specific squirrel image, Exemplar-level Search), 2) any images of an item (e.g., any squirrels, Basic-level Search), or 3) any items in a category (e.g., any animals, Superordinate-level Search). In addition to Target Present trials, Foil trials measured involuntary attentional selection of task-irrelevant images related to the targets (e.g., other squirrel images when searching for a specific squirrel image, or other animals when searching for squirrels). ERP results revealed 1) a larger N2pc amplitude during Foil trials in Exemplar-level Search for animals than man-made objects, and 2) faster onset latencies for animal search than man-made object search across all categorization levels. These results suggest that the search advantage for animals over man-made objects emerges early, and that attentional selection is more biased toward the basic-level (e.g., squirrel) for animals than for man-made objects during visual search.

The representation of emotion knowledge across development

The present study examined how children spontaneously represent facial cues associated with emotion. 106 three‐ to six‐year‐old children (48 male, 58 female; 9.4% Asian, 84.0% White, 6.6% more than one race) and 40 adults (10 male, 30 female; 10% Hispanic, 30% Asian, 2.5% Black, 57.5% White) were recruited from a Midwestern city (2019–2020), and sorted emotion cues in a spatial arrangement method that assesses emotion knowledge without reliance on emotion vocabulary. Using supervised and unsupervised analyses, the study found evidence for continuities and gradual changes in children's emotion knowledge compared to adults. Emotion knowledge develops through an incremental learning process in which children change their representations using combinations of factors—particularly valence—that are weighted differently across development.

Linguistic and developmental influences on superordinate facial configuration categorization in infancy

Humans perceive emotions in terms of categories, such as "happiness," "sadness," and "anger." To learn these complex conceptual emotion categories, humans must first be able to perceive regularities in expressive behaviors (e.g., facial configurations) across individuals. Recent research suggests that infants spontaneously form "basic-level" categories of facial configurations (e.g., happy vs. fear), but not "superordinate" categories of facial configurations (e.g., positive vs. negative). The current studies further explore how infant age and language impact superordinate categorization of facial configurations associated with different negative emotions. Across all experiments, infants were habituated to one person displaying facial configurations associated with anger and disgust. While 10-month-olds formed a category of person identity (Experiment 1), 14-month-olds formed a category that included negative facial configurations displayed by the same person (Experiment 2). However, neither age formed the hypothesized superordinate category of negative valence. When a verbal label ("toma") was added to each of the habituation events (Experiment 3), 10-month-olds formed a category similar to 14-month-olds in Experiment 2. These findings intersect a larger conversation about the nature and development of children's emotion categories and highlight the importance of considering developmental processes, such as language learning and attentional/memory development, in the design and interpretation of infant categorization studies.

Dual coding of knowledge in the human brain

How does the human brain code knowledge about the world? While disciplines such as artificial intelligence represent world knowledge based on human language, neurocognitive models of knowledge have been dominated by sensory embodiment, in which knowledge is derived from sensory/motor experience and supported by high-level sensory/motor and association cortices. The neural correlates of an alternative disembodied symbolic system had previously been difficult to establish. A recent line of studies exploring knowledge about visual properties, such as color, in visually deprived individuals converge to provide positive, compelling evidence for non-sensory, language-derived, knowledge representation in dorsal anterior temporal lobe and extended language network, in addition to the sensory-derived representations, leading to a sketch of a dual-coding knowledge neural framework.

Adversity and Emotional Functioning

Exposure to early adversity has been linked to variations in emotional functioning. To date, however, the precise nature of these variations has been difficult to pinpoint given widespread differences in the ways in which aspects of emotional functioning are defined and measured. Here, more consistent with models of emotional functioning in typically developing populations (e.g., Halberstadt et al., 2001), we propose defining emotional functioning as consisting of distinct domains of emotion expression, perception, knowledge, reactivity, and regulation. We argue that this framework is useful for guiding hypothesis generation about the specific impact of early adversity on children's emotional functioning. We operationalize the construct of emotional functioning, highlight what is currently known about the association between adversity exposure and each domain of emotional functioning, propose potential mechanisms for these associations, and set the stage for future research examining the development of emotional functioning in the context of early adversity.

Emotion Differentiation and Youth Mental Health: Current Understanding and Open Questions

A growing body of research identifies emotion differentiation-the ability to specifically identify one's emotions-as a key skill for well-being. High emotion differentiation is associated with healthier and more effective regulation of one's emotions, and low emotion differentiation has been documented in several forms of psychopathology. However, the lion's share of this research has focused on adult samples, even though approximately 50% of mental disorders onset before age 18. This review curates what we know about the development of emotion differentiation and its implications for youth mental health. I first review published studies investigating how emotion differentiation develops across childhood and adolescence, as well as studies testing relations between emotion differentiation and mental health in youth samples. Emerging evidence suggests that emotion differentiation actually falls across childhood and adolescence, a counterintuitive pattern that merits further investigation. Additionally, several studies find relations between emotion differentiation and youth mental health, but some instability in results emerged. I then identify open questions that limit our current understanding of emotion differentiation, including (i) lack of clarity as to the valid measurement of emotion differentiation, (ii) potential third variables that could explain relations between emotion differentiation and mental-health (e.g., mean negative affect, IQ, personality, and circularity with outcomes), and (iii) lack of clear mechanistic models regarding the development of emotion differentiation and how it facilitates well-being. I conclude with a discussion of future directions that can address open questions and work toward interventions that treat (or even prevent) psychopathology.

Language and Emotion: Introduction to the Special Issue

What is the relationship between language and emotion? The work that fills the pages of this special issue draws from interdisciplinary domains to weigh in on the relationship between language and emotion in semantics, cross-linguistic experience, development, emotion perception, emotion experience and regulation, and neural representation. These important new findings chart an exciting path forward for future basic and translational work in affective science.

The construction of awe in science communication

Awe is a frequently represented and commonly experienced emotion in science communication. According to a popular account of this emotion, awe is an innate and universal human affective experience that occurs when a person evaluates a target as vast, forcing a shift in their worldview. This shift is portrayed in science communication as resulting in an enhanced interest in the scientific material at hand. Based on the latest research in affective science, however, we challenge this narrow version of awe in science communication and instead advocate a broader account of this emotion in line with a constructionist perspective. We argue that there are a variety of awe types in science communication, each with different forms and functions in relation to the mandates within the multiplicity of contexts in this cultural space. We also contend that people's awe experiences result from their previous interactions with this emotion and the unique affordances provided by the science communication situation.

The Development of Emotion Reasoning in Infancy and Early Childhood

Historically, research characterizing the development of emotion recognition has focused on identifying specific skills and the age periods, or milestones, at which these abilities emerge. However, advances in emotion research raise questions about whether this conceptualization accurately reflects how children learn about, understand, and respond to others’ emotions in everyday life. In this review, we propose a developmental framework for the emergence of emotion reasoning—that is, how children develop the ability to make reasonably accurate inferences and predictions about the emotion states of other people. We describe how this framework holds promise for building upon extant research. Our review suggests that use of the term emotion recognition can be misleading and imprecise, with the developmental processes of interest better characterized by the term emotion reasoning. We also highlight how the age at which children succeed on many tasks reflects myriad developmental processes. This new framing of emotional development can open new lines of inquiry about how humans learn to navigate their social worlds.

Categorization in infancy based on novelty and co-occurrence

categories (i.e., groups of objects that do not share perceptual features, such as food) abound in everyday situations. The present looking time study investigated whether infants are able to distinguish between two abstract categories (food and toys), and how this ability may extend beyond perceived information by manipulating object familiarity in several ways. Test trials displayed 1) the exact familiarized objects paired as they were during familiarization, 2) a cross-pairing of these same familiar objects, 3) novel objects in the same category as the familiarized items, or 4) novel objects in a different category. Compared to the most familiar test trial (i.e., Familiar Category, Familiar Objects, Familiar Pairings), infants looked longer to all other test trials. Although there was a linear increase in looking time with increased novelty of the test trials (i.e., Novel Category as the most novel test trial), the looking times did not differ significantly between the Novel Category and Familiar Category, Unfamiliar Objects trials. This study contributes to our understanding of how infants form object categories based on object familiarity, object co-occurrence, and information abstraction.

The N400 indexes acquisition of novel emotion concepts via conceptual combination

The ability to learn new emotion concepts is adaptive and socially valuable as it communicates culturally held understandings about values, goals, and experiences. Yet, little work has examined the underlying mechanisms that allow for new emotion concepts and words to be integrated into the conceptual system. One such mechanism may be conceptual combination, or the ability to form novel concepts by dynamically combining previously acquired conceptual knowledge. In this study, we used event-related potentials (ERPs) to investigate the electrophysiological correlates of novel emotion concept acquisition via conceptual combination. Participants were briefly trained on 30 novel emotion combinations, each consisting of two English emotion words (the components; e.g., "sadness + fatigue") and a pseudoword (the target; e.g., "despip"). Participants then completed a semantic congruency task while ERPs were recorded. On each trial, two components were presented serially, followed by a target; participants judged whether the target was a valid combination of the preceding components. Targets could be correct or incorrect trained pseudowords, or new untrained pseudowords. Furthermore, components could be presented in reversed order (e.g., "fatigue" then "sadness") or as synonyms (e.g., "exhaustion" for "fatigue"). Consistent with our main hypotheses, we found a main effect of target, such that the correct combinations showed reduced N400 amplitudes when compared to both incorrect and untrained pseudowords. Critically, this effect held regardless of how the preceding components were presented, suggesting deeper semantic learning. These results extend prior findings on conceptual combination and novel word learning, and are congruent with predictive processing accounts of brain function.

A data-driven characterisation of natural facial expressions when giving good and bad news

Facial expressions carry key information about an individual's emotional state. Research into the perception of facial emotions typically employs static images of a small number of artificially posed expressions taken under tightly controlled experimental conditions. However, such approaches risk missing potentially important facial signals and within-person variability in expressions. The extent to which patterns of emotional variance in such images resemble more natural ambient facial expressions remains unclear. Here we advance a novel protocol for eliciting natural expressions from dynamic faces, using a dimension of emotional valence as a test case. Subjects were video recorded while delivering either positive or negative news to camera, but were not instructed to deliberately or artificially pose any specific expressions or actions. A PCA-based active appearance model was used to capture the key dimensions of facial variance across frames. Linear discriminant analysis distinguished facial change determined by the emotional valence of the message, and this also generalised across subjects. By sampling along the discriminant dimension, and back-projecting into the image space, we extracted a behaviourally interpretable dimension of emotional valence. This dimension highlighted changes commonly represented in traditional face stimuli such as variation in the internal features of the face, but also key postural changes that would typically be controlled away such as a dipping versus raising of the head posture from negative to positive valences. These results highlight the importance of natural patterns of facial behaviour in emotional expressions, and demonstrate the efficacy of using data-driven approaches to study the representation of these cues by the perceptual system. The protocol and model described here could be readily extended to other emotional and non-emotional dimensions of facial variance.

The Meaning of Emoji to Describe Food Experiences in Pre-Adolescents

Ongoing research has shown that emoji can be used by children to discriminate food products, but it is unclear if they express emotions and how they are linked to emotional words. Little is known about how children interpret emoji in terms of their emotional meaning in the context of food. This study aimed at investigating the emotional meaning of emoji used to describe food experiences in 9–13-year-old pre-adolescents and to measure related age and gender differences. The meaning of 46 emoji used to describe food experience was explored by: mapping emoji according to similarities and differences in their emotional meaning using the projective mapping technique, and linking emoji with emotion words using a check-all-that-apply (CATA) format. The two tasks gave consistent results and showed that emoji were discriminated along the valence (positive vs. negative) and power (dominant vs. submissive) dimension, and to a lower extent along the arousal dimension (high vs. low activation). In general, negative emoji had more distinct meanings than positive emoji in both studies, but differences in nuances of meaning were found also among positive emoji. Girls and older pre-adolescents (12–13 years old (y.o.)) discriminated positive emoji slightly better than boys and younger pre-adolescents (9–11 y.o.). This suggests that girls and older pre-adolescents may be higher in emotional granularity (the ability to experience and discriminate emotions), particularly of positive emotions. The results of the present work can be used for the development of an emoji-based tool to measure emotions elicited by foods in pre-adolescents.

Superordinate categorization of negative facial expressions in infancy: The influence of labels

Accurate perception of emotional (facial) expressions is an essential social skill. It is currently debated whether emotion categorization in infancy emerges in a "broad-to-narrow" pattern and the degree to which language influences this process. We used an habituation paradigm to explore (a) whether 14- and 18-month-old infants perceive different facial expressions (anger, sad, disgust) as belonging to a superordinate category of negative valence and (b) how verbal labels influence emotion category formation. Results indicated that infants did not spontaneously form a superordinate category of negative valence (Experiments 1 and 3). However, when a novel label ("toma") was added to each event during habituation trials (Experiments 2 and 4), infants formed this superordinate valance category when habituated to disgust and sad expressions (but not when habituated to anger and sadness). These labeling effects were obtained with two stimuli sets (Radboud Face Database and NimStim), even when controlling for the presence of teeth in the expressions. The results indicate that infants, at 14 and 18 months of age, show limited superordinate categorization based on the valence of different negative facial expressions. Specifically, infants only form this abstract emotion category when labels were provided, and the labeling effect depends on which emotions are presented during habituation. These findings have important implications for developmental theories of emotion. (PsycINFO Database Record (c) 2020 APA, all rights reserved).