Context reconsidered: Complex signal ensembles, relational meaning, and population thinking in psychological science

This article considers the status and study of "context" in psychological science through the lens of research on emotional expressions. The article begins by updating three well-trod methodological debates on the role of context in emotional expressions to reconsider several fundamental assumptions lurking within the field's dominant methodological tradition: namely, that certain expressive movements have biologically prepared, inherent emotional meanings that issue from singular, universal processes which are independent of but interact with contextual influences. The second part of this article considers the scientific opportunities that await if we set aside this traditional understanding of "context" as a moderator of signals with inherent psychological meaning and instead consider the possibility that psychological events emerge in ecosystems of signal ensembles, such that the psychological meaning of any individual signal is entirely relational. Such a fundamental shift has radical implications not only for the science of emotion but for psychological science more generally. It offers opportunities to improve the validity and trustworthiness of psychological science beyond what can be achieved with improvements to methodological rigor alone. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Emotional expression in human odour

Recent work has demonstrated that human body odour alters with changing emotional state and that emotionally laden odours can affect the physiology and behaviour of people exposed to them. Here we review these discoveries, which we believe add to a growing recognition that the human sense of smell and its potential role in social interactions have been underappreciated. However, we also critically evaluate the current evidence, with a particular focus on methodology and the interpretation of emotional odour studies. We argue that while the evidence convincingly indicates that humans retain a capacity for olfactory communication of emotion, the extent to which this occurs in ordinary social interaction remains an open question. Future studies should place fewer restrictions on participant selection and lifestyle and adopt more realistic experimental designs. We also need to devote more consideration to underlying mechanisms and to recognise the constraints that these may place on effective communication. Finally, we outline some promising approaches to address these issues, and raise some broader theoretical questions that such approaches may help us to answer.

Trustworthiness perception is mandatory: Task instructions do not modulate fast periodic visual stimulation trustworthiness responses

Although it is often assumed that humans spontaneously respond to the trustworthiness of others’ faces, it is still unclear whether responses to facial trust are mandatory or can be modulated by instructions. Considerable scientific interest lies in understanding whether trust processing is mandatory, given the societal consequences of biased trusting behavior. We tested whether neural responses indexing trustworthiness discrimination depended on whether the task involved focusing on facial trustworthiness or not, using a fast periodic visual stimulation electroencephalography oddball paradigm with a neural marker of trustworthiness discrimination at 1 Hz. Participants judged faces on size without any reference to trust, explicitly formed impressions of facial trust, or were given a financial lending context that primed trust, without explicit trust judgement instructions. Significant trustworthiness discrimination responses at 1 Hz were found in all three conditions, demonstrating the robust nature of trustworthiness discrimination at the neural level. Moreover, no effect of task instruction was observed, with Bayesian analyses providing moderate to decisive evidence that task instruction did not affect trustworthiness discrimination. Our finding that visual trustworthiness discrimination is mandatory points to the remarkable spontaneity of trustworthiness processing, providing clues regarding why these often unreliable impressions are ubiquitous.

Audio-Visual Interactions During Emotion Processing in Bicultural Bilinguals

Despite the growing number of bicultural bilinguals in the world, the way in which multisensory emotions are evaluated by bilinguals who identify with two or more cultures remains unknown. In the present study, Chinese-English bicultural bilinguals from Singapore viewed Asian or Caucasian faces and heard Mandarin or English speech, and evaluated the emotion from one of the two simultaneously-presented modalities. Reliance on the visual modality was greater when bicultural bilinguals processed Western audio-visual emotion information. Although no differences between modalities emerged when processing East-Asian audio-visual emotion information, correlations revealed that bicultural bilinguals increased their reliance on the auditory modality with more daily exposure to East-Asian cultures. Greater interference from the irrelevant modality was observed for Asian faces paired with English speech than for Caucasian faces paired with Mandarin speech. We conclude that processing of emotion in bicultural bilinguals is guided by culture-specific norms, and that familiarity influences how the emotions of those who speak a foreign language are perceived and evaluated.

Comunicación no verbal de emociones: variables sociodemográficas y ventaja endogrupal

En el campo de la comunicación no verbal de las emociones aún existe un debate en torno a la universalidad de las expresiones de emoción y el efecto que tiene la cultura en ellas. Actualmente existen dos teorías que tratan de explicar este fenómeno, la teoría neurocultural y la teoría de los dialectos. Ambas se enfocan en explicar la comunicación no verbal de emociones, pero la primera se centra en los aspectos universales, mientras que la segunda lo hace en la cultura. El objetivo del presente estudio fue indagar la ventaja endogrupal al interior de una cultura. Se diseñó un cuasiexperimento en el que se solicitó a 107 participantes que indicaran la emoción expresada en 42 estímulos en tres formatos de presentación distintos.  Los resultados indican la existencia de dicha ventaja en las mujeres y jóvenes. Los presentes resultados ilustran los efectos de la cultura en este fenómeno.

Contextual cues about reciprocity impact ratings of smile sincerity

Research has shown that context influences how sincere a smile appears to observers. That said, most studies on this topic have focused exclusively on situational cues (e.g. smiling while at a party versus smiling during a job interview) and few have examined other elements of context. One important element concerns any knowledge an observer might have about the smiler as an individual (e.g. their habitual behaviours, traits or attitudes). In this manuscript, we present three experiments that explored the influence of such knowledge on ratings of smile sincerity. In Experiments 1 and 2, participants rated the sincerity of Duchenne and non-Duchenne smiles after having been exposed to cues about the smiler's tendency to reciprocate (this person always, never or occasionally returns favours). In Experiment 3 they performed the same task but with cues about the smiler's love of learning (this person always, never or occasionally enjoys learning new tasks). The results show that cues about the smiler's reciprocity tendency influenced participants' ratings of smile sincerity and did so in a stronger manner than cues about the smiler's love of learning. Overall, these results both strengthen and broaden the literature on the role of context on judgements of smile sincerity.

Conceptualising Bhāvana: How do contemplative Hindu traditions inform understanding emotions and well-being?

There are more than 150 (grand and micro) theories of emotion. Even as European phenomenological perspectives do mention self and agency, the mainstream discourse on emotion in psychology is quite limited in presenting a coherent theory of affective process. A key aspect of Euro-American theories of emotion is that, these theories are topographically flat, thus, unable to provide mechanisms of transformation of emotion relevant for well-being. In this paper, a theory-based framework for emotional transformation through understanding Indian concepts in āyurveda, yoga sutras and the nātya is discussed. Second, the paper proposes that it is Śānta (the Indian conceptualisation of peace) alone, that permits a substantive possibility to a radical re-emotion or experiencing and articulating well-being. The concept for a radical re-emotion is called Bhāvanā, indicating the possibility of conscious and radical re-creation and re-imagination of affective relationships with objects, concepts, processes and people in the world, re-orienting from the isolated ‘re-appraisal’, ‘self-regulation and control’ of emotion as discussed in the mainstream paradigm. The paper contends that these culturally relevant models educate and inform global psychology theory and applied practice.

The Psychology of Communication: The Interplay Between Language and Culture Through Time

Human behavior is often guided by the development and use of language as a means of communication and as a way to represent thoughts and knowledge. Notions of linguistic relativity and the Sapir-Whorf hypothesis indicate that language plays a role in structuring the worldview and perceptions of individuals. The current work will explore how those perceptions are not only guided by language but are also moderated by culture and the beliefs, mores, and ideas that are sanctioned and regulated by a given cultural group. Papers, chapters, and books that have demonstrated the ways in which culture moderates behavior by way of the language that is used to express that culture will be of primary focus. Moreover, a developmental view of language learning will frame this approach in learning how language, culture, and thought have been examined and assessed as a way of describing how, together, they moderate human behavior. Is there truly evidence of linguistic relativity? Does language serve as the primary moderator of thought, or do cultural influences play a more pressing role in cognitive thought processes for a given group? Developments in theory, methods, and data that help explore these concepts through their publication in the Journal of Cross-Cultural Psychology as well as other outlets will be presented and discussed as a framework that can be developed to generate future research questions, testing paradigms, and experimental approaches—both basic and applied.

Modeling Subjective Affect Annotations with Multi-Task Learning

In supervised learning, the generalization capabilities of trained models are based on the available annotations. Usually, multiple annotators are asked to annotate the dataset samples and, then, the common practice is to aggregate the different annotations by computing average scores or majority voting, and train and test models on these aggregated annotations. However, this practice is not suitable for all types of problems, especially when the subjective information of each annotator matters for the task modeling. For example, emotions experienced while watching a video or evoked by other sources of content, such as news headlines, are subjective: different individuals might perceive or experience different emotions. The aggregated annotations in emotion modeling may lose the subjective information and actually represent an annotation bias. In this paper, we highlight the weaknesses of models that are trained on aggregated annotations for modeling tasks related to affect. More concretely, we compare two generic Deep Learning architectures: a Single-Task (ST) architecture and a Multi-Task (MT) architecture. While the ST architecture models single emotional perception each time, the MT architecture jointly models every single annotation and the aggregated annotations at once. Our results show that the MT approach can more accurately model every single annotation and the aggregated annotations when compared to methods that are directly trained on the aggregated annotations. Furthermore, the MT approach achieves state-of-the-art results on the COGNIMUSE, IEMOCAP, and SemEval_2007 benchmarks.

An examination of autonomic and facial responses to prototypical facial emotion expressions in psychopathy

Meta-analyses have found that people high in psychopathy categorize (or “recognize”) others’ prototypical facial emotion expressions with reduced accuracy. However, these have been contested with remaining questions regarding the strength, specificity, and mechanisms of this ability in psychopathy. In addition, few studies have tested holistically whether psychopathy is related to reduced facial mimicry or autonomic arousal in response to others’ dynamic facial expressions. Therefore, the current study presented 6 s videos of a target person making prototypical emotion expressions (anger, fear, disgust, sadness, joy, and neutral) to N = 88 incarcerated adult males while recording facial electromyography, skin conductance response (SCR), and heart rate. Participants identified the emotion category and rated the valence and intensity of the target person’s emotion. Psychopathy was assessed via the Psychopathy Checklist-Revised (PCL-R). We predicted that overall PCL-R scores and scores for the interpersonal/affective traits, in particular, would be related to reduced emotion categorization accuracy, valence ratings, intensity ratings, facial mimicry, SCR amplitude, and cardiac deceleration in response to the prototypical facial emotion expressions. In contrast to our hypotheses, PCL-R scores were unrelated to emotion categorization accuracy, valence ratings, and intensity ratings. Stimuli failed to elicit facial mimicry from the full sample, which does not allow drawing conclusions about the relationship between psychopathy and facial mimicry. However, participants displayed general autonomic arousal responses, but not to prototypical emotion expressions per se. PCL-R scores were also unrelated to SCR and cardiac deceleration. These findings failed to identify aberrant behavioral and physiological responses to prototypical facial emotion expressions in relation to psychopathy.

They're watching you: the impact of social evaluation and anxiety on threat-related perceptual decision-making

In day-to-day social interactions, we frequently use cues and contextual knowledge to make perceptual decisions regarding the presence or absence of threat in facial expressions. Such perceptual decisions are often made in socially evaluative contexts. However, the influence of such contexts on perceptual discrimination of threatening and neutral expressions has not been examined empirically. Furthermore, it is unclear how individual differences in anxiety interact with socially evaluative contexts to influence threat-related perceptual decision-making. In the present study, participants completed a 2-alternative forced choice perceptual decision-making task in which they used threatening and neutral cues to discriminate between threatening and neutral faces while being socially evaluated by purported peers or not. Perceptual sensitivity and reaction time were measured. Individual differences in state anxiety were assessed immediately after the task. In the presence of social evaluation, higher state anxiety was associated with worse perceptual sensitivity, i.e., worse discrimination of threatening and neutral faces. These findings suggest that individual differences in anxiety interact with social evaluation to impair the use of threatening cues to discriminate between threatening and neutral expressions. Such impairment in perceptual decision-making may contribute to maladaptive social behavior that often accompanies evaluative social contexts.

High Emotional Similarity Will Enhance the Face Memory and Face-Context Associative Memory

Previous research has explored how emotional valence (positive or negative) affected face-context associative memory, while little is known about how arousing stimuli that share the same valence but differ in emotionality are bound together and retained in memory. In this study, we manipulated the emotional similarity between the target face and the face associated with the context emotion (i.e., congruent, high similarity, and low similarity), and examined the effect of emotional similarity of negative emotion (i.e., disgust, anger, and fear) on face-context associative memory. Our results showed that the greater the emotional similarity between the faces, the better the face memory and face-context associative memory were. These findings suggest that the processing of facial expression and its associated context may benefit from taking into account the emotional similarity between the faces.

Representations of facial expressions since Darwin

Darwin's book on expressions of emotion was one of the first publications to include photographs (Darwin, The expression of the emotions in Man and animals, 1872). The inclusion of expression photographs meant that readers could form their own opinions and could, like Darwin, survey others for their interpretations. As such, the images provided an evidence base and an 'open source'. Since Darwin, increases in the representativeness and realism of emotional expressions have come from the use of composite images, colour, multiple views and dynamic displays. Research on understanding emotional expressions has been aided by the use of computer graphics to interpolate parametrically between different expressions and to extrapolate exaggerations. This review tracks the developments in how emotions are illustrated and studied and considers where to go next.

Activation of the default network during a theory of mind task predicts individual differences in agreeableness and social cognitive ability

Social cognitive processes, such as emotion perception and empathy, allow humans to navigate complex social landscapes and are associated with specific neural systems. In particular, theory of mind (ToM), which refers to our ability to decipher the mental states of others, is related to the dorsal medial prefrontal cortex and temporoparietal junction, which include portions of the default network. Both social cognition and the default network have been linked to the personality trait Agreeableness. We hypothesized that default network activity during a ToM task would positively predict social cognitive abilities and Agreeableness. In a 3T fMRI scanner, participants (N = 1050) completed a ToM task in which they observed triangles displaying random or social (i.e., human-like) movement. Participants also completed self-report measures of Agreeableness and tests of intelligence and social cognitive ability. In each participant, average blood oxygen level dependent responses were calculated for default network regions associated with social cognition, and structural equation modeling was used to test associations of personality and task performance with activation in those brain regions. Default network activation in the dorsal medial subsystem was greater for social versus random animations. Default network activation in response to social animations predicted better performance on social cognition tasks and, to a lesser degree, higher Agreeableness. Neural response to social stimuli in the default network may be associated with effective social processing and could have downstream effects on social interactions. We discuss theoretical and methodological implications of this work for social and personality neuroscience.

Features and Extra-Striate Body Area Representations of Diagnostic Body Parts in Anger and Fear Perception

Social species perceive emotion via extracting diagnostic features of body movements. Although extensive studies have contributed to knowledge on how the entire body is used as context for decoding bodily expression, we know little about whether specific body parts (e.g., arms and legs) transmit enough information for body understanding. In this study, we performed behavioral experiments using the Bubbles paradigm on static body images to directly explore diagnostic body parts for categorizing angry, fearful and neutral expressions. Results showed that subjects recognized emotional bodies through diagnostic features from the torso with arms. We then conducted a follow-up functional magnetic resonance imaging (fMRI) experiment on body part images to examine whether diagnostic parts modulated body-related brain activity and corresponding neural representations. We found greater activations of the extra-striate body area (EBA) in response to both anger and fear than neutral for the torso and arms. Representational similarity analysis showed that neural patterns of the EBA distinguished different bodily expressions. Furthermore, the torso with arms and whole body had higher similarities in EBA representations relative to the legs and whole body, and to the head and whole body. Taken together, these results indicate that diagnostic body parts (i.e., torso with arms) can communicate bodily expression in a detectable manner.

Cultural Experience Influences Multisensory Emotion Perception in Bilinguals

Emotion perception frequently involves the integration of visual and auditory information. During multisensory emotion perception, the attention devoted to each modality can be measured by calculating the difference between trials in which the facial expression and speech input exhibit the same emotion (congruent) and trials in which the facial expression and speech input exhibit different emotions (incongruent) to determine the modality that has the strongest influence. Previous cross-cultural studies have found that individuals from Western cultures are more distracted by information in the visual modality (i.e., visual interference), whereas individuals from Eastern cultures are more distracted by information in the auditory modality (i.e., auditory interference). These results suggest that culture shapes modality interference in multisensory emotion perception. It is unclear, however, how emotion perception is influenced by cultural immersion and exposure due to migration to a new country with distinct social norms. In the present study, we investigated how the amount of daily exposure to a new culture and the length of immersion impact multisensory emotion perception in Chinese-English bilinguals who moved from China to the United States. In an emotion recognition task, participants viewed facial expressions and heard emotional but meaningless speech either from their previous Eastern culture (i.e., Asian face-Mandarin speech) or from their new Western culture (i.e., Caucasian face-English speech) and were asked to identify the emotion from either the face or voice, while ignoring the other modality. Analyses of daily cultural exposure revealed that bilinguals with low daily exposure to the U.S. culture experienced greater interference from the auditory modality, whereas bilinguals with high daily exposure to the U.S. culture experienced greater interference from the visual modality. These results demonstrate that everyday exposure to new cultural norms increases the likelihood of showing a modality interference pattern that is more common in the new culture. Analyses of immersion duration revealed that bilinguals who spent more time in the United States were equally distracted by faces and voices, whereas bilinguals who spent less time in the United States experienced greater visual interference when evaluating emotional information from the West, possibly due to over-compensation when evaluating emotional information from the less familiar culture. These findings suggest that the amount of daily exposure to a new culture and length of cultural immersion influence multisensory emotion perception in bilingual immigrants. While increased daily exposure to the new culture aids with the adaptation to new cultural norms, increased length of cultural immersion leads to similar patterns in modality interference between the old and new cultures. We conclude that cultural experience shapes the way we perceive and evaluate the emotions of others.

Cultural differences in vocal emotion recognition: a behavioural and skin conductance study in Portugal and Guinea-Bissau

Cross-cultural studies of emotion recognition in nonverbal vocalizations not only support the universality hypothesis for its innate features, but also an in-group advantage for culture-dependent features. Nevertheless, in such studies, differences in socio-economic-educational status have not always been accounted for, with idiomatic translation of emotional concepts being a limitation, and the underlying psychophysiological mechanisms still un-researched. We set out to investigate whether native residents from Guinea-Bissau (West African culture) and Portugal (Western European culture)-matched for socio-economic-educational status, sex and language-varied in behavioural and autonomic system response during emotion recognition of nonverbal vocalizations from Portuguese individuals. Overall, Guinea-Bissauans (as out-group) responded significantly less accurately (corrected p < .05), slower, and showed a trend for higher concomitant skin conductance, compared to Portuguese (as in-group)-findings which may indicate a higher cognitive effort stemming from higher difficulty in discerning emotions from another culture. Specifically, accuracy differences were particularly found for pleasure, amusement, and anger, rather than for sadness, relief or fear. Nevertheless, both cultures recognized all emotions above-chance level. The perceived authenticity, measured for the first time in nonverbal cross-cultural research, in the same vocalizations, retrieved no difference between cultures in accuracy, but still a slower response from the out-group. Lastly, we provide-to our knowledge-a first account of how skin conductance response varies between nonverbally vocalized emotions, with significant differences (p < .05). In sum, we provide behavioural and psychophysiological data, demographically and language-matched, that supports cultural and emotion effects on vocal emotion recognition and perceived authenticity, as well as the universality hypothesis.

Face Masks Impair Basic Emotion Recognition

Abstract. With the widespread adoption of masks, there is a need for understanding how facial obstruction affects emotion recognition. We asked 120 participants to identify emotions from faces with and without masks. We also examined if recognition performance was related to autistic traits and personality. Masks impacted recognition of expressions with diagnostic lower face features the most and those with diagnostic upper face features the least. Persons with higher autistic traits were worse at identifying unmasked expressions, while persons with lower extraversion and higher agreeableness were better at recognizing masked expressions. These results show that different features play different roles in emotion recognition and suggest that obscuring features affects social communication differently as a function of autistic traits and personality.

A robust multivariate structure of interindividual covariation between psychosocial characteristics and arousal responses to visual narratives

People experience the same event but do not feel the same way. Such individual differences in emotion response are believed to be far greater than those in any other mental functions. Thus, to understand what makes people individuals, it is important to identify the systematic structures of individual differences in emotion response and elucidate how such structures relate to what aspects of psychological characteristics. Reflecting this importance, many studies have attempted to relate emotions to psychological characteristics such as personality traits, psychosocial states, and pathological symptoms across individuals. However, systematic and global structures that govern the across-individual covariation between the domain of emotion responses and that of psychological characteristics have been rarely explored previously, which limits our understanding of the relationship between individual differences in emotion response and psychological characteristics. To overcome this limitation, we acquired high-dimensional data sets in both emotion-response (8 measures) and psychological-characteristic (68 measures) domains from the same pool of individuals (86 undergraduate or graduate students) and carried out the canonical correlation analysis in conjunction with the principal component analysis on those data sets. For each participant, the emotion-response measures were quantified by regressing affective-rating responses to visual narrative stimuli onto the across-participant average responses to those stimuli, while the psychological-characteristic measures were acquired from 19 different psychometric questionnaires grounded in personality, psychosocial-factor, and clinical-problem taxonomies. We found a single robust mode of population covariation, particularly between the ’accuracy’ and ’sensitivity’ measures of arousal responses in the emotion domain and many ‘psychosocial’ measures in the psychological-characteristics domain. This mode of covariation suggests that individuals characterized with positive social assets tend to show polarized arousal responses to life events.

An exploration of the impact of contextual information on the emotion recognition ability of autistic adults

Studies of non‐autistic individuals and people with an intellectual disability show that contextual information impacts positively on emotion recognition ability, however, this area is not well researched with autistic adults. We investigated this using a static emotion recognition task. Participants completed an emotion recognition task in person or online. In total, 46 autistic participants and 379 non‐autistic participants completed the task. A linear mixed model showed that autistic adults had significantly lower accuracy when identifying emotions across all contexts, compared to control participants, even when contextual information was present. No significant effect of context was found in either group, nor was gender shown to be an influential variable. A supplementary analysis showed that higher scores on the Autism‐Spectrum Quotient led to lower scores on the emotion recognition task; no effect of context was found here either. This research adds to the limited work investigating the influence of contextual factors in emotion recognition in autistic adults. Overall, it shows that context may not aid emotion recognition in this group in the same way as it does for non‐autistic individuals.

Long-Term Influence of Incidental Emotions on the Emotional Judgment of Neutral Faces

Background: Evidence indicates that the processing of facial stimuli may be influenced by incidental factors, and these influences are particularly powerful when facial expressions are ambiguous, such as neutral faces. However, limited research investigated whether emotional contextual information presented in a preceding and unrelated experiment could be pervasively carried over to another experiment to modulate neutral face processing.

Objective: The present study aims to investigate whether an emotional text presented in a first experiment could generate negative emotion toward neutral faces in a second experiment unrelated to the previous experiment.

Methods: Ninety-nine students (all women) were randomly assigned to read and evaluate a negative text (negative context) or a neutral text (neutral text) in the first experiment. In the subsequent second experiment, the participants performed the following two tasks: (1) an attentional task in which neutral faces were presented as distractors and (2) a task involving the emotional judgment of neutral faces.

Results: The results show that compared to the neutral context, in the negative context, the participants rated more faces as negative. No significant result was found in the attentional task.

Conclusion: Our study demonstrates that incidental emotional information available in a previous experiment can increase participants’ propensity to interpret neutral faces as more negative when emotional information is directly evaluated. Therefore, the present study adds important evidence to the literature suggesting that our behavior and actions are modulated by previous information in an incidental or low perceived way similar to what occurs in everyday life, thereby modulating our judgments and emotions.

How does the presence of a surgical face mask impair the perceived intensity of facial emotions?

The use of surgical-type face masks has become increasingly common during the COVID-19 pandemic. Recent findings suggest that it is harder to categorise the facial expressions of masked faces, than of unmasked faces. To date, studies of the effects of mask-wearing on emotion recognition have used categorisation paradigms: authors have presented facial expression stimuli and examined participants’ ability to attach the correct label (e.g., happiness, disgust). While the ability to categorise particular expressions is important, this approach overlooks the fact that expression intensity is also informative during social interaction. For example, when predicting an interactant’s future behaviour, it is useful to know whether they are slightly fearful or terrified, contented or very happy, slightly annoyed or angry. Moreover, because categorisation paradigms force observers to pick a single label to describe their percept, any additional dimensionality within observers’ interpretation is lost. In the present study, we adopted a complementary emotion-intensity rating paradigm to study the effects of mask-wearing on expression interpretation. In an online experiment with 120 participants (82 female), we investigated how the presence of face masks affects the perceived emotional profile of prototypical expressions of happiness, sadness, anger, fear, disgust, and surprise. For each of these facial expressions, we measured the perceived intensity of all six emotions. We found that the perceived intensity of intended emotions (i.e., the emotion that the actor intended to convey) was reduced by the presence of a mask for all expressions except for anger. Additionally, when viewing all expressions except surprise, masks increased the perceived intensity of non-intended emotions (i.e., emotions that the actor did not intend to convey). Intensity ratings were unaffected by presentation duration (500ms vs 3000ms), or attitudes towards mask wearing. These findings shed light on the ambiguity that arises when interpreting the facial expressions of masked faces.

The impact of security countermeasures on human behavior during active shooter incidents

Active shooter incidents represent an increasing threat to American society, especially in commercial and educational buildings. In recent years, a wide variety of security countermeasures have been recommended by public and governmental agencies. Many of these countermeasures are aimed to increase building security, yet their impact on human behavior when an active shooter incident occurs remains underexplored. To fill this research gap, we conducted virtual experiments to evaluate the impact of countermeasures on human behavior during active shooter incidents. A total of 162 office workers and middle/high school teachers were recruited to respond to an active shooter incident in virtual office and school buildings with or without the implementation of multiple countermeasures. The experiment results showed countermeasures significantly influenced participants’ response time and decisions (e.g., run, hide, fight). Participants’ responses and perceptions of the active shooter incident were also contingent on their daily roles, as well as building and social contexts. Teachers had more concerns for occupants’ safety than office workers. Moreover, teachers had more positive perceptions of occupants in the school, whereas office workers had more positive perceptions of occupants in the office.

Characteristics of emotional gaze on threatening faces in children with autism spectrum disorders

Most evidence suggested that individuals with autism spectrum disorder (ASD) experienced gaze avoidance when looking at the eyes compared to typically developing (TD) individuals. Children with ASD magnified their fears when received threatening stimuli, resulting in a reduced duration of eye contact. Few studies have explored the gaze characteristics of children with ASD by dividing emotional faces into threatening and non-threatening pairs. In addition, although dynamic videos are more helpful in understanding the gaze characteristics of children with ASD, the experimental stimuli for some of the previous studies were still emotional pictures. We explored the viewing of dynamic threatening and non-threatening faces by children with ASD in different areas of interest (AOIs). In this study, 6-10 years old children with and without ASD viewed faces with threatening (fearful and angry) and non-threatening (sad and happy) expressions, respectively, with their eyes movements recorded. The results showed that when confronted with threatening faces, children with ASD, rather than TD, showed substantial eye avoidances, particularly non-specific avoidances in the fixation time on the mouths and significantly less time gazing at the mouths in any emotions, which was not observed for non-threatening faces. No correlations were found between the severity of symptoms and characteristics of gaze at the eyes and mouths in children with ASD. These results further enhance the understanding of the gaze characteristics of children with ASD on threatening and non-threatening faces and possibly provide additional evidence for their social interaction improvements.

Flexibility in autism during unpredictable shifts of socio-emotional stimuli: Investigation of group and sex differences

LAY ABSTRACT

Flexibility difficulties in autism might be particularly common in complex situations, when shifts (i.e. the switch of attentional resources or strategy according to the situation) are unpredictable, implicit (i.e. not guided by explicit rules) and the stimuli are complex. We analyzed the data of 101 autistic and 145 non-autistic adults, without intellectual deficiency, on two flexibility tasks performed online. The first task involved unpredictable and non-explicit shifts of complex socio-emotional stimuli, whereas the second task involved predictable and explicit shifts of character stimuli. Considering the discrepancies between laboratory results and the real-life flexibility-related challenges faced by autistic individuals, we need to determine which factor could be of particular importance in flexibility difficulties. We point out that the switch cost (i.e. the difference between shift and non-shift condition) was larger for autistic than for non-autistic participants on the complex flexibility task with unpredictable and non-explicit shifts of socio-emotional stimuli, whereas this was not the case when shifts were predictable, explicit and involved less complex stimuli. We also highlight sex differences, suggesting that autistic females have better social skills than autistic males and that they also have a specific cognitive profile, which could contribute to social camouflaging. The findings of this work help us understand which factors could influence flexibility difficulties in autism and are important for designing future studies. They also add to the literature on sex differences in autism which underpin better social skills, executive function, and camouflaging in autistic females.

Influence of scene-based expectation on facial expression perception: the moderating effect of cognitive load

Prior expectations play an important role in the process of perception. In real life, facial expressions always appear within a scene, which enables individuals to generate predictions that affect facial expression judgments. In the present study, using event-related potentials, we investigated the influence of scene-based expectation on facial expression processing. In addition, we used a cognitive task to manipulate cognitive load to interfere with scene-based expectation. Results showed that under the condition of sufficient cognitive resources, faces elicited more negative N170 amplitudes and more positive N400 amplitudes when the emotional valence of the scenes and faces was congruent. However, in the condition of cognitive load, no such difference was observed. The findings suggested that the effect of expectation on facial expression recognition emerges during both the early and late stages of facial expression processing, and the effect is weakened when cognitive resources are occupied by unrelated tasks.

The influence of emotional salience on gaze behavior in low and high trait empathy: an exploratory eye-tracking study

Previous studies have shown that situational factors like emotional salience are associated with higher subjective levels of state empathy. The present eye-tracking study explored whether gaze behavior varies as a function of emotional salience between individuals with low and high self-reported trait empathy. In a between-subjects design, we presented three social scene images in the context of different emotion conditions (Scene 1: neutral versus positive; Scene 2: neutral versus negative; Scene 3: positive versus negative) and assessed the dwell times of individuals with low versus high self-reported empathy (measured with the Toronto Empathy Questionnaire; TEQ). Analyses revealed that whereas low- and high-TEQ participants differed in their gaze behavior after receiving neutral information, they did not differ after receiving positive or negative information. Our preliminary results suggest that gaze behavior may be more indicative of self-reported trait empathy in situations with low emotional salience than in situations with high emotional salience.

Testing whether implicit emotion regulation mediates the association between discrimination and symptoms of psychopathology in late childhood: An RDoC perspective

Discrimination has been associated with adverse mental health outcomes, though it is unclear how early in life this association becomes apparent. Implicit emotion regulation, developing during childhood, is a foundational skill tied to a range of outcomes. Implicit emotion regulation has yet to be tested as an associated process for mental illness symptoms that can often emerge during this sensitive developmental period. Youth aged 9-11 were recruited for the Adolescent Brain Cognitive Development (ABCD) study. Associations between psychotic-like experiences, depressive symptoms, and total discrimination (due to race, ethnicity, nationality, weight, or sexual minority status) were tested, as well as associations with implicit emotion regulation measures (emotional updating working memory and inhibitory control). Analyses examined whether associations with symptoms were mediated by implicit emotion regulation. Discrimination related to decreased implicit emotion regulation performance, and increased endorsement of depressive symptoms and psychotic-like experiences. Emotional updating working memory performance partially mediated the association between discrimination and psychotic-like experiences, while emotional inhibitory control did not. Discrimination and implicit emotion regulation could serve as putative transdiagnostic markers of vulnerability. Results support the utility of using multiple units of analysis to improve understanding of complex emerging neurocognitive functions and developmentally sensitive periods.

Translating RDoC to Real-World Impact in Developmental Psychopathology: A Neurodevelopmental Framework for Application of Mental Health Risk Calculators

The National Institute of Mental Health Research Domain Criteria's (RDoC) has prompted a paradigm shift from categorical psychiatric disorders to considering multiple levels of vulnerability for probabilistic risk of disorder. However, the lack of neurodevelopmentally-based tools for clinical decision-making has limited RDoC's real-world impact. Integration with developmental psychopathology principles and statistical methods actualize the clinical implementation of RDoC to inform neurodevelopmental risk. In this conceptual paper, we introduce the probabilistic mental health risk calculator as an innovation for such translation and lay out a research agenda for generating an RDoC- and developmentally-informed paradigm that could be applied to predict a range of developmental psychopathologies from early childhood to young adulthood. We discuss methods that weigh the incremental utility for prediction based on intensity and burden of assessment, the addition of developmental change patterns, considerations for assessing outcomes, and integrative data approaches. Throughout, we illustrate the risk calculator approach with different neurodevelopmental pathways and phenotypes. Finally, we discuss real-world implementation of these methods for improving early identification and prevention of developmental psychopathology. We propose that mental health risk calculators can build a needed bridge between RDoC's multiple units of analysis and developmental science.

Human face and gaze perception is highly context specific and involves bottom-up and top-down neural processing

This review summarizes human perception and processing of face and gaze signals. Face and gaze signals are important means of non-verbal social communication. The review highlights that: (1) some evidence is available suggesting that the perception and processing of facial information starts in the prenatal period; (2) the perception and processing of face identity, expression and gaze direction is highly context specific, the effect of race and culture being a case in point. Culture affects by means of experiential shaping and social categorization the way in which information on face and gaze is collected and perceived; (3) face and gaze processing occurs in the so-called 'social brain'. Accumulating evidence suggests that the processing of facial identity, facial emotional expression and gaze involves two parallel and interacting pathways: a fast and crude subcortical route and a slower cortical pathway. The flow of information is bi-directional and includes bottom-up and top-down processing. The cortical networks particularly include the fusiform gyrus, superior temporal sulcus (STS), intraparietal sulcus, temporoparietal junction and medial prefrontal cortex.

Can a beautiful smile win the vote?: The role of candidates' physical attractiveness and facial expressions in elections

Several empirical studies have linked political candidates' electoral success to their physical appearance. We reexamine the effects of candidates' physical attractiveness by taking into account emotional facial expressions as measured by automated facial recognition software. The analysis is based on an observational case study of candidate characteristics in the 2017 German federal election. Using hierarchical regression modeling and controlling for candidates' displays of happiness, consistent effects of physical attractiveness remain. The results suggest that a potential interaction effect between displays of happiness and attractiveness positively affects vote shares. The study emphasizes the importance of considering emotional expressions when analyzing the impact of candidate appearance on electoral outcomes.

Establishing a role of the semantic control network in social cognitive processing: A meta-analysis of functional neuroimaging studies

The contribution and neural basis of cognitive control is under-specified in many prominent models of socio-cognitive processing. Important outstanding questions include whether there are multiple, distinguishable systems underpinning control and whether control is ubiquitously or selectively engaged across different social behaviours and task demands. Recently, it has been proposed that the regulation of social behaviours could rely on brain regions specialised in the controlled retrieval of semantic information, namely the anterior inferior frontal gyrus (IFG) and posterior middle temporal gyrus. Accordingly, we investigated for the first time whether the neural activation commonly found in social functional neuroimaging studies extends to these 'semantic control' regions. We conducted five coordinate-based meta-analyses to combine results of 499 fMRI/PET experiments and identified the brain regions consistently involved in semantic control, as well as four social abilities: theory of mind, trait inference, empathy and moral reasoning. This allowed an unprecedented parallel review of the neural networks associated with each of these cognitive domains. The results confirmed that the anterior left IFG region involved in semantic control is reliably engaged in all four social domains. This supports the hypothesis that social cognition is partly regulated by the neurocognitive system underpinning semantic control.

Affective Saturation Index: A Lexical Measure of Affect

Affect plays a major role in the individual's daily life, driving the sensemaking of experience, psychopathological conditions, social representations of phenomena, and ways of coping with others. The characteristics of affect have been traditionally investigated through physiological, self-report, and behavioral measures. The present article proposes a text-based measure to detect affect intensity: the Affective Saturation Index (ASI). The ASI rationale and the conceptualization of affect are overviewed, and an initial validation study on the ASI's convergent and concurrent validity is presented. Forty individuals completed a non-clinical semi-structured interview. For each interview transcript, the ASI was esteemed and compared to the individual's physiological index of propensity to affective arousal (measured by heart rate variability (HRV)); transcript semantic complexity (measured through the Semantic Entropy Index (SEI)); and lexical syntactic complexity (measured through the Flesch-Vacca Index (FVI)). ANOVAs and bi-variate correlations estimated the size of the relationships between indexes and sample characteristics (age, gender), then a set of multiple linear regressions tested the ASI's association with HRV, the SEI, and the FVI. Results support the ASI construct and criteria validity. The ASI proved able to detect affective saturation in interview transcripts (SEI and FVI, adjusted R2 = 0.428 and adjusted R2 = 0.241, respectively) and the way the text's affective saturation reflected the intensity of the individual's affective state (HRV, adjusted R2 = 0.428). In conclusion, although the specificity of the sample (psychology students) limits the findings' generalizability, the ASI provides the chance to use written texts to measure affect in accordance with a dynamic approach, independent of the spatio-temporal setting in which they were produced. In doing so, the ASI provides a way to empower the empirical analysis of fields such as psychotherapy and social group dynamics.

Visual and auditory brain areas share a representational structure that supports emotion perception

Emotionally expressive music and dance occur together across the world. This may be because features shared across the senses are represented the same way even in different sensory brain areas, putting music and movement in directly comparable terms. These shared representations may arise from a general need to identify environmentally relevant combinations of sensory features, particularly those that communicate emotion. To test the hypothesis that visual and auditory brain areas share a representational structure, we created music and animation stimuli with crossmodally matched features expressing a range of emotions. Participants confirmed that each emotion corresponded to a set of features shared across music and movement. A subset of participants viewed both music and animation during brain scanning, revealing that representations in auditory and visual brain areas were similar to one another. This shared representation captured not only simple stimulus features but also combinations of features associated with emotion judgments. The posterior superior temporal cortex represented both music and movement using this same structure, suggesting supramodal abstraction of sensory content. Further exploratory analysis revealed that early visual cortex used this shared representational structure even when stimuli were presented auditorily. We propose that crossmodally shared representations support mutually reinforcing dynamics across auditory and visual brain areas, facilitating crossmodal comparison. These shared representations may help explain why emotions are so readily perceived and why some dynamic emotional expressions can generalize across cultural contexts.

Hierarchical Attention-Based Multimodal Fusion Network for Video Emotion Recognition

The context, such as scenes and objects, plays an important role in video emotion recognition. The emotion recognition accuracy can be further improved when the context information is incorporated. Although previous research has considered the context information, the emotional clues contained in different images may be different, which is often ignored. To address the problem of emotion difference between different modes and different images, this paper proposes a hierarchical attention-based multimodal fusion network for video emotion recognition, which consists of a multimodal feature extraction module and a multimodal feature fusion module. The multimodal feature extraction module has three subnetworks used to extract features of facial, scene, and global images. Each subnetwork consists of two branches, where the first branch extracts the features of different modes, and the other branch generates the emotion score for each image. Features and emotion scores of all images in a modal are aggregated to generate the emotion feature of the modal. The other module takes multimodal features as input and generates the emotion score for each modal. Finally, features and emotion scores of multiple modes are aggregated, and the final emotion representation of the video will be produced. Experimental results show that our proposed method is effective on the emotion recognition dataset.

Gender Word Semantic Satiation Inhibits Facial Gender Information Processing

Abstract. In order to explore the time course of the influence of gender words semantic satiation on facial gender information processing, the semantic satiation paradigm was used to induce semantic satiation by presenting Chinese gender words “男, 女 (Male, Female)” for a long duration (25 s), with conjunction words “及(And), 且(Moreover)” served as the baseline (the Chinese words and their English translations do not completely equal in terms of pronunciation, form, and sense). Participants were asked to judge whether the two simultaneously presented faces (Experiment 1) or two successively presented faces (Experiment 2) were of the same gender. The results of Experiment 1 showed that the response time in semantic satiation condition was significantly longer than that of the baseline condition. The event-related potential (ERP) results of Experiment 2 showed that the peak amplitude of P1 component in semantic satiation condition was significantly smaller than that of the baseline condition in the early stage of face processing; N170, a specific component of face perception, in semantic satiation condition was significantly larger than that of the baseline condition. The average amplitude of LPC in semantic satiation condition was significantly smaller than that of the baseline condition. This study shows that facial gender information processing is affected by its semantic contextual information. The inhibition effect of gender word semantic satiation on facial gender information processing starts at the attention orientation stage, then continues to the face structural encoding stage, and eventually ends at the advanced cognitive response stage.

Using genetic algorithms to uncover individual differences in how humans represent facial emotion

Emotional facial expressions critically impact social interactions and cognition. However, emotion research to date has generally relied on the assumption that people represent categorical emotions in the same way, using standardized stimulus sets and overlooking important individual differences. To resolve this problem, we developed and tested a task using genetic algorithms to derive assumption-free, participant-generated emotional expressions. One hundred and five participants generated a subjective representation of happy, angry, fearful and sad faces. Population-level consistency was observed for happy faces, but fearful and sad faces showed a high degree of variability. High test-retest reliability was observed across all emotions. A separate group of 108 individuals accurately identified happy and angry faces from the first study, while fearful and sad faces were commonly misidentified. These findings are an important first step towards understanding individual differences in emotion representation, with the potential to reconceptualize the way we study atypical emotion processing in future research.

Cortical feedback and gating in odor discrimination and generalization

A central question in neuroscience is how context changes perception. In the olfactory system, for example, experiments show that task demands can drive divergence and convergence of cortical odor responses, likely underpinning olfactory discrimination and generalization. Here, we propose a simple statistical mechanism for this effect based on unstructured feedback from the central brain to the olfactory bulb, which represents the context associated with an odor, and sufficiently selective cortical gating of sensory inputs. Strikingly, the model predicts that both convergence and divergence of cortical odor patterns should increase when odors are initially more similar, an effect reported in recent experiments. The theory in turn predicts reversals of these trends following experimental manipulations and in neurological conditions that increase cortical excitability.

Joint Growth Trajectories of Trait Emotional Intelligence Subdomains Among L2 Language Learners: Estimating a Second-Order Factor-of-Curves Model With Emotion Perception

The present study assessed the developmental dynamics of trait emotional intelligence (TEI) and its subdomains during English as a foreign language (EFL) learning in a longitudinal study. A sample of 309 EFL learners (217 females, 92 males) was used to assess the trajectories of the global factor of TEI and the parallel development of the TEI subdomains over 1 year in the context of the EFL classroom using parallel process modeling (PPM) and factor of curve modeling (FCM). Additionally, emotion perception (EP) was used as a distal outcome to investigate how growth parameters, including intercept and slope factors in a TEI-FCM, influence the distal outcome of EP. The results revealed that there was sufficient inter-individual variation and intra-individual trends within each subdomain and a significant increase over time across the four subdomains. Additionally, concerning the covariances within and among the subdomains of TEI, the PPM results revealed moderate to high associations between the intercept and slope growth factors within and between these subdomains. Finally, regarding the direct association of the global growth factors (intercept and slope) of TEI on EP, the results indicated that the intercept and slope of global TEI were associated with EP (γ₀ = 1.127, p < 0.001; γ₁ = 0.321, p < 0.001). Specifically, the intercepts and slopes of emotionality and sociability turned out to be significantly linked to EP (γ₀₃ = 1.311, p < 0.001; γ₁₃ = 0.684, p < 0.001; γ₀₄ = 0.497, p < 0.001; γ₁₄ = 0.127, p < 0.001). These results suggest the dynamicity of TEI during learning a foreign language are discussed in this study in light of the potential variables associated with TEI and its related literature.