Perceptual and affective mechanisms in facial expression recognition: An integrative review

EEG During Dynamic Facial Emotion Processing Reveals Neural Activity Patterns Associated with Autistic Traits in Children

Altered brain connectivity and atypical neural oscillations have been observed in autism, yet their relationship with autistic traits in non-clinical populations remains underexplored. Here, we employ electroencephalography (EEG) to examine functional connectivity, oscillatory power, and broadband aperiodic activity during a dynamic facial emotion processing (FEP) task in 101 typically developing children aged 4-12 years. We investigate associations between these electrophysiological measures of brain dynamics and autistic traits as assessed by the Social Responsiveness Scale, 2nd Edition (SRS-2). Our results revealed that increased FEP-related connectivity across theta (4-7 Hz) and beta (13-30 Hz) frequencies correlated positively with higher SRS-2 scores, predominantly in right-lateralized (theta) and bilateral (beta) cortical networks. Additionally, a steeper 1/f-like aperiodic slope (spectral exponent) across fronto-central electrodes was associated with higher SRS-2 scores. Greater aperiodic-adjusted theta and alpha oscillatory power further correlated with both higher SRS-2 scores and steeper aperiodic slopes. These findings underscore important links between FEP-related brain dynamics and autistic traits in typically developing children. Future work could extend these findings to assess these EEG-derived markers as potential mechanisms underlying behavioural difficulties in autism.

The role of functional emotion circuits in distinct dimensions of psychopathology in youth

Several mental disorders emerge during childhood or adolescence and are often characterized by socioemotional difficulties, including alterations in emotion perception. Emotional facial expressions are processed in discrete functional brain modules whose connectivity patterns encode emotion categories, but the involvement of these neural circuits in psychopathology in youth is poorly understood. This study examined the associations between activation and functional connectivity patterns in emotion circuits and psychopathology during development. We used task-based fMRI data from the Philadelphia Neurodevelopmental Cohort (PNC, N = 1221, 8-23 years) and conducted generalized psycho-physiological interaction (gPPI) analyses. Measures of psychopathology were derived from an independent component analysis of questionnaire data. The results showed positive associations between identifying fearful, sad, and angry faces and depressive symptoms, and a negative relationship between sadness recognition and positive psychosis symptoms. We found a positive main effect of depressive symptoms on BOLD activation in regions overlapping with the default mode network, while individuals reporting higher levels of norm-violating behavior exhibited emotion-specific lower functional connectivity within regions of the salience network and between modules that overlapped with the salience and default mode network. Our findings illustrate the relevance of functional connectivity patterns underlying emotion processing for behavioral problems in children and adolescents.

Idiosyncratic fixation patterns generalize across dynamic and static facial expression recognition

Facial expression recognition (FER) is crucial for understanding the emotional state of others during human social interactions. It has been assumed that humans share universal visual sampling strategies to achieve this task. However, recent studies in face identification have revealed striking idiosyncratic fixation patterns, questioning the universality of face processing. More importantly, very little is known about whether such idiosyncrasies extend to the biological relevant recognition of static and dynamic facial expressions of emotion (FEEs). To clarify this issue, we tracked observers' eye movements categorizing static and ecologically valid dynamic faces displaying the six basic FEEs, all normalized for time presentation (1 s), contrast and global luminance across exposure time. We then used robust data-driven analyses combining statistical fixation maps with hidden Markov Models to explore eye-movements across FEEs and stimulus modalities. Our data revealed three spatially and temporally distinct equally occurring face scanning strategies during FER. Crucially, such visual sampling strategies were mostly comparably effective in FER and highly consistent across FEEs and modalities. Our findings show that spatiotemporal idiosyncratic gaze strategies also occur for the biologically relevant recognition of FEEs, further questioning the universality of FER and, more generally, face processing.

Expression perceptive fields explain individual differences in the recognition of facial emotions

Humans can use the facial expressions of another to infer their emotional state, although it remains unknown how this process occurs. Here we suppose the presence of perceptive fields within expression space, analogous to feature-tuned receptive-fields of early visual cortex. We developed genetic algorithms to explore a multidimensional space of possible expressions and identify those that individuals associated with different emotions. We next defined perceptive fields as probabilistic maps within expression space, and found that they could predict the emotions that individuals infer from expressions presented in a separate task. We found profound individual variability in their size, location, and specificity, and that individuals with more similar perceptive fields had similar interpretations of the emotion communicated by an expression, providing possible channels for social communication. Modelling perceptive fields therefore provides a predictive framework in which to understand how individuals infer emotions from facial expressions.

Visual Field Restriction in the Recognition of Basic Facial Expressions: A Combined Eye Tracking and Gaze Contingency Study

Uncertainties and discrepant results in identifying crucial areas for emotional facial expression recognition may stem from the eye tracking data analysis methods used. Many studies employ parameters of analysis that predominantly prioritize the examination of the foveal vision angle, ignoring the potential influences of simultaneous parafoveal and peripheral information. To explore the possible underlying causes of these discrepancies, we investigated the role of the visual field aperture in emotional facial expression recognition with 163 volunteers randomly assigned to three groups: no visual restriction (NVR), parafoveal and foveal vision (PFFV), and foveal vision (FV). Employing eye tracking and gaze contingency, we collected visual inspection and judgment data over 30 frontal face images, equally distributed among five emotions. Raw eye tracking data underwent Eye Movements Metrics and Visualizations (EyeMMV) processing. Accordingly, the visual inspection time, number of fixations, and fixation duration increased with the visual field restriction. Nevertheless, the accuracy showed significant differences among the NVR/FV and PFFV/FV groups, despite there being no difference in NVR/PFFV. The findings underscore the impact of specific visual field areas on facial expression recognition, highlighting the importance of parafoveal vision. The results suggest that eye tracking data analysis methods should incorporate projection angles extending to at least the parafoveal level.

Reading of ingroup politicians' smiles triggers smiling in the corner of one's eyes

Spontaneous smiles in response to politicians can serve as an implicit barometer for gauging electorate preferences. However, it is unclear whether a subtle Duchenne smile-an authentic expression involving the coactivation of the zygomaticus major (ZM) and orbicularis oculi (OO) muscles-would be elicited while reading about a favored politician smiling, indicating a more positive disposition and political endorsement. From an embodied simulation perspective, we investigated whether written descriptions of a politician's smile would trigger morphologically different smiles in readers depending on shared or opposing political orientation. In a controlled reading task in the laboratory, participants were presented with subject-verb phrases describing left and right-wing politicians smiling or frowning. Concurrently, their facial muscular reactions were measured via electromyography (EMG) recording at three facial muscles: the ZM and OO, coactive during Duchenne smiles, and the corrugator supercilii (CS) involved in frowning. We found that participants responded with a Duchenne smile detected at the ZM and OO facial muscles when exposed to portrayals of smiling politicians of same political orientation and reported more positive emotions towards these latter. In contrast, when reading about outgroup politicians smiling, there was a weaker activation of the ZM muscle and no activation of the OO muscle, suggesting a weak non-Duchenne smile, while emotions reported towards outgroup politicians were significantly more negative. Also, a more enhanced frown response in the CS was found for ingroup compared to outgroup politicians' frown expressions. Present findings suggest that a politician's smile may go a long way to influence electorates through both non-verbal and verbal pathways. They add another layer to our understanding of how language and social information shape embodied effects in a highly nuanced manner. Implications for verbal communication in the political context are discussed.

Independence Threat or Interdependence Threat? The Focusing Effect on Social or Physical Threat Modulates Brain Activity

OBJECTIVE

The neural basis of threat perception has mostly been examined separately for social or physical threats. However, most of the threats encountered in everyday life are complex. The features of interactions between social and physiological threats under different attentional conditions are unclear.

METHOD

The present study explores this issue using an attention-guided paradigm based on ERP techniques. The screen displays social threats (face threats) and physical threats (action threats), instructing participants to concentrate on only one type of threat, thereby exploring brain activation characteristics.

RESULTS

It was found that action threats did not affect the processing of face threats in the face-attention condition, and electrophysiological evidence from the brain suggests a comparable situation to that when processing face threats alone, with higher amplitudes of the N170 and EPN (Early Posterior Negativity) components of anger than neutral emotions. However, when focusing on the action-attention condition, the brain was affected by face threats, as evidenced by a greater N190 elicited by stimuli containing threatening emotions, regardless of whether the action was threatening or not. This trend was also reflected in EPN.

CONCLUSIONS

The current study reveals important similarities and differences between physical and social threats, suggesting that the brain has a greater processing advantage for social threats.

An exploratory study of the effect of age and gender on face scanning during affect recognition in immersive virtual reality

A person with impaired emotion recognition is not able to correctly identify facial expressions represented by other individuals. The aim of the present study is to assess eyes gaze and facial emotion recognition in a healthy population using dynamic avatars in immersive virtual reality (IVR). For the first time, the viewing of each area of interest of the face in IVR is studied by gender and age. This work in healthy people is conducted to assess the future usefulness of IVR in patients with deficits in the recognition of facial expressions. Seventy-four healthy volunteers participated in the study. The materials used were a laptop computer, a game controller, and a head-mounted display. Dynamic virtual faces randomly representing the six basic emotions plus neutral expression were used as stimuli. After the virtual human represented an emotion, a response panel was displayed with the seven possible options. Besides storing the hits and misses, the software program internally divided the faces into different areas of interest (AOIs) and recorded how long participants looked at each AOI. As regards the overall accuracy of the participants' responses, hits decreased from the youngest to the middle-aged and older adults. Also, all three groups spent the highest percentage of time looking at the eyes, but younger adults had the highest percentage. It is also noteworthy that attention to the face compared to the background decreased with age. Moreover, the hits between women and men were remarkably similar and, in fact, there were no statistically significant differences between them. In general, men paid more attention to the eyes than women, but women paid more attention to the forehead and mouth. In contrast to previous work, our study indicates that there are no differences between men and women in facial emotion recognition. Moreover, in line with previous work, the percentage of face viewing time for younger adults is higher than for older adults. However, contrary to earlier studies, older adults look more at the eyes than at the mouth.Consistent with other studies, the eyes are the AOI with the highest percentage of viewing time. For men the most viewed AOI is the eyes for all emotions in both hits and misses. Women look more at the eyes for all emotions, except for joy, fear, and anger on hits. On misses, they look more into the eyes for almost all emotions except surprise and fear.

Ensemble coding of multiple facial expressions is not affected by attentional load

Human observers can extract the mean emotion from multiple faces rapidly and precisely. However, whether attention is required in the ensemble coding of facial expressions remains debated. In this study, we examined the effect of attentional load on mean emotion processing with the dual-task paradigm. Individual emotion processing was also investigated as the control task. In the experiment, the letter string and a set of four happy or angry faces of various emotional intensities were shown. Participants had to complete the string task first, judging either the string color (low attention load) or the presence of the target letter (high attention load). Then a cue appeared indicating whether the secondary task was to evaluate the mean emotion of the faces or the emotion of the cued single face, and participants made their judgments on the visual analog scale. The results showed that compared with the color task, the letter task had a longer response time and lower accuracy, which verified the valid manipulation of the attention loads. More importantly, there was no significant difference in averaging performance between the low and high attention loads. By contrast, the individual face processing was impaired under the high attention load relative to the low attentional load. In addition, the advantage of extracting mean emotion over individual emotion was larger under the high attentional load. These results support the power of averaging and provide new evidence that a rather small amount of attention is needed in the ensemble coding of multiple facial expressions.

Facial Expression Recognition of Emotional Situations in Mild and Moderate Alzheimer's Disease

Background: Recognizing emotional situations may be impaired in people with Alzheimer's disease (AD). Purpose: We examined differences in the comprehension of an emotional situation in healthy older controls (HOC) and individuals with mild and moderate AD. Research Design: cross-sectional study. Study Sample: We assessed a convenience sample of 115 participants in 3 contexts: understanding the situation, ability to name the congruent emotion, and choice of the correct face in 4 emotional situations (sadness, surprise, anger, happiness). Data Colection: Chi-square and Mann-Whitney U tests were used for comparison between groups separated by CDR 1 and 2. Chi-square and Kruskal-Wallis tests were also used for comparison between groups separated by CDR 0, 1, and 2, with a pairwise comparisons analysis. Results: We found that the ability to understand, name, and choose the proper emotion is not linked and depends on the portrayed emotion. Conclusions: The findings suggest an interaction between emotional processing and cognitive functioning. Therefore, knowledge of an emotional condition and the connection to a specific facial choice most likely involve 2 degraded areas of knowledge, resulting in even higher odds of inaccuracy.

Emotional self-body odors do not influence the access to visual awareness by emotional faces

A growing body of research suggests that emotional chemosignals in others' body odor (BO), particularly those sampled during fearful states, enhance emotional face perception in conscious and preconscious stages. For instance, emotional faces access visual awareness faster when presented with others' fear BOs. However, the effect of these emotional signals in self-BO, that is, one's own BO, is still neglected in the literature. In the present work, we sought to determine whether emotional self-BOs modify the access to visual awareness of emotional faces. Thirty-eight women underwent a breaking-Continuous Flash Suppression task in which they were asked to detect fearful, happy, and neutral faces, as quickly and accurately as possible, while being exposed to their fear, happiness, and neutral self-BOs. Self-BOs were previously collected and later delivered via an olfactometer, using an event-related design. Results showed a main effect of emotional faces, with happy faces being detected significantly faster than fearful and neutral faces. However, our hypothesis that fear self-BOs would lead to faster emotional face detection was not confirmed, as no effect of emotional self-BOs was found-this was confirmed with Bayesian analysis. Although caution is warranted when interpreting these results, our findings suggest that emotional face perception is not modulated by emotional self-BOs, contrasting with the literature on others' BOs. Further research is needed to understand the role of self-BOs in visual processing and emotion perception.

Structural brain changes in emotion recognition across the adult lifespan

Emotion recognition (ER) declines with increasing age, yet little is known whether this observation is based on structural brain changes conveyed by differential atrophy. To investigate whether age-related ER decline correlates with reduced grey matter (GM) volume in emotion-related brain regions, we conducted a voxel-based morphometry analysis using data of the Human Connectome Project-Aging (N = 238, aged 36-87) in which facial ER was tested. We expected to find brain regions that show an additive or super-additive age-related change in GM volume indicating atrophic processes that reduce ER in older adults. The data did not support our hypotheses after correction for multiple comparisons. Exploratory analyses with a threshold of P < 0.001 (uncorrected), however, suggested that relationships between GM volume and age-related general ER may be widely distributed across the cortex. Yet, small effect sizes imply that only a small fraction of the decline of ER in older adults can be attributed to local GM volume changes in single voxels or their multivariate patterns.

Mass-univariate analysis of scalp ERPs reveals large effects of gaze fixation location during face processing that only weakly interact with face emotional expression

Decoding others' facial expressions is critical for social functioning. To clarify the neural correlates of expression perception depending on where we look on the face, three combined gaze-contingent ERP experiments were analyzed using robust mass-univariate statistics. Regardless of task, fixation location impacted face processing from 50 to 350 ms, maximally around 120 ms, reflecting retinotopic mapping around C2 and P1 components. Fixation location also impacted majorly the N170-P2 interval while weak effects were seen at the face-sensitive N170 peak. Results question the widespread assumption that faces are processed holistically into an indecomposable perceptual whole around the N170. Rather, face processing is a complex and view-dependent process that continues well beyond the N170. Expression and fixation location interacted weakly during the P1-N170 interval, supporting a role for the mouth and left eye in fearful and happy expression decoding. Expression effects were weakest at the N170 peak but strongest around P2, especially for fear, reflecting task-independent affective processing. Results suggest N170 reflects a transition between processes rather than the maximum of a holistic face processing stage. Focus on this peak should be replaced by data-driven analyses of the epoch using robust statistics to fully unravel the early visual processing of faces and their affective content.

The influence of selective attention to specific emotions on the processing of faces as revealed by event-related brain potentials

Event-related potential studies using affective words have indicated that selective attention to valence can increase affective discrimination at early perceptual stages. This effect most likely relies on neural associations between perceptual features of a stimulus and its affective value. Similar to words, emotional expressions in human faces are linked to specific visual elements. Therefore, selectively attending to a given emotion should allow for the preactivation of neural networks coding for the emotion and associated first-order visual elements, leading to enhanced early processing of faces expressing the attended emotion. To investigate this, we employed an expression detection task (N = 65). Fearful, happy, and neutral faces were randomly presented in three blocks while participants were instructed to respond only to one predefined target level of expression in each block. Reaction times were the fastest for happy target faces, which was accompanied by an increased occipital P1 for happy compared with fearful faces. The N170 yielded an arousal effect (emotional > neutral) while both components were not modulated by target status. In contrast, the early posterior negativity (EPN) arousal effect tended to be larger for target compared with nontarget faces. The late positive potential (LPP) revealed large effects of status and expression as well as an interaction driven by an increased LPP specifically for nontarget fearful faces. These findings tentatively indicate that selective attention to facial affect may enhance early emotional processing (EPN) even though further research is needed. Moreover, late controlled processing of facial emotions appears to involve a negativity bias.

Detection of emotional faces: The role of spatial frequencies and local features

Models of emotion processing suggest that threat-related stimuli such as fearful faces can be detected based on the rapid extraction of low spatial frequencies. However, this remains debated as other models argue that the decoding of facial expressions occurs with a more flexible use of spatial frequencies. The purpose of this study was to clarify the role of spatial frequencies and differences in luminance contrast between spatial frequencies, on the detection of facial emotions. We used a saccadic choice task in which emotional-neutral face pairs were presented and participants were asked to make a saccade toward the neutral or the emotional (happy or fearful) face. Faces were displayed either in low, high, or broad spatial frequencies. Results showed that participants were better to saccade toward the emotional face. They were also better for high or broad than low spatial frequencies, and the accuracy was higher with a happy target. An analysis of the eye and mouth saliency ofour stimuli revealed that the mouth saliency of the target correlates with participants' performance. Overall, this study underlines the importance of local more than global information, and of the saliency of the mouth region in the detection of emotional and neutral faces.

Specialized late cingulo-opercular network activation elucidates the mechanisms underlying decisions about ambiguity

Cortical task control networks, including the cingulo-opercular (CO) network play a key role in decision-making across a variety of functional domains. In particular, the CO network functions in a performance reporting capacity that supports successful task performance, especially in response to errors and ambiguity. In two studies testing the contribution of the CO network to ambiguity processing, we presented a valence bias task in which masked clearly and ambiguously valenced emotional expressions were slowly revealed over several seconds. This slow reveal task design provides a window into the decision-making mechanisms as they unfold over the course of a trial. In the main study, the slow reveal task was administered to 32 young adults in the fMRI environment and BOLD time courses were extracted from regions of interest in three control networks. In a follow-up study, the task was administered to a larger, online sample (n = 81) using a more extended slow reveal design with additional unmasking frames. Positive judgments of surprised faces were uniquely accompanied by slower response times and strong, late activation in the CO network. These results support the initial negativity hypothesis, which posits that the default response to ambiguity is negative and positive judgments are associated with a more effortful controlled process, and additionally suggest that this controlled process is mediated by the CO network. Moreover, ambiguous trials were characterized by a second CO response at the end of the trial, firmly placing CO function late in the decision-making process.

Human and machine recognition of dynamic and static facial expressions: prototypicality, ambiguity, and complexity

A growing body of research suggests that movement aids facial expression recognition. However, less is known about the conditions under which the dynamic advantage occurs. The aim of this research was to test emotion recognition in static and dynamic facial expressions, thereby exploring the role of three featural parameters (prototypicality, ambiguity, and complexity) in human and machine analysis. In two studies, facial expression videos and corresponding images depicting the peak of the target and non-target emotion were presented to human observers and the machine classifier (FACET). Results revealed higher recognition rates for dynamic stimuli compared to non-target images. Such benefit disappeared in the context of target-emotion images which were similarly well (or even better) recognised than videos, and more prototypical, less ambiguous, and more complex in appearance than non-target images. While prototypicality and ambiguity exerted more predictive power in machine performance, complexity was more indicative of human emotion recognition. Interestingly, recognition performance by the machine was found to be superior to humans for both target and non-target images. Together, the findings point towards a compensatory role of dynamic information, particularly when static-based stimuli lack relevant features of the target emotion. Implications for research using automatic facial expression analysis (AFEA) are discussed.

Developmental eye movement strategies for decoding facial expressions of emotion

In our daily lives, we routinely look at the faces of others to try to understand how they are feeling. Few studies have examined the perceptual strategies that are used to recognize facial expressions of emotion, and none have attempted to isolate visual information use with eye movements throughout development. Therefore, we recorded the eye movements of children from 5 years of age up to adulthood during recognition of the six "basic emotions" to investigate when perceptual strategies for emotion recognition become mature (i.e., most adult-like). Using iMap4, we identified the eye movement fixation patterns for recognition of the six emotions across age groups in natural viewing and gaze-contingent (i.e., expanding spotlight) conditions. While univariate analyses failed to reveal significant differences in fixation patterns, more sensitive multivariate distance analyses revealed a U-shaped developmental trajectory with the eye movement strategies of the 17- to 18-year-old group most similar to adults for all expressions. A developmental dip in strategy similarity was found for each emotional expression revealing which age group had the most distinct eye movement strategy from the adult group: the 13- to 14-year-olds for sadness recognition; the 11- to 12-year-olds for fear, anger, surprise, and disgust; and the 7- to 8-year-olds for happiness. Recognition performance for happy, angry, and sad expressions did not differ significantly across age groups, but the eye movement strategies for these expressions diverged for each group. Therefore, a unique strategy was not a prerequisite for optimal recognition performance for these expressions. Our data provide novel insights into the developmental trajectories underlying facial expression recognition, a critical ability for adaptive social relations.

Neural timing of the other-race effect across the lifespan: A review

Face race influences the way we process faces, so that faces of a different ethnic group are processed for identity less efficiently than faces of one's ethnic group - a phenomenon known as the Other-Race Effect (ORE). Although widely replicated, the ORE is still poorly characterized in terms of its development and the underlying mechanisms. In the last two decades, the Event-Related Potential (ERP) technique has brought insight into the mechanisms underlying the ORE and has demonstrated potential to clarify its development. Here, we review the ERP evidence for a differential neural processing of own-race and other-race faces throughout the lifespan. In infants, race-related processing differences emerged at the N290 and P400 (structural encoding) stages. In children, race affected the P100 (early processing, attention) perceptual stage and was implicitly encoded at the N400 (semantic processing) stage. In adults, processing difficulties for other-race faces emerged at the N170 (structural encoding), P200 (configuration processing) and N250 (accessing individual representations) perceptual stages. Early in processing, race was implicitly encoded from other-race faces (N100, P200 attentional biases) and in-depth processing preferentially applied to own-race faces (N200 attentional bias). Encoding appeared less efficient (Dm effects) and retrieval less recollection-based (old/new effects) for other-race faces. Evidence admits the contribution of perceptual, attentional, and motivational processes to the development and functioning of the ORE, offering no conclusive support for perceptual or socio-cognitive accounts. Cross-racial and non-cross-racial studies provided convergent evidence. Future research would need to include less represented ethnic populations and the developmental population.

Use of Technologies in the Therapy of Social Cognition Deficits in Neurological and Mental Diseases: A Systematic Review

Objective: This article systematically reviews the effects of technology-based (TB) treatments on impaired social cognition (SC) in neurological and mental disorders. Methods: Strictly adhering to the PRISMA guidelines, a systematic search was carried out in PsycINFO, PubMed, and Web of Science (last search: April 22, 2021) to identify studies that, implementing a control group design, evaluated TB treatments targeting deficits in emotion recognition, Theory of Mind (ToM) and social behavior in adult patients with nondevelopmental and nonprogressive neurological or mental disorders. Risk of bias was assessed using the PEDro Scale, certainty assessment followed the GRADE approach. Results: Sixteen studies involving 857 patients, all focusing on psychotic disorders, were retrieved. The most pronounced effects were observed concerning emotion recognition with all studies revealing overall improvements. Regarding ToM and social behavior, results were mixed. However, the number of studies including outcome measures for these domains, is significantly lower compared to the domain of emotion recognition, limiting the validity of the results. Risk of bias and certainty assessment revealed further limitations of evidence. Conclusion: TB treatment achieves positive effects especially with regard to emotion recognition impairments, at least for patients with schizophrenia. Future research should expand the evaluation of TB training of other SC domains, ought to be carried out in more diverse patient populations, rely on different devices, and include follow-up measurements.

Gauging response time distributions to examine the effect of facial expression inversion

Introduction

We used images of facial expressions (FEs) of emotion in a speeded Same/Different task to examine (i) distributional characteristics of response times (RTs) in relation to inter-stimulus similarity and (ii) the impact of inversion on FE processing.

Methods

Stimuli were seven emotion prototypes, posed by one male and one female, and eight intermediate morphs. Image pairs (N = 225) were presented for 500 ms, upright or inverted, in a block design, each 100 times.

Results

For both upright and inverted FEs, RTs were a non-monotonic function: median values were longest for stimulus pairs of intermediate similarity, decreasing for both more-dissimilar and more-similar pairs. RTs of "Same" and "Different" judgments followed ex-Gaussian distributions. The non-monotonicity is interpreted within a dual-process decision model framework as reflecting the infrequency of identical pairs, shifting the balance between the Same and Different processes. The effect of stimulus inversion was gauged by comparing RT-based multidimensional scaling solutions for the two presentation modes. Solutions for upright and inverted FEs showed little difference, with both displaying some evidence of categorical perception. The same features appeared in hierarchical clustering solutions.

Discussion

This outcome replicates and reinforces the solutions derived from accuracy of "Different" responses reported in our earlier companion paper. We attribute this lack of inversion effect to the brief exposure time, allowing low-level visual processing to dominate Same/Different decisions while elevating early featural analysis, which is insensitive to face orientation but enables initial positive/negative valence categorization of FEs.

The angry versus happy recognition advantage: the role of emotional and physical properties

Facial emotional expressions are pivotal for social communication. Their fast and accurate recognition is crucial to promote adaptive responses to social demands, for the development of functional relationships, and for well-being. However, the literature has been inconsistent in showing differentiated recognition patterns for positive vs. negative facial expressions (e.g., happy and angry expressions, respectively), likely due to affective and perceptual factors. Accordingly, the present study explored differences in recognition performance between angry and happy faces, while specifically assessing the role of emotional intensity and global/regional low-level visual features. 98 participants categorized angry and happy faces morphed between neutral and emotional across 9 levels of expression intensity (10-90%). We observed a significantly higher recognition efficiency (higher accuracy and shorter response latencies) for angry compared to happy faces in lower levels of expression intensity, suggesting that our cognitive resources are biased to prioritize the recognition of potentially harmful stimuli, especially when briefly presented at an ambiguous stage of expression. Conversely, an advantage for happy faces was observed from the midpoint of expression intensity, regarding response speed. However, when compensating for the contribution of regional low-level properties of distinct facial key regions, the effect of emotion was maintained only for response accuracy. Altogether, these results shed new light on the processing of facial emotional stimuli, emphasizing the need to consider emotional intensity and regional low-level image properties in emotion recognition analysis.

Emotional face recognition when a colored mask is worn: a cross-sectional study

Studies of the impact of face masks on emotional facial expression recognition are sparse in children. Moreover, to our knowledge no study has so far considered mask color (in adults and in children), even though this esthetic property is thought to have an impact on information processing. In order to explore these issues, the present study looked at whether first- and fifth-graders and young adults were influenced by the absence or presence (and color: pink, green, red, black, or white) of a face mask when asked to judge emotional facial expressions of fear, anger, sadness, or neutrality. Analysis of results suggested that the presence of a mask did affect the recognition of sad or fearful faces but did not influence significantly the perception of angry and neutral faces. Mask color slightly modulated the recognition of facial emotional expressions, without a systematic pattern that would allow a clear conclusion to be drawn. Moreover, none of these findings varied according to age group. The contribution of different facial areas to efficient emotion recognition is discussed with reference to methodological and theoretical considerations, and in the light of recent studies.

Interpersonal Consequences of Deceptive Expressions of Sadness

Emotional expressions evoke predictable responses from observers; displays of sadness are commonly met with sympathy and help from others. Accordingly, people may be motivated to feign emotions to elicit a desired response. In the absence of suspicion, we predicted that emotional and behavioral responses to genuine (vs. deceptive) expressers would be guided by empirically valid cues of sadness authenticity. Consistent with this hypothesis, untrained observers (total N = 1,300) reported less sympathy and offered less help to deceptive (vs. genuine) expressers of sadness. This effect was replicated using both posed, low-stakes, laboratory-created stimuli, and spontaneous, real, high-stakes emotional appeals to the public. Furthermore, lens models suggest that sympathy reactions were guided by difficult-to-fake facial actions associated with sadness. Results suggest that naive observers use empirically valid cues to deception to coordinate social interactions, providing novel evidence that people are sensitive to subtle cues to deception.

What is missing in the study of emotion expression?

While approaching celebrations for the 150 years of "The Expression of the Emotions in Man and Animals", scientists' conclusions on emotion expression are still debated. Emotion expression has been traditionally anchored to prototypical and mutually exclusive facial expressions (e.g., anger, disgust, fear, happiness, sadness, and surprise). However, people express emotions in nuanced patterns and - crucially - not everything is in the face. In recent decades considerable work has critiqued this classical view, calling for a more fluid and flexible approach that considers how humans dynamically perform genuine expressions with their bodies in context. A growing body of evidence suggests that each emotional display is a complex, multi-component, motoric event. The human face is never static, but continuously acts and reacts to internal and environmental stimuli, with the coordinated action of muscles throughout the body. Moreover, two anatomically and functionally different neural pathways sub-serve voluntary and involuntary expressions. An interesting implication is that we have distinct and independent pathways for genuine and posed facial expressions, and different combinations may occur across the vertical facial axis. Investigating the time course of these facial blends, which can be controlled consciously only in part, is recently providing a useful operational test for comparing the different predictions of various models on the lateralization of emotions. This concise review will identify shortcomings and new challenges regarding the study of emotion expressions at face, body, and contextual levels, eventually resulting in a theoretical and methodological shift in the study of emotions. We contend that the most feasible solution to address the complex world of emotion expression is defining a completely new and more complete approach to emotional investigation. This approach can potentially lead us to the roots of emotional display, and to the individual mechanisms underlying their expression (i.e., individual emotional signatures).

The time course of categorical perception of facial expressions

Decoding emotions on others' faces is one of the most important functions of the human brain, which has been widely studied in cognitive neuroscience. However, the precise time course of facial expression categorization in the human brain is still a matter of debate. Here we used an original paradigm to measure categorical perception of facial expression changes during event-related potentials (ERPs) recording, in which a face stimulus dynamically switched either to a different expression (between-category condition) or to the same expression (within-category condition), the physical distance between the two successive faces being equal across conditions. The switch between faces generated a negative differential potential peaking at around 160 ms over occipito-temporal regions, similar in term of latency and topography to the well-known face-selective N170 component. This response was larger in the condition where the switch occurred between faces that were perceived as having different facial expressions compared to the same expression. In addition, happy expressions were categorized around 20 ms faster than fearful expressions (respectively, 135 and 156 ms). These findings provide evidence that changes of facial expressions are categorically perceived as early as 160 ms following stimulus onset over the occipito-temporal cortex.

The age bias in labeling facial expressions in children: Effects of intensity and expression

Emotion reasoning, including labeling of facial expressions, is an important building block for a child's social development. This study investigated age biases in labeling facial expressions in children and adults, focusing on the influence of intensity and expression on age bias. Children (5 to 14 years old; N = 152) and adults (19 to 25 years old; N = 30) labeled happiness, disgust or sadness at five intensity levels (0%; 25%; 50%; 75%; and 100%) in facial images of children and adults. Sensitivity was computed for each of the expression-intensity combinations, separately for the child and adult faces. Results show that children and adults have an age bias at low levels of intensity (25%). In the case of sadness, children have an age bias for all intensities. Thus, the impact of the age of the face seems largest for expressions which might be most difficult to recognise. Moreover, both adults and children label most expressions best in adult rather than child faces, leading to an other-age bias in children and an own-age bias in adults. Overall, these findings reveal that both children and adults exhibit an age bias in labeling subtle facial expressions of emotions.

High spatial frequencies disrupt conscious visual recognition: evidence from an attentional blink paradigm

In this article, we tested the respective importance of low spatial frequencies (LSF) and high spatial frequencies (HSF) for conscious visual recognition of emotional stimuli by using an attentional blink paradigm. Thirty-eight participants were asked to identify and report two targets (happy faces) embedded in a rapid serial visual presentation of distractors (angry faces). During attentional blink, conscious perception of the second target (T2) is usually altered when the lag between the two targets is short (200-500 ms) but is restored at longer lags. The distractors between T1 and T2 were either non-filtered (broad spatial frequencies, BSF), low-pass filtered (LSF), or high-pass filtered (HSF). Assuming that prediction abilities could be at the root of conscious visual recognition, we expected that LSF distractors could result in a greater disturbance of T2 reporting than HSF distractors. Results showed that both LSF and HSF play a role in the emergence of exogenous consciousness in the visual system. Furthermore, HSF distractors strongly affected T1 and T2 reporting irrespective of the lag between targets, suggesting their role for facial emotion processing. We discuss these results with regards to other models of visual recognition. .

A deep neural network model of the primate superior colliculus for emotion recognition

Although sensory processing is pivotal to nearly every theory of emotion, the evaluation of the visual input as 'emotional' (e.g. a smile as signalling happiness) has been traditionally assumed to take place in supramodal 'limbic' brain regions. Accordingly, subcortical structures of ancient evolutionary origin that receive direct input from the retina, such as the superior colliculus (SC), are traditionally conceptualized as passive relay centres. However, mounting evidence suggests that the SC is endowed with the necessary infrastructure and computational capabilities for the innate recognition and initial categorization of emotionally salient features from retinal information. Here, we built a neurobiologically inspired convolutional deep neural network (DNN) model that approximates physiological, anatomical and connectional properties of the retino-collicular circuit. This enabled us to characterize and isolate the initial computations and discriminations that the DNN model of the SC can perform on facial expressions, based uniquely on the information it directly receives from the virtual retina. Trained to discriminate facial expressions of basic emotions, our model matches human error patterns and above chance, yet suboptimal, classification accuracy analogous to that reported in patients with V1 damage, who rely on retino-collicular pathways for non-conscious vision of emotional attributes. When presented with gratings of different spatial frequencies and orientations never 'seen' before, the SC model exhibits spontaneous tuning to low spatial frequencies and reduced orientation discrimination, as can be expected from the prevalence of the magnocellular (M) over parvocellular (P) projections. Likewise, face manipulation that biases processing towards the M or P pathway affects expression recognition in the SC model accordingly, an effect that dovetails with variations of activity in the human SC purposely measured with ultra-high field functional magnetic resonance imaging. Lastly, the DNN generates saliency maps and extracts visual features, demonstrating that certain face parts, like the mouth or the eyes, provide higher discriminative information than other parts as a function of emotional expressions like happiness and sadness. The present findings support the contention that the SC possesses the necessary infrastructure to analyse the visual features that define facial emotional stimuli also without additional processing stages in the visual cortex or in 'limbic' areas. This article is part of the theme issue 'Cracking the laugh code: laughter through the lens of biology, psychology and neuroscience'.

Law Enforcement Officers’ Ability to Recognize Emotions: The Role of Personality Traits and Basic Needs’ Satisfaction

Background: This study intended to explore the role of personality traits and basic psychological needs in law enforcement officers’ ability to recognize emotions: anger, joy, sadness, fear, surprise, disgust, and neutral. It was significant to analyze law enforcement officers’ emotion recognition and the contributing factors, as this field has been under-researched despite increased excessive force use by officers in many countries. Methods: This study applied the Big Five–2 (BFI-2), the Basic Psychological Needs Satisfaction and Frustration Scale (BPNSFS), and the Karolinska Directed Emotional Faces set of stimuli (KDEF). The data was gathered using an online questionnaire provided directly to law enforcement agencies. A total of 154 law enforcement officers participated in the study, 50.65% were females, and 49.35% were males. The mean age was 41.2 (age range = 22–61). In order to analyze the data, SEM and multiple linear regression methods were used. Results: This study analyzed variables of motion recognition, personality traits, and needs satisfaction and confirmed that law enforcement officers’ personality traits play a significant role in emotion recognition. Respondents’ agreeableness significantly predicted increased overall emotion recognition; conscientiousness predicted increased anger recognition; joy recognition was significantly predicted by extraversion, neuroticism, and agreeableness. This study also confirmed that law enforcement officers’ basic psychological needs satisfaction/frustration play a significant role in emotion recognition. Respondents’ relatedness satisfaction significantly predicted increased overall emotion recognition, fear recognition, joy recognition, and sadness recognition. Relatedness frustration significantly predicted decreased anger recognition, surprise recognition, and neutral face recognition. Furthermore, this study confirmed links between law enforcement officers’ personality traits, satisfaction/frustration of basic psychological needs, and emotion recognition, χ2 = 57.924; df = 41; p = 0.042; TLI = 0.929; CFI = 0.956; RMSEA = 0.042 [0.009–0.065]. Discussion: The findings suggested that agreeableness, conscientiousness, extraversion, and neuroticism play an essential role in satisfaction and frustration of relatedness needs, which, subsequently, link to emotion recognition. Due to the relatively small sample size, the issues of validity/reliability of some instruments, and other limitations, the results of this study should preferably be regarded with concern.

Facial Expressions and Self-Reported Emotions When Viewing Nature Images

Many studies have demonstrated that exposure to simulated natural scenes has positive effects on emotions and reduces stress. In the present study, we investigated emotional facial expressions while viewing images of various types of natural environments. Both automated facial expression analysis by iMotions' AFFDEX 8.1 software (iMotions, Copenhagen, Denmark) and self-reported emotions were analyzed. Attractive and unattractive natural images were used, representing either open or closed natural environments. The goal was to further understand the actual features and characteristics of natural scenes that could positively affect emotional states and to evaluate face reading technology to measure such effects. It was predicted that attractive natural scenes would evoke significantly higher levels of positive emotions than unattractive scenes. The results showed generally small values of emotional facial expressions while observing the images. The facial expression of joy was significantly higher than that of other registered emotions. Contrary to predictions, there was no difference between facial emotions while viewing attractive and unattractive scenes. However, the self-reported emotions evoked by the images showed significantly larger differences between specific categories of images in accordance with the predictions. The differences between the registered emotional facial expressions and self-reported emotions suggested that the participants more likely described images in terms of common stereotypes linked with the beauty of natural environments. This result might be an important finding for further methodological considerations.

A neural marker of rapid discrimination of facial expression in 3.5- and 7-month-old infants

Infants’ ability to discriminate facial expressions has been widely explored, but little is known about the rapid and automatic ability to discriminate a given expression against many others in a single experiment. Here we investigated the development of facial expression discrimination in infancy with fast periodic visual stimulation coupled with scalp electroencephalography (EEG). EEG was recorded in eighteen 3.5- and eighteen 7-month-old infants presented with a female face expressing disgust, happiness, or a neutral emotion (in different stimulation sequences) at a base stimulation frequency of 6 Hz. Pictures of the same individual expressing other emotions (either anger, disgust, fear, happiness, sadness, or neutrality, randomly and excluding the expression presented at the base frequency) were introduced every six stimuli (at 1 Hz). Frequency-domain analysis revealed an objective (i.e., at the predefined 1-Hz frequency and harmonics) expression-change brain response in both 3.5- and 7-month-olds, indicating the visual discrimination of various expressions from disgust, happiness and neutrality from these early ages. At 3.5 months, the responses to the discrimination from disgust and happiness expressions were located mainly on medial occipital sites, whereas a more lateral topography was found for the response to the discrimination from neutrality, suggesting that expression discrimination from an emotionally neutral face relies on distinct visual cues than discrimination from a disgust or happy face. Finally, expression discrimination from happiness was associated with a reduced activity over posterior areas and an additional response over central frontal scalp regions at 7 months as compared to 3.5 months. This result suggests developmental changes in the processing of happiness expressions as compared to negative/neutral ones within this age range.

Intranasal oxytocin alters attention to emotional facial expressions, particularly for males and those with depressive symptoms

Intranasal oxytocin (OT) can enhance emotion recognition, perhaps by promoting increased attention to social cues. Some studies indicate that individuals with difficulties processing social information, including those with psychopathology, show more pronounced effects in response to OT. As such, there is interest in the potential therapeutic use of OT in populations with deficits in social cognition. The present study examined the effects of intranasal OT on the processing of facial features and selective attention to emotional facial expressions, as well as whether individual differences in depressive symptom severity predict sensitivity to intranasal OT. In a double-blind placebo-controlled within-subject design, eye tracking was used to measure attention to facial features in an emotional expression appraisal task, and attention to emotional expressions in a free-viewing task with a quadrant of multiple faces. OT facilitated the processing of positive cues, enhancing the maintenance of attention to the mouth region of happy faces and to happy faces within a quadrant, with similar effect sizes, despite the latter effect not being statistically significant. Further, persons with depressive symptoms, and particularly males, were sensitive to OT's effects. For males only, OT, relative to placebo, increased attentional focus to the mouth region of all faces. Individuals with depressive symptoms showed less attentional focus on angry (males only) and sad facial expressions, and more attention to happy faces (particularly for males). Results indicate increased sensitivity to OT in males and persons at risk for depression, with OT administration promoting a positive bias in selective attention to social stimuli.

The Relations of Attention to and Clarity of Feelings With Facial Affect Perception

Attention to emotions and emotional clarity are core dimensions of individual differences in emotion awareness. Findings from prior research based on self-report indicate that attention to and recognition of one's own emotions are related to attention to and recognition of other people's emotions. In the present experimental study, we examined the relations of attention to and clarity of emotions with the efficiency of facial affect perception. Moreover, it was explored whether attention to and clarity of emotions are linked to negative interpretations of facial expressions. A perception of facial expressions (PFE) task based on schematic faces with neutral, ambiguous, or unambiguous emotional expressions and a gender decision task were administered to healthy individuals along with measures of emotion awareness, state and trait anxiety, depression, and verbal intelligence. Participants had to decide how much the faces express six basic affects. Evaluative ratings and decision latencies were analyzed. Attention to feelings was negatively correlated with evaluative decision latency, whereas clarity of feelings was not related to decision latency in the PFE task. Attention to feelings was positively correlated with the perception of negative affects in ambiguous faces. Attention to feelings and emotional clarity were not related to gender decision latency. According to our results, dispositional attention to feelings goes along with an enhanced efficiency of facial affect perception. Habitually paying attention to one's own emotions may facilitate processing of external emotional information. Preliminary evidence was obtained suggesting a relationship of dispositional attention to feelings with negative interpretations of facial expressions.

Are Face Masks a Problem for Emotion Recognition? Not When the Whole Body Is Visible

The rise of the novel COVID-19 virus has made face masks commonplace items around the globe. Recent research found that face masks significantly impair emotion recognition on isolated faces. However, faces are rarely seen in isolation and the body is also a key cue for emotional portrayal. Here, therefore, we investigated the impact of face masks on emotion recognition when surveying the full body. Stimuli expressing anger, happiness, sadness, and fear were selected from the BEAST stimuli set. Masks were added to these images and participants were asked to recognize the emotion and give a confidence level for that decision for both the masked and unmasked stimuli. We found that, contrary to some work viewing faces in isolation, emotion recognition was generally not impaired by face masks when the whole body is present. We did, however, find that when viewing masked faces, only the recognition of happiness significantly decreased when the whole body was present. In contrast to actual performance, confidence levels were found to decline during the Mask condition across all emotional conditions. This research suggests that the impact of masks on emotion recognition may not be as pronounced as previously thought, as long as the whole body is also visible.

EEG decoding of multidimensional information from emotional faces

Humans can detect and recognize faces quickly, but there has been little research on the temporal dynamics of the different dimensional face information that is extracted. The present study aimed to investigate the time course of neural responses to the representation of different dimensional face information, such as age, gender, emotion, and identity. We used support vector machine decoding to obtain representational dissimilarity matrices of event-related potential responses to different faces for each subject over time. In addition, we performed representational similarity analysis with the model representational dissimilarity matrices that contained different dimensional face information. Three significant findings were observed. First, the extraction process of facial emotion occurred before that of facial identity and lasted for a long time, which was specific to the right frontal region. Second, arousal was preferentially extracted before valence during the processing of facial emotional information. Third, different dimensional face information exhibited representational stability during different periods. In conclusion, these findings reveal the precise temporal dynamics of multidimensional information processing in faces and provide powerful support for computational models on emotional face perception.

The role of sex and emotion on emotion perception in artificial faces: An ERP study

Sex has a significant impact on the perception of emotional expressions. However, it remains unclear whether sex influences the perception of emotions in artificial faces, which are becoming popular in emotion research. We used an emotion recognition task with FaceGen faces portraying six basic emotions aiming to investigate the effect of sex and emotion on behavioural and electrophysiological parameters. 71 participants performed the task while EEG was recorded. The recognition of sadness was the poorest, however, females recognized sadness better than males. ERP results indicated that fear, disgust, and anger evoked higher amplitudes of late positive potential over the left parietal region compared to neutral expression. Females demonstrated higher values of global field power as compared to males. The interaction between sex and emotion on ERPs was not significant. The results of our study may be valuable for future therapies and research, as it emphasizes possibly distinct processing of emotions and potential sex differences in the recognition of emotional expressions in FaceGen faces.

Shared Attention Amplifies the Neural Processing of Emotional Faces

Sharing an experience, without communicating, affects people's subjective perception of the experience, often by intensifying it. We investigated the neural mechanisms underlying shared attention by implementing an EEG study where participants attended to and rated the intensity of emotional faces, simultaneously or independently. Participants performed the task in three experimental conditions: (a) alone; (b) simultaneously next to each other in pairs, without receiving feedback of the other's responses (shared without feedback); and (c) simultaneously while receiving the feedback (shared with feedback). We focused on two face-sensitive ERP components: The amplitude of the N170 was greater in the "shared with feedback" condition compared to the alone condition, reflecting a top-down effect of shared attention on the structural encoding of faces, whereas the EPN was greater in both shared context conditions compared to the alone condition, reflecting an enhanced attention allocation in the processing of emotional content of faces, modulated by the social context. Taken together, these results suggest that shared attention amplifies the neural processing of faces, regardless of the valence of facial expressions.

Face Masks Impair Basic Emotion Recognition

Abstract. With the widespread adoption of masks, there is a need for understanding how facial obstruction affects emotion recognition. We asked 120 participants to identify emotions from faces with and without masks. We also examined if recognition performance was related to autistic traits and personality. Masks impacted recognition of expressions with diagnostic lower face features the most and those with diagnostic upper face features the least. Persons with higher autistic traits were worse at identifying unmasked expressions, while persons with lower extraversion and higher agreeableness were better at recognizing masked expressions. These results show that different features play different roles in emotion recognition and suggest that obscuring features affects social communication differently as a function of autistic traits and personality.

Perceptive and Affective Impairments In Emotive Eye-Region Processing in Alexithymia

Alexithymia is characterized by impairments in emotion processing, frequently linked to facial expressions of emotion. The eye-region conveys information necessary for emotion processing. It has been demonstrated that alexithymia is associated with reduced attention to the eyes, but little is known regarding the cognitive and electrophysiological mechanisms underlying emotive eye-region processing in alexithymia. Here, we recorded behavioral and electrophysiological responses of individuals with alexithymia (ALEX; n = 25) and individuals without alexithymia (NonALEX; n = 23) while they viewed intact and eyeless faces with angry and sad expressions during a dual-target rapid serial visual presentation task. Results showed different eye-region focuses and differentiating N1 responses between intact and eyeless faces to anger and sadness in NonALEX, but not in ALEX, suggesting deficient perceptual processing of the eye-region in alexithymia. Reduced eye-region focus and smaller differences in frontal alpha asymmetry in response to sadness between intact and eyeless faces were observed in ALEX than NonALEX, indicative of impaired affective processing of the eye-region in alexithymia. These findings highlight perceptual and affective abnormalities of emotive eye-region processing in alexithymia. Our results contribute to understanding the neuropsychopathology of alexithymia and alexithymia-related disorders.

Are adaptation aftereffects for facial emotional expressions affected by prior knowledge about the emotion?

Accurate perception of the emotional signals conveyed by others is crucial for successful social interaction. Such perception is influenced not only by sensory input, but also by knowledge we have about the others' emotions. This study addresses the issue of whether knowing that the other's emotional state is congruent or incongruent with their displayed emotional expression ("genuine" and "fake", respectively) affects the neural mechanisms underpinning the perception of their facial emotional expressions. We used a visual adaptation paradigm to investigate this question in three experiments employing increasing adaptation durations. The adapting stimuli consisted of photographs of emotional facial expressions of joy and anger, purported to reflect (in-)congruency between felt and expressed emotion, displayed by professional actors. A Validity checking procedure ensured participants had the correct knowledge about the (in-)congruency. Significantly smaller adaptation aftereffects were obtained when participants knew that the displayed expression was incongruent with the felt emotion, following all tested adaptation periods. This study shows that knowledge relating to the congruency between felt and expressed emotion modulates face expression aftereffects. We argue that this reflects that the neural substrate responsible for the perception of facial expressions of emotion incorporates the presumed felt emotion underpinning the expression.

Flexibility in autism during unpredictable shifts of socio-emotional stimuli: Investigation of group and sex differences

LAY ABSTRACT

Flexibility difficulties in autism might be particularly common in complex situations, when shifts (i.e. the switch of attentional resources or strategy according to the situation) are unpredictable, implicit (i.e. not guided by explicit rules) and the stimuli are complex. We analyzed the data of 101 autistic and 145 non-autistic adults, without intellectual deficiency, on two flexibility tasks performed online. The first task involved unpredictable and non-explicit shifts of complex socio-emotional stimuli, whereas the second task involved predictable and explicit shifts of character stimuli. Considering the discrepancies between laboratory results and the real-life flexibility-related challenges faced by autistic individuals, we need to determine which factor could be of particular importance in flexibility difficulties. We point out that the switch cost (i.e. the difference between shift and non-shift condition) was larger for autistic than for non-autistic participants on the complex flexibility task with unpredictable and non-explicit shifts of socio-emotional stimuli, whereas this was not the case when shifts were predictable, explicit and involved less complex stimuli. We also highlight sex differences, suggesting that autistic females have better social skills than autistic males and that they also have a specific cognitive profile, which could contribute to social camouflaging. The findings of this work help us understand which factors could influence flexibility difficulties in autism and are important for designing future studies. They also add to the literature on sex differences in autism which underpin better social skills, executive function, and camouflaging in autistic females.

High spatial frequency filtered primes hastens happy faces categorization in autistic adults

Coarse information of a visual stimulus is conveyed by Low Spatial Frequencies (LSF) and is thought to be rapidly extracted to generate predictions. This may guide fast recognition with the subsequent integration of fine information, conveyed by High Spatial Frequencies (HSF). In autism, emotional face recognition is challenging, and might be related to alterations in LSF predictive processes. We analyzed the data of 27 autistic and 34 non autistic (NA) adults on an emotional Stroop task (i.e., emotional face with congruent or incongruent emotional word) with spatially filtered primes (HSF vs.LSF). We hypothesized that LSF primes would generate predictions leading to faster categorization of the target face compared to HSF primes, in the NA group but not in autism. Surprisingly, HSF primes led to faster categorization than LSF primes in both groups. Moreover, the advantage of HSF vs.LSF primes was stronger for angry than happy faces in NA, but was stronger for happy than angry faces in autistic participants. Drift diffusion modelling confirmed HSF advantage and showed a longer non-decision time (e.g., encoding) in autism. Despite LSF predictive impairments in autism was not corroborated, our analyses suggest low level processing specificities in autism.

The rapid and automatic categorization of facial expression changes in highly variable natural images

Emotional expressions are quickly and automatically read from human faces under natural viewing conditions. Yet, categorization of facial expressions is typically measured in experimental contexts with homogenous sets of face stimuli. Here we evaluated how the 6 basic facial emotions (Fear, Disgust, Happiness, Anger, Surprise or Sadness) can be rapidly and automatically categorized with faces varying in head orientation, lighting condition, identity, gender, age, ethnic origin and background context. High-density electroencephalography was recorded in 17 participants viewing 50 s sequences with natural variable images of neutral-expression faces alternating at a 6 Hz rate. Every five stimuli (1.2 Hz), variable natural images of one of the six basic expressions were presented. Despite the wide physical variability across images, a significant F/5 = 1.2 Hz response and its harmonics (e.g., 2F/5 = 2.4 Hz, etc.) was observed for all expression changes at the group-level and in every individual participant. Facial categorization responses were found mainly over occipito-temporal sites, with distinct hemispheric lateralization and cortical topographies according to the different expressions. Specifically, a stronger response was found to Sadness categorization, especially over the left hemisphere, as compared to Fear and Happiness, together with a right hemispheric dominance for categorization of Fearful faces. Importantly, these differences were specific to upright faces, ruling out the contribution of low-level visual cues. Overall, these observations point to robust rapid and automatic facial expression categorization processes in the human brain.

Mask exposure during COVID-19 changes emotional face processing

Faces are one of the key ways that we obtain social information about others. They allow people to identify individuals, understand conversational cues, and make judgements about others' mental states. When the COVID-19 pandemic hit the United States, widespread mask-wearing practices were implemented, causing a shift in the way Americans typically interact. This introduction of masks into social exchanges posed a potential challenge-how would people make these important inferences about others when a large source of information was no longer available? We conducted two studies that investigated the impact of mask exposure on emotion perception. In particular, we measured how participants used facial landmarks (visual cues) and the expressed valence and arousal (affective cues), to make similarity judgements about pairs of emotion faces. Study 1 found that in August 2020, participants with higher levels of mask exposure used cues from the eyes to a greater extent when judging emotion similarity than participants with less mask exposure. Study 2 measured participants' emotion perception in both April and September 2020 -before and after widespread mask adoption-in the same group of participants to examine changes in the use of facial cues over time. Results revealed an overall increase in the use of visual cues from April to September. Further, as mask exposure increased, people with the most social interaction showed the largest increase in the use of visual facial cues. These results provide evidence that a shift has occurred in how people process faces such that the more people are interacting with others that are wearing masks, the more they have learned to focus on visual cues from the eye area of the face.

The Role of Emotional Content and Perceptual Saliency During the Programming of Saccades Toward Faces

Previous studies have shown that the human visual system can detect a face and elicit a saccadic eye movement toward it very efficiently compared to other categories of visual stimuli. In the first experiment, we tested the influence of facial expressions on fast face detection using a saccadic choice task. Face-vehicle pairs were simultaneously presented and participants were asked to saccade toward the target (the face or the vehicle). We observed that saccades toward faces were initiated faster, and more often in the correct direction, than saccades toward vehicles, regardless of the facial expressions (happy, fearful, or neutral). We also observed that saccade endpoints on face images were lower when the face was happy and higher when it was neutral. In the second experiment, we explicitly tested the detection of facial expressions. We used a saccadic choice task with emotional-neutral pairs of faces and participants were asked to saccade toward the emotional (happy or fearful) or the neutral face. Participants were faster when they were asked to saccade toward the emotional face. They also made fewer errors, especially when the emotional face was happy. Using computational modeling, we showed that this happy face advantage can, at least partly, be explained by perceptual factors. Also, saccade endpoints were lower when the target was happy than when it was fearful. Overall, we suggest that there is no automatic prioritization of emotional faces, at least for saccades with short latencies, but that salient local face features can automatically attract attention.

Emotion, Gender and Gaze Discrimination Tasks do not Differentially Impact the Neural Processing of Angry or Happy Facial Expressions-a Mass Univariate ERP Analysis

Facial expression processing is a critical component of social cognition yet, whether it is influenced by task demands at the neural level remains controversial. Past ERP studies have found mixed results with classic statistical analyses, known to increase both Type I and Type II errors, which Mass Univariate statistics (MUS) control better. However, MUS open-access toolboxes can use different fundamental statistics, which may lead to inconsistent results. Here, we compared the output of two MUS toolboxes, LIMO and FMUT, on the same data recorded during the processing of angry and happy facial expressions investigated under three tasks in a within-subjects design. Both toolboxes revealed main effects of emotion during the N170 timing and main effects of task during later time points typically associated with the LPP component. Neither toolbox yielded an interaction between the two factors at the group level, nor at the individual level in LIMO, confirming that the neural processing of these two face expressions is largely independent from task demands. Behavioural data revealed main effects of task on reaction time and accuracy, but no influence of expression or an interaction between the two. Expression processing and task demands are discussed in the context of the consistencies and discrepancies between the two toolboxes and existing literature.

Using genetic algorithms to uncover individual differences in how humans represent facial emotion

Emotional facial expressions critically impact social interactions and cognition. However, emotion research to date has generally relied on the assumption that people represent categorical emotions in the same way, using standardized stimulus sets and overlooking important individual differences. To resolve this problem, we developed and tested a task using genetic algorithms to derive assumption-free, participant-generated emotional expressions. One hundred and five participants generated a subjective representation of happy, angry, fearful and sad faces. Population-level consistency was observed for happy faces, but fearful and sad faces showed a high degree of variability. High test-retest reliability was observed across all emotions. A separate group of 108 individuals accurately identified happy and angry faces from the first study, while fearful and sad faces were commonly misidentified. These findings are an important first step towards understanding individual differences in emotion representation, with the potential to reconceptualize the way we study atypical emotion processing in future research.

Facial masks affect emotion recognition in the general population and individuals with autistic traits

Facial expressions, and the ability to recognize these expressions, have evolved in humans to communicate information to one another. Face masks are equipment used in healthcare by health professionals to prevent the transmission of airborne infections. As part of the social distancing efforts related to COVID-19, wearing facial masks has been practiced globally. Such practice might influence affective information communication among humans. Previous research suggests that masks disrupt expression recognition of some emotions (e.g., fear, sadness or neutrality) and lower the confidence in their identification. To extend the previous research, in the current study we tested a larger and more diverse sample of individuals and also investigated the effect of masks on perceived intensity of expressions. Moreover, for the first time in the literature we examined these questions using individuals with autistic traits. Specifically, across three experiments using different populations (college students and general population), and the 10-item Autism Spectrum Quotient (AQ-10; lower and higher scorers), we tested the effect of facial masks on facial emotion recognition of anger, disgust, fear, happiness, sadness, and neutrality. Results showed that the ability to identify all facial expressions decreased when faces were masked, a finding observed across all three studies, contradicting previous research on fear, sad, and neutral expressions. Participants were also less confident in their judgements for all emotions, supporting previous research; and participants perceived emotions as less expressive in the mask condition compared to the unmasked condition, a finding novel to the literature. An additional novel finding was that participants with higher scores on the AQ-10 were less accurate and less confident overall in facial expression recognition, as well as perceiving expressions as less intense. Our findings reveal that wearing face masks decreases facial expression recognition, confidence in expression identification, as well as the perception of intensity for all expressions, affecting high-scoring AQ-10 individuals more than low-scoring individuals.