Facial masks affect emotion recognition in the general population and individuals with autistic traits

Recognizing and Looking at Masked Emotional Faces in Alexithymia

Alexithymia is a clinically relevant personality construct characterized by difficulties identifying and communicating one's emotions and externally oriented thinking. Alexithymia has been found to be related to poor emotion decoding and diminished attention to the eyes. The present eye tracking study investigated whether high levels of alexithymia are related to impairments in recognizing emotions in masked faces and reduced attentional preference for the eyes. An emotion recognition task with happy, fearful, disgusted, and neutral faces with face masks was administered to high-alexithymic and non-alexithymic individuals. Hit rates, latencies of correct responses, and fixation duration on eyes and face mask were analyzed as a function of group and sex. Alexithymia had no effects on accuracy and speed of emotion recognition. However, alexithymic men showed less attentional preference for the eyes relative to the mask than non-alexithymic men, which was due to their increased attention to face masks. No fixation duration differences were observed between alexithymic and non-alexithymic women. Our data indicate that high levels of alexithymia might not have adverse effects on the efficiency of emotion recognition from faces wearing masks. Future research on gaze behavior during facial emotion recognition in high alexithymia should consider sex as a moderating variable.

Duration of face mask exposure matters: evidence from Swiss and Brazilian kindergartners' ability to recognise emotions

Wearing facial masks became a common practice worldwide during the COVID-19 pandemic. This study investigated (1) whether facial masks that cover adult faces affect 4- to 6-year-old children's recognition of emotions in those faces and (2) whether the duration of children's exposure to masks is associated with emotion recognition. We tested children from Switzerland (N = 38) and Brazil (N = 41). Brazil represented longer mask exposure due to a stricter mandate during COVID-19. Children had to choose a face displaying a specific emotion (happy, angry, or sad) when the face wore either no cover, a facial mask, or sunglasses. The longer hours of mask exposure were associated with better emotion recognition. Controlling for the hours of exposure, children were less likely to recognise emotions in partially hideen faces. Moreover, Brazilian children were more accurate in recognising happy faces than Swiss children. Overall, facial masks may negatively impact children's emotion recognition. However, prolonged exposure appears to buffer the lack of facial cues from the nose and mouth. In conclusion, restricting facial cues due to masks may impair kindergarten children's emotion recognition in the short run. However, it may facilitate their broader reading of facial emotional cues in the long run.

Child mask mandates for COVID-19: a systematic review

BACKGROUND

Mask mandates for children during the COVID-19 pandemic varied in different locations. A risk-benefit analysis of this intervention has not yet been performed. In this study, we performed a systematic review to assess research on the effectiveness of mask wearing in children.

METHODS

We performed database searches up to February 2023. The studies were screened by title and abstract, and included studies were further screened as full-text references. A risk-of-bias analysis was performed by two independent reviewers and adjudicated by a third reviewer.

RESULTS

We screened 597 studies and included 22 in the final analysis. There were no randomised controlled trials in children assessing the benefits of mask wearing to reduce SARS-CoV-2 infection or transmission. The six observational studies reporting an association between child masking and lower infection rate or antibody seropositivity had critical (n=5) or serious (n=1) risk of bias; all six were potentially confounded by important differences between masked and unmasked groups and two were shown to have non-significant results when reanalysed. Sixteen other observational studies found no association between mask wearing and infection or transmission.

CONCLUSIONS

Real-world effectiveness of child mask mandates against SARS-CoV-2 transmission or infection has not been demonstrated with high-quality evidence. The current body of scientific data does not support masking children for protection against COVID-19.

Accuracy and Speed of Emotion Recognition With Face Masks

Wearing face masks is one of the important actions to prevent the spread of COVID-19 among people around the world. Nevertheless, social interaction is limited via masks, and this impacts the accuracy and speed of emotional perception. In the present study, we assess the impact of mask-wearing on the accuracy and speed of emotion recognition. Fifty people (female n = 39, male n = 11) aged 19-28 participated in the study (M = 21.1 years). We used frontal photos of a Kosova woman who belonged to the same participants' age group, with a grey background. Twelve different pictures were used that showed the emotional states of fear, joy, sadness, anger, neutrality, and disgust, in masked and unmasked conditions. The experiment was conducted in a controlled laboratory setting. Participants were faster for identifying emotions like joy (1.507 ms) and neutral (1.971 ms). The participants were more accurate (emotions identification) in unmasked faces (M = 85.7%) than in masked faces (M = 73.8%), F(1,98) = 20.73, MSE = 1027.66, p ≤ .001, partial η² = 0.17. Masks make confusion and reduce the accuracy and speediness of emotional detection. This may have a notable impact on social interactions among peoples.

Seeing emotions in the eyes: a validated test to study individual differences in the perception of basic emotions

People are able to perceive emotions in the eyes of others and can therefore see emotions when individuals wear face masks. Research has been hampered by the lack of a good test to measure basic emotions in the eyes. In two studies respectively with 358 and 200 participants, we developed a test to see anger, disgust, fear, happiness, sadness and surprise in images of eyes. Each emotion is measured with 8 stimuli (4 male actors and 4 female actors), matched in terms of difficulty and item discrimination. Participants reliably differed in their performance on the Seeing Emotions in the Eyes test (SEE-48). The test correlated well not only with Reading the Mind in the Eyes Test (RMET) but also with the Situational Test of Emotion Understanding (STEU), indicating that the SEE-48 not only measures low-level perceptual skills but also broader skills of emotion perception and emotional intelligence. The test is freely available for research and clinical purposes.

Effects of wearing an opaque or transparent face mask on the perception of facial expressions: A comparative study between Japanese school-aged children and adults

The negative side effects of mask-wearing on reading facial emotional cues have been investigated in several studies with adults post-2020. However, little is known about children. This study aimed to determine the negative influence of mask-wearing on reading emotions of adult faces by Japanese school-aged children, compared to Japanese adults. We also examined whether this negative influence could be alleviated by using a transparent face mask instead of an opaque one (surgical mask). The performance on reading emotions was measured using emotion categorization and emotion intensity rating tasks for adult faces. As per the findings, the accuracy of emotion recognition in children was impaired for various facial expressions (disgust, fear, happy, neutral, sad, and surprise faces), except for angry faces. Conversely, in adults, it was impaired for a few facial expressions. The perceived intensity for happy faces with a surgical mask was weaker in both children and adults than in those without the mask. A negative influence of wearing surgical masks was generally not observed for faces wearing a transparent mask in both children and adults. Thus, negative side effects of mask-wearing on reading emotions are observed for more facial expressions in children than in adults; transparent masks can help remedy these.

Drift-diffusion modeling reveals that masked faces are preconceived as unfriendly

During the COVID-19 pandemic, the use of face masks has become a daily routine. Studies have shown that face masks increase the ambiguity of facial expressions which not only affects (the development of) emotion recognition, but also interferes with social interaction and judgement. To disambiguate facial expressions, we rely on perceptual (stimulus-driven) as well as preconceptual (top-down) processes. However, it is unknown which of these two mechanisms accounts for the misinterpretation of masked expressions. To investigate this, we asked participants (N = 136) to decide whether ambiguous (morphed) facial expressions, with or without a mask, were perceived as friendly or unfriendly. To test for the independent effects of perceptual and preconceptual biases we fitted a drift-diffusion model (DDM) to the behavioral data of each participant. Results show that face masks induce a clear loss of information leading to a slight perceptual bias towards friendly choices, but also a clear preconceptual bias towards unfriendly choices for masked faces. These results suggest that, although face masks can increase the perceptual friendliness of faces, people have the prior preconception to interpret masked faces as unfriendly.

Self-aware face emotion accelerated recognition algorithm: a novel neural network acceleration algorithm of emotion recognition for international students

With an increasing number of human-computer interaction application scenarios, researchers are looking for computers to recognize human emotions more accurately and efficiently. Such applications are desperately needed at universities, where people want to understand the students' psychology in real time to avoid catastrophes. This research proposed a self-aware face emotion accelerated recognition algorithm (SFEARA) that improves the efficiency of convolutional neural networks (CNNs) in the recognition of facial emotions. SFEARA will recognize that critical and non-critical regions of input data perform high-precision computation and convolutive low-precision computation during the inference process, and finally combine the results, which can help us get the emotional recognition model for international students. Based on a comparison of experimental data, the SFEARA algorithm has 1.3× to 1.6× higher computational efficiency and 30% to 40% lower energy consumption than conventional CNNs in emotion recognition applications, is better suited to the real-time scenario with more background information.

Emotions behind a mask: the value of disgust

The impact of face masks on social cognition and interaction became a popular topic due to the long-lasting COVID-19 pandemic. This theme persists in the focus of attention beyond the pandemic, since face covering not only reduces the overall amount of face information available but also introduces biases and prejudices affecting social perception at large. Many questions are still open. One of them is whether gender of beholders affects inferring of emotions covered by face masks. Reading covered faces may be particularly challenging for individuals with mental disorders, most of which are gender-specific. Previous findings are not only sparse, but inconclusive because most research had been conducted online with resulting samples heavily dominated by females. Here in a face-to-face study, females and males were presented with a randomized set of faces covered by masks. In a two-alternative forced-choice paradigm, participants had to indicate facial emotions displayed by posers. In general, the outcome dovetails with earlier findings that face masks affect emotion recognition in a dissimilar way: Inferring some emotions suffers more severely than others, with the most pronounced influence of mask wearing on disgust and close to ceiling recognition of fear and neutral expressions. Contrary to our expectations, however, males were on overall more proficient in emotion recognition. In particular, males substantially excelled in inferring disgust. The findings help to understand gender differences in recognition of disgust, the forgotten emotion of psychiatry, that is of substantial value for a wide range of mental disorders including schizophrenia. Watch Prof. Marina Pavlova discussing this her work and this article: https://vimeo.com/860126397/5966610f49?share=copy .

Face coverings differentially alter valence judgments of emotional expressions

Face masks that prevent disease transmission obscure facial expressions, impairing nonverbal communication. We assessed the impact of lower (masks) and upper (sunglasses) face coverings on emotional valence judgments of clearly valenced (fearful, happy) and ambiguously valenced (surprised) expressions, the latter of which have both positive and negative meaning. Masks, but not sunglasses, impaired judgments of clearly valenced expressions compared to faces without coverings. Drift diffusion models revealed that lower, but not upper, face coverings slowed evidence accumulation and affected differences in non-judgment processes (i.e., stimulus encoding, response execution time) for all expressions. Our results confirm mask-interference effects in nonverbal communication. The findings have implications for nonverbal and intergroup communication, and we propose guidance for implementing strategies to overcome mask-related interference.

Face masks influence emotion judgments of facial expressions: a drift-diffusion model

Face masks slow the spread of SARS-CoV-2, but it has been unknown how masks might reshape social interaction. One important possibility is that masks may influence how individuals communicate emotion through facial expressions. Here, we clarify to what extent-and how-masks influence facial emotion communication, through drift-diffusion modeling (DDM). Over two independent pre-registered studies, conducted three and 6 months into the COVID-19 pandemic, online participants judged expressions of 6 emotions (anger, disgust, fear, happiness, sadness, surprise) with the lower or upper face "masked" or unmasked. Participants in Study 1 (N = 228) correctly identified expressions above chance with lower face masks. However, they were less likely-and slower-to correctly identify these expressions relative to without masks, and they accumulated evidence for emotion more slowly-via decreased drift rate in DDM. This pattern replicated and intensified 3 months later in Study 2 (N = 264). These findings highlight how effectively individuals still communicate with masks, but also explain why they can experience difficulties communicating when masked. By revealing evidence accumulation as the underlying mechanism, this work suggests that time-sensitive situations may risk miscommunication with masks. This research could inform critical interventions to promote continued mask wearing as needed.

Empathising with masked targets: limited side effects of face masks on empathy for dynamic, context-rich stimuli

Multiple studies revealed detrimental effects of face masks on communication, including reduced empathic accuracy and enhanced listening effort. Yet, extant research relied on artificial, decontextualised stimuli, which prevented assessing empathy under more ecologically valid conditions. In this preregistered online experiment (N = 272), we used film clips featuring targets reporting autobiographical events to address motivational mechanisms underlying face mask effects on cognitive (empathic accuracy) and emotional facets (emotional congruence, sympathy) of empathy. Surprisingly, targets whose faces were covered by a mask (or a black bar) elicited the same level of empathy motives (affiliation, cognitive effort), and accordingly, the same level of cognitive and emotional empathy compared to targets with uncovered faces. We only found a negative direct effect of face coverings on sympathy. Additional analyses revealed that older (compared to young) adults showed higher empathy, but age did not moderate face mask effects. Our findings speak against strong negative face mask effects on empathy when using dynamic, context-rich stimuli, yet support motivational mechanisms of empathy.

Reading the room: Autistic traits, gaze behaviour, and the ability to infer social relationships

Individuals high in autistic traits can have difficulty understanding verbal and non-verbal cues, and may display atypical gaze behaviour during social interactions. The aim of this study was to examine differences among neurotypical individuals with high and low levels of autistic traits with regard to their gaze behaviour and their ability to assess peers' social status accurately. Fifty-four university students who completed the 10-item Autism Quotient (AQ-10) were eye-tracked as they watched six 20-second video clips of people ("targets") involved in a group decision-making task. Simulating natural, everyday social interactions, the video clips included moments of debate, humour, interruptions, and cross talk. Results showed that high-scorers on the AQ-10 (i.e., those with more autistic traits) did not differ from the low-scorers in either gaze behaviour or assessing the targets' relative social status. The results based on this neurotypical group of participants suggest that the ability of individuals high in autistic traits to read social cues may be preserved in certain tasks crucial to navigating day-to-day social relationships. These findings are discussed in terms of their implications for theory of mind, weak central coherence, and social motivation theories of autism.

Social sensitivity predicts accurate emotion inference from facial expressions in a face mask: a study in Japan

Most prior research examining whether emotions can be accurately inferred from facial expressions with masks have been conducted with Western samples. Thus, there is a current lack of studies on this topic, as well as on factors affecting individual differences in the accuracy of emotion inference, among non-Western samples. This study examined the effects of social anxiety and social sensitivity on the accuracy of emotion inference from masked facial expressions in a Japanese sample. The results showed that wearing a mask made it difficult to identify the emotions of sadness and fear, happy and neutral expressions remained unaffected, and angry expressions were read more accurately. Further, while the findings show that a general higher ability to infer emotions from facial expressions may help ensure the accuracy of emotion inference from facial expressions with a mask, social sensitivity directly predicted the accuracy of emotion inference from facial expressions with a mask. These findings suggest that people who can infer complex mental states of others from subtle cues may be less susceptible to the effects of face masks.

Masked Face Emotion Recognition Based on Facial Landmarks and Deep Learning Approaches for Visually Impaired People

Current artificial intelligence systems for determining a person's emotions rely heavily on lip and mouth movement and other facial features such as eyebrows, eyes, and the forehead. Furthermore, low-light images are typically classified incorrectly because of the dark region around the eyes and eyebrows. In this work, we propose a facial emotion recognition method for masked facial images using low-light image enhancement and feature analysis of the upper features of the face with a convolutional neural network. The proposed approach employs the AffectNet image dataset, which includes eight types of facial expressions and 420,299 images. Initially, the facial input image's lower parts are covered behind a synthetic mask. Boundary and regional representation methods are used to indicate the head and upper features of the face. Secondly, we effectively adopt a facial landmark detection method-based feature extraction strategy using the partially covered masked face's features. Finally, the features, the coordinates of the landmarks that have been identified, and the histograms of the oriented gradients are then incorporated into the classification procedure using a convolutional neural network. An experimental evaluation shows that the proposed method surpasses others by achieving an accuracy of 69.3% on the AffectNet dataset.

Dynamic eye avoidance patterns in the high autistic traits group: An eye-tracking study

Introduction

Reduced fixation to the eye area is the main characteristic of social deficits associated with Autism Spectrum Disorder; a similar pattern may exist in individuals with high autistic traits. However, their scanning patterns to the eye area of emotional faces are still unclear on the time scale.

Methods

In the present study, we recruited 46 participants and divided them into the high autistic traits (HAT) group (23 participants) and the low autistic traits (LAT) group (20 participants) based on their Autism Spectrum Quotient (AQ) scores. Moreover, we captured their eye movement patterns when observing different angular emotional faces. We extracted the proportional fixation time to the eye area under different time windows.

Results

The results showed that the fixation time of the HAT group was always significantly smaller than that of the LAT group (p < 0.05), and the difference between the two groups increased in the middle and late stages of face presentation. The results of the linear regression analysis showed that the proportional fixation time was negatively correlated with AQ scores (p < 0.05), indicating that the proportional fixation time to the eye area could be a potential indicator to measure the level of autistic traits. We then calculated the latency to orient the eye area and the latency to disengage the eye area to explore the priority of observation of the eyes. The results showed that compared with the LAT group, the HAT group has a longer latency to orient the eye area (p < 0.05) and has longer latency to disengage the eye area (p < 0.05), illustrating that the HAT group saw the eyes more slowly and left them faster.

Face memory and facial expression recognition are both affected by wearing disposable surgical face masks

Faces are fundamental stimuli for social interactions since they provide significant information about people's identity and emotional states. With the outburst of the COVID-19 pandemic, global use of preventive measures, such as disposable surgical face masks (DSFMs), has been imposed. The massive use of DSFMs covering a large part of the face could interfere with identity and emotion recognition. Thus, the main aim of the current study was (i) to assess how DSFMs affect identity recognition (Experiment 1), (ii) how DSFMs affect emotion recognition (Experiment 2), and (iii) whether individual empathy levels correlate with emotion recognition with DSFMs. The potential relation between identity and emotion recognition with and without DSFMs was also investigated. Two tasks were administered to 101 healthy participants: (i) the Old-new face memory task aimed to assess whether the learning context (i.e., DSFMs on/off) affects recognition performance, whereas (ii) the Facial affect task explored DSFMs' effect on emotion recognition. Results from the former showed that the stimuli's features in the learning stage affect recognition performances; that is, faces wearing DSFMs were better recognized if wearing DSFMs at first exposure and vice versa. Results from the Facial affect task showed that DSFMs lead to reduced disgust, happiness, and sadness recognition. No significant correlation emerged between identity and emotion recognition. The Interpersonal Reactivity Index (IRI) was administered to assess affective and cognitive empathy; however, IRI scores did not correlate with either face memory recognition or facial affect recognition. Overall, our results demonstrate (a) a "context effect" for face memory with and without DSFMs; (b) a disruptive effect of DSFMs depending on the expressed emotion; and (c) no correlation between empathy and emotion recognition with DSFMs.

The effect of masks on the recognition of facial expressions: A true-to-life study on the perception of basic emotions

Mouth-to-nose face masks became ubiquitous due to the COVID-19 pandemic. This ignited studies on the perception of emotions in masked faces. Most of these studies presented still images of an emotional face with a face mask digitally superimposed upon the nose-mouth region. A common finding of these studies is that smiles become less perceivable. The present study investigated the recognition of basic emotions in video sequences of faces. We replicated much of the evidence gathered from presenting still images with digitally superimposed masks. We also unearthed fundamental differences in comparison to existing studies with regard to the perception of smile which is less impeded than previous studies implied.

Face-mask-aware Facial Expression Recognition based on Face Parsing and Vision Transformer

As wearing face masks is becoming an embedded practice due to the COVID-19 pandemic, facial expression recognition (FER) that takes face masks into account is now a problem that needs to be solved. In this paper, we propose a face parsing and vision Transformer-based method to improve the accuracy of face-mask-aware FER. First, in order to improve the precision of distinguishing the unobstructed facial region as well as those parts of the face covered by a mask, we re-train a face-mask-aware face parsing model, based on the existing face parsing dataset automatically relabeled with a face mask and pixel label. Second, we propose a vision Transformer with a cross attention mechanism-based FER classifier, capable of taking both occluded and non-occluded facial regions into account and reweigh these two parts automatically to get the best facial expression recognition performance. The proposed method outperforms existing state-of-the-art face-mask-aware FER methods, as well as other occlusion-aware FER methods, on two datasets that contain three kinds of emotions (M-LFW-FER and M-KDDI-FER datasets) and two datasets that contain seven kinds of emotions (M-FER-2013 and M-CK+ datasets).

Perceptions of persons who wear face coverings are modulated by the perceivers' attitude

We examined if the effect of facial coverings on person perception is influenced by the perceiver's attitudes. We used two online experiments in which participants saw the same human target persons repeatedly appearing with and without a specific piece of clothing and had to judge the target persons' character. In Experiment 1 (N = 101), we investigated how the wearing of a facial mask influences a person's perception depending on the perceiver's attitude toward measures against the COVID-19 pandemic. In Experiment 2 (N = 114), we examined the effect of wearing a head cover associated with Arabic culture on a person's perception depending on the perceiver's attitude toward Islam. Both studies were preregistered; both found evidence that a person's perception is a process shaped by the personal attitudes of the perceiver as well as merely the target person's outward appearance. Integrating previous findings, we demonstrate that facial covers, as well as head covers, operate as cues which are used by the perceivers to infer the target persons' underlying attitudes. The judgment of the target person is shaped by the perceived attitude toward what the facial covering stereotypically symbolizes.

Face mask use as a categorical dimension in social perception

Prevention measures aimed at combating COVID-19 pandemic strongly impact several aspects of social life. In particular, interpersonal perception is affected as a function of whether the persons perceived wear or not face masks. In two experimental studies, we here explored whether people rely on the presence vs. absence of face masks when encoding information in memory about other individuals. In a memory confusion paradigm, participants were initially presented with individuals either wearing a face mask or not, each conveying a series of sentences. Next, participants were probed about the identity of the speaker of each sentence. Results showed that it was more likely to erroneously attribute a sentence to a speaker who also was wearing a face mask (or not) as the original speaker, demonstrating that the cue about wearing or not a face mask was spontaneously used to encode information. Study 2 ruled out an alternative explanation based on perceptual processes, suggesting that face masks represent meaningful social objects. Overall, it emerged that participants spontaneously categorize others as a function of whether they wear a mask or not. Findings also confirmed previous research evidence about the more positive evaluation of mask wearers as compared to non-wearers, and the overall detrimental impact that face masks may have on the correct identification of social targets.

I looked at you, you looked at me, I smiled at you, you smiled at me—The impact of eye contact on emotional mimicry

Eye contact is an essential element of human interaction and direct eye gaze has been shown to have effects on a range of attentional and cognitive processes. Specifically, direct eye contact evokes a positive affective reaction. As such, it has been proposed that obstructed eye contact reduces emotional mimicry (i.e., the imitation of our counterpart’s emotions). So far, emotional mimicry research has used averted-gaze faces or unnaturally covered eyes (with black censor bars) to analyze the effect of eye contact on emotional mimicry. However, averted gaze can also signal disinterest/ disengagement and censor bars obscure eye-adjacent areas as well and hence impede emotion recognition. In the present study (N = 44), we used a more ecological valid approach by showing photos of actors who expressed either happiness, sadness, anger, or disgust while either wearing mirroring sunglasses that obstruct eye contact or clear glasses. The glasses covered only the direct eye region but not the brows, nose ridge, and cheeks. Our results confirm that participants were equally accurate in recognizing the emotions of their counterparts in both conditions (sunglasses vs. glasses). Further, in line with our hypotheses, participants felt closer to the targets and mimicked affiliative emotions more intensely when their counterparts wore glasses instead of sunglasses. For antagonistic emotions, we found the opposite pattern: Disgust mimicry, which was interpreted as an affective reaction rather than genuine mimicry, could be only found in the sunglasses condition. It may be that obstructed eye contact increased the negative impression of disgusted facial expressions and hence the negative feelings disgust faces evoked. The present study provides further evidence for the notion that eye contact is an important prerequisite for emotional mimicry and hence for smooth and satisfying social interactions.

COVID-19 masks: A barrier to facial and vocal information

With the COVID-19 pandemic, we have become used to wearing masks and have experienced how masks seem to impair emotion and speech recognition. While several studies have focused on facial emotion recognition by adding images of masks on photographs of emotional faces, we have created a video database with actors really wearing masks to test its effect in more ecological conditions. After validating the emotions displayed by the actors, we found that surgical mask impaired happiness and sadness recognition but not neutrality. Moreover, for happiness, this effect was specific to the mask and not to covering the lower part of the face, possibly due to a cognitive bias associated with the surgical mask. We also created videos with speech and tested the effect of mask on emotion and speech recognition when displayed in auditory, visual, or audiovisual modalities. In visual and audiovisual modalities, mask impaired happiness and sadness but improved neutrality recognition. Mask impaired the recognition of bilabial syllables regardless of modality. In addition, it altered speech recognition only in the audiovisual modality for participants above 70 years old. Overall, COVID-19 masks mainly impair emotion recognition, except for older participants for whom it also impacts speech recognition, probably because they rely more on visual information to compensate age-related hearing loss.

How facial masks alter the interaction of gaze direction, head orientation, and emotion recognition

The COVID-19 pandemic has altered the way we interact with each other: mandatory mask-wearing obscures facial information that is crucial for emotion recognition. Whereas the influence of wearing a mask on emotion recognition has been repeatedly investigated, little is known about the impact on interaction effects among emotional signals and other social signals. Therefore, the current study sought to explore how gaze direction, head orientation, and emotional expression interact with respect to emotion perception, and how these interactions are altered by wearing a face mask. In two online experiments, we presented face stimuli from the Radboud Faces Database displaying different facial expressions (anger, fear, happiness, neutral, and sadness), gaze directions (−13°, 0°, and 13°), and head orientations (−45°, 0°, and 45°) – either without (Experiment 1) or with mask (Experiment 2). Participants categorized the displayed emotional expressions. Not surprisingly, masks impaired emotion recognition. Surprisingly, without the mask, emotion recognition was unaffected by averted head orientations and only slightly affected by gaze direction. The mask strongly interfered with this ability. The mask increased the influence of head orientation and gaze direction, in particular for the emotions that were poorly recognized with mask. The results suggest that in case of uncertainty due to ambiguity or absence of signals, we seem to unconsciously factor in extraneous information.

Autistic children and adolescents with frequent restricted interest and repetitive behavior showed more difficulty in social cognition during mask-wearing during the COVID-19 pandemic: a multisite survey

BACKGROUND

The public health measures enacted in order to control the coronavirus disease (COVID-19) pandemic have caused considerable changes to daily life. For autistic children and adolescents, adapting to the "new normal," including mask-wearing, may be difficult because of their restricted interest and repetitive behavior (RRB) characteristics. We aimed to examine the relationships between RRB characteristics and the impact of mask-wearing on their social communications during the pandemic.

METHODS

We recruited participants with a clinical diagnosis of autism spectrum disorder based on DSM-5 diagnostic criteria from two outpatient clinics in Tokyo, Japan, between November 2020 and April 2021 using a convenience sampling methodology. As a result, the participants consisted of 102 children and adolescents (mean (SD) age = 11.6 (5.3)). We collected data on RRB characteristics frequency before and during the pandemic using the CoRonavIruS Health Impact Survey (CRISIS) - Adapted for Autism and Related Neurodevelopmental conditions (AFAR). We then conducted factor analyses to compute the RRB severity composite scores, which are divided into lower- (e.g., sensory seeking), and higher-order (e.g., restricted interest). We also investigated mask-wearing culture using a bespoke questionnaire, and using Spearman's rank correlation analyses, we examined the relationships between before pandemic RRB characteristics, and the impact of mask-wearing on social communications during the pandemic.

RESULTS

We found that children and adolescents who exhibited lower-order RRB before the pandemic had difficulties in going-out with mask-wearing (rho = -0.25, q = .031), more challenges with mask-wearing (rho =  - 0.34, q = .0018), and difficulty in referring to others' emotions while wearing masks (rho =  - 0.36, q = .0016). We also found an association between higher-order RRB before the pandemic and an uncomfortable sensation (rho =  - 0.42, q = .0002) and difficulties in referring to other's emotions while wearing masks (rho =  - 0.25, q = .031).

CONCLUSIONS

We revealed that various behaviors, such as sensory seeking, repetitive motor mannerisms and movements, and rituals and routines, undertaken before the pandemic could be important predictors of difficulties with mask-wearing and social communication for autistic children and adolescents during the pandemic. Caregivers and teachers wearing masks may need to provide extra support for social communication to autistic children and adolescents showing RRB characteristics frequently.

Mapping the perception-space of facial expressions in the era of face masks

With the advent of the severe acute respiratory syndrome-Corona Virus type 2 (SARS-CoV-2) pandemic, the theme of emotion recognition from facial expressions has become highly relevant due to the widespread use of face masks as one of the main devices imposed to counter the spread of the virus. Unsurprisingly, several studies published in the last 2 years have shown that accuracy in the recognition of basic emotions expressed by faces wearing masks is reduced. However, less is known about the impact that wearing face masks has on the ability to recognize emotions from subtle expressions. Furthermore, even less is known regarding the role of interindividual differences (such as alexithymic and autistic traits) in emotion processing. This study investigated the perception of all the six basic emotions (anger, disgust, fear, happiness, sadness, and surprise), both as a function of the face mask and as a function of the facial expressions’ intensity (full vs. subtle) in terms of participants’ uncertainty in their responses, misattribution errors, and perceived intensity. The experiment was conducted online on a large sample of participants (N = 129). Participants completed the 20-item Toronto Alexithymia Scale and the Autistic Spectrum Quotient and then performed an emotion-recognition task that involved face stimuli wearing a mask or not, and displaying full or subtle expressions. Each face stimulus was presented alongside the Geneva Emotion Wheel (GEW), and participants had to indicate what emotion they believed the other person was feeling and its intensity using the GEW. For each combination of our variables, we computed the indices of ‘uncertainty’ (i.e., the spread of responses around the correct emotion category), ‘bias’ (i.e., the systematic errors in recognition), and ‘perceived intensity’ (i.e., the distance from the center of the GEW). We found that face masks increase uncertainty for all facial expressions of emotion, except for fear when intense, and that disgust was systematically confused with anger (i.e., response bias). Furthermore, when faces were covered by the mask, all the emotions were perceived as less intense, and this was particularly evident for subtle expressions. Finally, we did not find any evidence of a relationship between these indices and alexithymic/autistic traits.

Face Mask Reduces the Effect of Proposer’s (Un)Trustworthiness on Intertemporal and Risky Choices

Previous literature suggested that individuals increase temporal and risk discounting at the presence of a proposer whose face is perceived as untrustworthy, suggesting the activation of protective choice patterns. By the way, the COVID-19 pandemic has substantially transformed the way we interact with other people, even bringing us into situations where the face of the person making a proposal is not fully visible, because of the mask. With the current study, we aimed at verifying if the effect of proposer’s facial (un)trustworthiness on discounting behavior is modulated by mask wearing. In two different experiments, participants performed traditional delay and probability discounting tasks with masked proposers manipulated across trustworthiness levels. Results highlighted that, even after checking for subject-specific emotion recognition ability with masked faces, the presence of a masked untrustworthy proposer increases both delay and probability discounting parameters, although the effect is not statistically significant and smaller than the one detected at the presence of an untrustworthy proposer without a mask. These results suggest that the ability to perceive the proposer’s (un)trustworthiness is affected by the mask, with a consequent less strong effect of proposer’s (un)trustworthiness on choice behavior on both intertemporal and risky choices. Limits and possible implications are outlined and discussed.

Effects of wearing a transparent face mask on perception of facial expressions

Wearing face masks in public has become the norm in many countries post-2020. Although mask-wearing is effective in controlling infection, it has the negative side effect of occluding the mask wearer’s facial expressions. The purpose of this study was to investigate the effects of wearing transparent masks on the perception of facial expressions. Participants were required to categorize the perceived facial emotion of female (Experiment 1) and male (Experiment 2) faces with different facial expressions and to rate the perceived emotion intensity of the faces. Based on the group, the participants were assigned to, the faces were presented with a surgical mask, a transparent mask, or without a mask. The results showed that wearing a surgical mask impaired the performance of reading facial expressions, both with respect to recognition and perceived intensity of facial emotions. Specifically, the impairments were robustly observed in fear and happy faces for emotion recognition, and in happy faces for perceived intensity of emotion in Experiments 1 and 2. However, the impairments were moderated by wearing a transparent mask instead of a surgical mask. During the coronavirus disease 2019 (COVID-19) pandemic, the transparent mask can be used in a range of situations where face-to-face communication is important.

Masked emotions: Do face mask patterns and colors affect the recognition of emotions?

Previous research has shown that face masks impair the ability to perceive social information and the readability of emotions. These studies mostly explored the effect of standard medical, often white, masks on emotion recognition. However, in reality, many individuals prefer masks with different styles. We investigated whether the appearance of the mask (pattern: angular vs. curvy and color: black vs. white) affected the recognition of emotional states. Participants were asked to identify the emotions on faces covered by masks with different designs. The presence of masks resulted in decreasing accuracy and confidence and increasing reaction times, indicating that masks impair emotion recognition. There were no significant effects of angularity versus curvature or color on emotion recognition, which suggests that mask design may not impair the recognition beyond the effect of mere mask wearing. Besides, we found relationships between individual difference variables such as mask wearing attitudes, mask design preferences, individual traits and emotion recognition. The majority of participants demonstrated positive attitudes toward mask wearing and preferred non-patterned black and white masks. Preferences for white masks were associated with better emotion recognition of masked faces. In contrast, those with negative attitudes toward masks showed marginally poorer performance in emotion recognition for masked faces, and preferred patterned more than plain masks, perhaps viewing masks as a fashion item rather than a necessity. Moreover, preferences to wear patterned masks were negatively related to actual wearing of masks indoors and perceived risks of COVID.

Empathy through the Pandemic: Changes of Different Emphatic Dimensions during the COVID-19 Outbreak

Growing evidence suggests that empathy is a relevant psychological trait to face the challenges imposed by the COVID-19 pandemic, but at the present very little is known on whether this multi-dimensional construct has been affected by the pandemic outbreak differently in its separate components. Here, we aimed at filling this gap by capitalizing on the opportunity of having collected data from different self-report measures and cognitive tasks assessing the main dimensions of empathy immediately before the beginning of the global pandemic and about one year later. The results showed a detrimental impact of the pandemic outbreak on empathic social skills but not on both cognitive (perspective-taking) and emotional empathy that instead significantly improved. Thus, reduced empathic social skills could be a weakness to be targeted in psychological interventions to help people cope with the mental health challenges related to COVID-19 pandemic, whereas the ability of understanding another's mental states and emotions could represent a strength in dealing with the current long-lasting crisis.

Deep Neural Network Approach for Pose, Illumination, and Occlusion Invariant Driver Emotion Detection

Monitoring drivers’ emotions is the key aspect of designing advanced driver assistance systems (ADAS) in intelligent vehicles. To ensure safety and track the possibility of vehicles’ road accidents, emotional monitoring will play a key role in justifying the mental status of the driver while driving the vehicle. However, the pose variations, illumination conditions, and occlusions are the factors that affect the detection of driver emotions from proper monitoring. To overcome these challenges, two novel approaches using machine learning methods and deep neural networks are proposed to monitor various drivers’ expressions in different pose variations, illuminations, and occlusions. We obtained the remarkable accuracy of 93.41%, 83.68%, 98.47%, and 98.18% for CK+, FER 2013, KDEF, and KMU-FED datasets, respectively, for the first approach and improved accuracy of 96.15%, 84.58%, 99.18%, and 99.09% for CK+, FER 2013, KDEF, and KMU-FED datasets respectively in the second approach, compared to the existing state-of-the-art methods.

Face masks have emotion-dependent dissociable effects on accuracy and confidence in identifying facial expressions of emotion

The coronavirus pandemic has resulted in increased use of face masks worldwide. Here, we examined the effect of wearing a face mask on the ability to recognise facial expressions of emotion. In a within-subjects design, 100 UK-based undergraduate students were shown facial expressions of anger, disgust, fear, happiness, sadness, and neutral expression; these were either posed with or without a face mask, or with a face mask artificially imposed onto them. Participants identified the emotion portrayed in the photographs from a fixed choice array of answers and rated their confidence in their selection. While overall accuracy was higher without than with masks, the effect varied across emotions, with a clear advantage without masks in disgust, happiness, and sadness; no effect for neutral, and lower accuracy without masks for anger and fear. In contrast, confidence was generally higher without masks, with the effect clear for all emotions other than anger. These results confirm that emotion recognition is affected by face mask wearing, but reveal that the effect depends on the emotion being displayed—with this emotion-dependence not reflected in subjects’ confidence. The disparity between the effects of mask wearing on different emotions and the failure of this to be reflected in confidence ratings suggests that mask wearing not only effects emotion recognition, but may also create biases in the perception of facial expressions of emotion of which perceivers are unaware. In addition, the similarity of results between the Imposed Mask and Posed Mask conditions suggests that prior research using artificially imposed masks has not been deleteriously affected by the use of this manipulation.

Unattractive faces are more attractive when the bottom-half is masked, an effect that reverses when the top-half is concealed

Facial attractiveness in humans signals an individual's genetic condition, underlying physiology and health status, serving as a cue to one's mate value. The practice of wearing face masks for prevention of transmission of airborne infections may disrupt one's ability to evaluate facial attractiveness, and with it, cues to an individual's health and genetic condition. The current research investigated the effect of face masks on the perception of face attractiveness. Across four studies, we tested if below- and above-average attractive full faces are equally affected by wearing facial masks. The results reveal that for young faces (Study 1) and old faces (Study 2) a facial mask increases the perceived attractiveness of relatively unattractive faces, but there is no effect of wearing a face mask for highly attractive faces. Study 3 shows that the same pattern of ratings emerged when the bottom-half of the faces are cropped rather than masked, indicating that the effect is not mask-specific. Our final Study 4, in which information from only the lower half of the faces was made available, showed that contrary to our previous findings, highly attractive half-faces are perceived to be less attractive than their full-face counterpart; but there is no such effect for the less attractive faces. This demonstrates the importance of the eye-region in the perception of attractiveness, especially for highly attractive faces. Collectively these findings suggest that a positivity-bias enhances the perception of unattractive faces when only the upper face is visible, a finding that may not extend to attractive faces because of the perceptual weight placed on their eye-region.

Human face and gaze perception is highly context specific and involves bottom-up and top-down neural processing

This review summarizes human perception and processing of face and gaze signals. Face and gaze signals are important means of non-verbal social communication. The review highlights that: (1) some evidence is available suggesting that the perception and processing of facial information starts in the prenatal period; (2) the perception and processing of face identity, expression and gaze direction is highly context specific, the effect of race and culture being a case in point. Culture affects by means of experiential shaping and social categorization the way in which information on face and gaze is collected and perceived; (3) face and gaze processing occurs in the so-called 'social brain'. Accumulating evidence suggests that the processing of facial identity, facial emotional expression and gaze involves two parallel and interacting pathways: a fast and crude subcortical route and a slower cortical pathway. The flow of information is bi-directional and includes bottom-up and top-down processing. The cortical networks particularly include the fusiform gyrus, superior temporal sulcus (STS), intraparietal sulcus, temporoparietal junction and medial prefrontal cortex.