1
|
Pavlova MA, Moosavi J, Carbon CC, Fallgatter AJ, Sokolov AN. Emotions behind a mask: the value of disgust. SCHIZOPHRENIA (HEIDELBERG, GERMANY) 2023; 9:58. [PMID: 37709796 PMCID: PMC10502067 DOI: 10.1038/s41537-023-00388-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Accepted: 08/15/2023] [Indexed: 09/16/2023]
Abstract
The impact of face masks on social cognition and interaction became a popular topic due to the long-lasting COVID-19 pandemic. This theme persists in the focus of attention beyond the pandemic, since face covering not only reduces the overall amount of face information available but also introduces biases and prejudices affecting social perception at large. Many questions are still open. One of them is whether gender of beholders affects inferring of emotions covered by face masks. Reading covered faces may be particularly challenging for individuals with mental disorders, most of which are gender-specific. Previous findings are not only sparse, but inconclusive because most research had been conducted online with resulting samples heavily dominated by females. Here in a face-to-face study, females and males were presented with a randomized set of faces covered by masks. In a two-alternative forced-choice paradigm, participants had to indicate facial emotions displayed by posers. In general, the outcome dovetails with earlier findings that face masks affect emotion recognition in a dissimilar way: Inferring some emotions suffers more severely than others, with the most pronounced influence of mask wearing on disgust and close to ceiling recognition of fear and neutral expressions. Contrary to our expectations, however, males were on overall more proficient in emotion recognition. In particular, males substantially excelled in inferring disgust. The findings help to understand gender differences in recognition of disgust, the forgotten emotion of psychiatry, that is of substantial value for a wide range of mental disorders including schizophrenia. Watch Prof. Marina Pavlova discussing this her work and this article: https://vimeo.com/860126397/5966610f49?share=copy .
Collapse
Affiliation(s)
- Marina A Pavlova
- Department of Psychiatry and Psychotherapy, Tübingen Center for Mental Health (TüCMH), Medical School and University Hospital, Eberhard Karls University of Tübingen, Tübingen, Germany.
| | - Jonas Moosavi
- Department of Psychiatry and Psychotherapy, Tübingen Center for Mental Health (TüCMH), Medical School and University Hospital, Eberhard Karls University of Tübingen, Tübingen, Germany
| | - Claus-Christian Carbon
- Department of General Psychology and Methodology, University of Bamberg, Bamberg, Germany
| | - Andreas J Fallgatter
- Department of Psychiatry and Psychotherapy, Tübingen Center for Mental Health (TüCMH), Medical School and University Hospital, Eberhard Karls University of Tübingen, Tübingen, Germany
| | - Alexander N Sokolov
- Department of Psychiatry and Psychotherapy, Tübingen Center for Mental Health (TüCMH), Medical School and University Hospital, Eberhard Karls University of Tübingen, Tübingen, Germany
| |
Collapse
|
2
|
Harp NR, Langbehn AT, Larsen JT, Niedenthal PM, Neta M. Face coverings differentially alter valence judgments of emotional expressions. BASIC AND APPLIED SOCIAL PSYCHOLOGY 2023; 45:91-106. [PMID: 37469671 PMCID: PMC10353716 DOI: 10.1080/01973533.2023.2221360] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/21/2023]
Abstract
Face masks that prevent disease transmission obscure facial expressions, impairing nonverbal communication. We assessed the impact of lower (masks) and upper (sunglasses) face coverings on emotional valence judgments of clearly valenced (fearful, happy) and ambiguously valenced (surprised) expressions, the latter of which have both positive and negative meaning. Masks, but not sunglasses, impaired judgments of clearly valenced expressions compared to faces without coverings. Drift diffusion models revealed that lower, but not upper, face coverings slowed evidence accumulation and affected differences in non-judgment processes (i.e., stimulus encoding, response execution time) for all expressions. Our results confirm mask-interference effects in nonverbal communication. The findings have implications for nonverbal and intergroup communication, and we propose guidance for implementing strategies to overcome mask-related interference.
Collapse
Affiliation(s)
| | - Andrew T. Langbehn
- Department of Psychology, University of Tennessee-Knoxville, Knoxville, TN, USA
| | - Jeff T. Larsen
- Department of Psychology, University of Tennessee-Knoxville, Knoxville, TN, USA
| | | | - Maital Neta
- Department of Psychology, University of Nebraska-Lincoln, Lincoln, NE, USA
- Center for Brain, Biology, and Behavior, University of Nebraska-Lincoln, Lincoln, NE, USA
| |
Collapse
|
3
|
Yang B, Wu J, Ikeda K, Hattori G, Sugano M, Iwasawa Y, Matsuo Y. Face-mask-aware Facial Expression Recognition based on Face Parsing and Vision Transformer. Pattern Recognit Lett 2022; 164:173-182. [PMID: 36407855 PMCID: PMC9645067 DOI: 10.1016/j.patrec.2022.11.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2022] [Revised: 10/05/2022] [Accepted: 11/04/2022] [Indexed: 11/11/2022]
Abstract
As wearing face masks is becoming an embedded practice due to the COVID-19 pandemic, facial expression recognition (FER) that takes face masks into account is now a problem that needs to be solved. In this paper, we propose a face parsing and vision Transformer-based method to improve the accuracy of face-mask-aware FER. First, in order to improve the precision of distinguishing the unobstructed facial region as well as those parts of the face covered by a mask, we re-train a face-mask-aware face parsing model, based on the existing face parsing dataset automatically relabeled with a face mask and pixel label. Second, we propose a vision Transformer with a cross attention mechanism-based FER classifier, capable of taking both occluded and non-occluded facial regions into account and reweigh these two parts automatically to get the best facial expression recognition performance. The proposed method outperforms existing state-of-the-art face-mask-aware FER methods, as well as other occlusion-aware FER methods, on two datasets that contain three kinds of emotions (M-LFW-FER and M-KDDI-FER datasets) and two datasets that contain seven kinds of emotions (M-FER-2013 and M-CK+ datasets).
Collapse
Affiliation(s)
- Bo Yang
- KDDI Research, Inc., 2-1-15 Ohara, Fujimino-shi, Saitama, 356–8502, Japan,The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113–8654 Japan,Corresponding author
| | - Jianming Wu
- KDDI Research, Inc., 2-1-15 Ohara, Fujimino-shi, Saitama, 356–8502, Japan
| | - Kazushi Ikeda
- KDDI Research, Inc., 2-1-15 Ohara, Fujimino-shi, Saitama, 356–8502, Japan
| | - Gen Hattori
- KDDI Research, Inc., 2-1-15 Ohara, Fujimino-shi, Saitama, 356–8502, Japan
| | - Masaru Sugano
- KDDI Research, Inc., 2-1-15 Ohara, Fujimino-shi, Saitama, 356–8502, Japan
| | - Yusuke Iwasawa
- The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113–8654 Japan
| | - Yutaka Matsuo
- The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113–8654 Japan
| |
Collapse
|
4
|
Verroca A, de Rienzo CM, Gambarota F, Sessa P. Mapping the perception-space of facial expressions in the era of face masks. Front Psychol 2022; 13:956832. [PMID: 36176786 PMCID: PMC9514388 DOI: 10.3389/fpsyg.2022.956832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 08/05/2022] [Indexed: 11/28/2022] Open
Abstract
With the advent of the severe acute respiratory syndrome-Corona Virus type 2 (SARS-CoV-2) pandemic, the theme of emotion recognition from facial expressions has become highly relevant due to the widespread use of face masks as one of the main devices imposed to counter the spread of the virus. Unsurprisingly, several studies published in the last 2 years have shown that accuracy in the recognition of basic emotions expressed by faces wearing masks is reduced. However, less is known about the impact that wearing face masks has on the ability to recognize emotions from subtle expressions. Furthermore, even less is known regarding the role of interindividual differences (such as alexithymic and autistic traits) in emotion processing. This study investigated the perception of all the six basic emotions (anger, disgust, fear, happiness, sadness, and surprise), both as a function of the face mask and as a function of the facial expressions’ intensity (full vs. subtle) in terms of participants’ uncertainty in their responses, misattribution errors, and perceived intensity. The experiment was conducted online on a large sample of participants (N = 129). Participants completed the 20-item Toronto Alexithymia Scale and the Autistic Spectrum Quotient and then performed an emotion-recognition task that involved face stimuli wearing a mask or not, and displaying full or subtle expressions. Each face stimulus was presented alongside the Geneva Emotion Wheel (GEW), and participants had to indicate what emotion they believed the other person was feeling and its intensity using the GEW. For each combination of our variables, we computed the indices of ‘uncertainty’ (i.e., the spread of responses around the correct emotion category), ‘bias’ (i.e., the systematic errors in recognition), and ‘perceived intensity’ (i.e., the distance from the center of the GEW). We found that face masks increase uncertainty for all facial expressions of emotion, except for fear when intense, and that disgust was systematically confused with anger (i.e., response bias). Furthermore, when faces were covered by the mask, all the emotions were perceived as less intense, and this was particularly evident for subtle expressions. Finally, we did not find any evidence of a relationship between these indices and alexithymic/autistic traits.
Collapse
Affiliation(s)
- Alessia Verroca
- Department of Developmental and Social Psychology, University of Padova, Padova, Italy
| | | | - Filippo Gambarota
- Department of Developmental and Social Psychology, University of Padova, Padova, Italy
- Padova Neuroscience Center (PNC), University of Padova, Padova, Italy
| | - Paola Sessa
- Department of Developmental and Social Psychology, University of Padova, Padova, Italy
- Padova Neuroscience Center (PNC), University of Padova, Padova, Italy
- *Correspondence: Paola Sessa,
| |
Collapse
|
5
|
The Effect of Surgical Masks on the Featural and Configural Processing of Emotions. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19042420. [PMID: 35206620 PMCID: PMC8872142 DOI: 10.3390/ijerph19042420] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/28/2022] [Revised: 02/15/2022] [Accepted: 02/17/2022] [Indexed: 12/27/2022]
Abstract
From the start of the COVID-19 pandemic, the use of surgical masks became widespread. However, they occlude an important part of the face and make it difficult to decode and interpret other people's emotions. To clarify the effect of surgical masks on configural and featural processing, participants completed a facial emotion recognition task to discriminate between happy, sad, angry, and neutral faces. Stimuli included fully visible faces, masked faces, and a cropped photo of the eyes or mouth region. Occlusion due to the surgical mask affects emotion recognition for sadness, anger, and neutral faces, although no significative differences were found in happiness recognition. Our findings suggest that happiness is recognized predominantly via featural processing.
Collapse
|
6
|
Deep Neural Network Approach for Pose, Illumination, and Occlusion Invariant Driver Emotion Detection. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19042352. [PMID: 35206540 PMCID: PMC8871818 DOI: 10.3390/ijerph19042352] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Revised: 02/02/2022] [Accepted: 02/15/2022] [Indexed: 11/24/2022]
Abstract
Monitoring drivers’ emotions is the key aspect of designing advanced driver assistance systems (ADAS) in intelligent vehicles. To ensure safety and track the possibility of vehicles’ road accidents, emotional monitoring will play a key role in justifying the mental status of the driver while driving the vehicle. However, the pose variations, illumination conditions, and occlusions are the factors that affect the detection of driver emotions from proper monitoring. To overcome these challenges, two novel approaches using machine learning methods and deep neural networks are proposed to monitor various drivers’ expressions in different pose variations, illuminations, and occlusions. We obtained the remarkable accuracy of 93.41%, 83.68%, 98.47%, and 98.18% for CK+, FER 2013, KDEF, and KMU-FED datasets, respectively, for the first approach and improved accuracy of 96.15%, 84.58%, 99.18%, and 99.09% for CK+, FER 2013, KDEF, and KMU-FED datasets respectively in the second approach, compared to the existing state-of-the-art methods.
Collapse
|