1
|
Mulder MJ, Prummer F, Terburg D, Kenemans JL. Drift-diffusion modeling reveals that masked faces are preconceived as unfriendly. Sci Rep 2023; 13:16982. [PMID: 37813970 PMCID: PMC10562405 DOI: 10.1038/s41598-023-44162-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Accepted: 10/04/2023] [Indexed: 10/11/2023] Open
Abstract
During the COVID-19 pandemic, the use of face masks has become a daily routine. Studies have shown that face masks increase the ambiguity of facial expressions which not only affects (the development of) emotion recognition, but also interferes with social interaction and judgement. To disambiguate facial expressions, we rely on perceptual (stimulus-driven) as well as preconceptual (top-down) processes. However, it is unknown which of these two mechanisms accounts for the misinterpretation of masked expressions. To investigate this, we asked participants (N = 136) to decide whether ambiguous (morphed) facial expressions, with or without a mask, were perceived as friendly or unfriendly. To test for the independent effects of perceptual and preconceptual biases we fitted a drift-diffusion model (DDM) to the behavioral data of each participant. Results show that face masks induce a clear loss of information leading to a slight perceptual bias towards friendly choices, but also a clear preconceptual bias towards unfriendly choices for masked faces. These results suggest that, although face masks can increase the perceptual friendliness of faces, people have the prior preconception to interpret masked faces as unfriendly.
Collapse
Affiliation(s)
- Martijn J Mulder
- Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands.
| | - Franziska Prummer
- School of Computing and Communications, Lancaster University, Lancaster, UK
| | - David Terburg
- Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| | - J Leon Kenemans
- Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
2
|
Todd E, Subendran S, Wright G, Guo K. Emotion category-modulated interpretation bias in perceiving ambiguous facial expressions. Perception 2023; 52:695-711. [PMID: 37427421 PMCID: PMC10510303 DOI: 10.1177/03010066231186936] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Accepted: 06/22/2023] [Indexed: 07/11/2023]
Abstract
In contrast to prototypical facial expressions, we show less perceptual tolerance in perceiving vague expressions by demonstrating an interpretation bias, such as more frequent perception of anger or happiness when categorizing ambiguous expressions of angry and happy faces that are morphed in different proportions and displayed under high- or low-quality conditions. However, it remains unclear whether this interpretation bias is specific to emotion categories or reflects a general negativity versus positivity bias and whether the degree of this bias is affected by the valence or category of two morphed expressions. These questions were examined in two eye-tracking experiments by systematically manipulating expression ambiguity and image quality in fear- and sad-happiness faces (Experiment 1) and by directly comparing anger-, fear-, sadness-, and disgust-happiness expressions (Experiment 2). We found that increasing expression ambiguity and degrading image quality induced a general negativity versus positivity bias in expression categorization. The degree of negativity bias, the associated reaction time and face-viewing gaze allocation were further manipulated by different expression combinations. It seems that although we show a viewing condition-dependent bias in interpreting vague facial expressions that display valence-contradicting expressive cues, it appears that the perception of these ambiguous expressions is guided by a categorical process similar to that involved in perceiving prototypical expressions.
Collapse
|
3
|
Scheibe S, Grundmann F, Kranenborg B, Epstude K. Empathising with masked targets: limited side effects of face masks on empathy for dynamic, context-rich stimuli. Cogn Emot 2023; 37:683-695. [PMID: 37013854 DOI: 10.1080/02699931.2023.2193385] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Revised: 03/14/2023] [Accepted: 03/15/2023] [Indexed: 04/05/2023]
Abstract
Multiple studies revealed detrimental effects of face masks on communication, including reduced empathic accuracy and enhanced listening effort. Yet, extant research relied on artificial, decontextualised stimuli, which prevented assessing empathy under more ecologically valid conditions. In this preregistered online experiment (N = 272), we used film clips featuring targets reporting autobiographical events to address motivational mechanisms underlying face mask effects on cognitive (empathic accuracy) and emotional facets (emotional congruence, sympathy) of empathy. Surprisingly, targets whose faces were covered by a mask (or a black bar) elicited the same level of empathy motives (affiliation, cognitive effort), and accordingly, the same level of cognitive and emotional empathy compared to targets with uncovered faces. We only found a negative direct effect of face coverings on sympathy. Additional analyses revealed that older (compared to young) adults showed higher empathy, but age did not moderate face mask effects. Our findings speak against strong negative face mask effects on empathy when using dynamic, context-rich stimuli, yet support motivational mechanisms of empathy.
Collapse
Affiliation(s)
- Susanne Scheibe
- Department of Psychology, University of Groningen, Groningen, Netherlands
| | - Felix Grundmann
- Department of Psychology, University of Groningen, Groningen, Netherlands
| | - Bart Kranenborg
- Department of Psychology, University of Groningen, Groningen, Netherlands
| | - Kai Epstude
- Department of Psychology, University of Groningen, Groningen, Netherlands
| |
Collapse
|
4
|
Mukhiddinov M, Djuraev O, Akhmedov F, Mukhamadiyev A, Cho J. Masked Face Emotion Recognition Based on Facial Landmarks and Deep Learning Approaches for Visually Impaired People. SENSORS (BASEL, SWITZERLAND) 2023; 23:1080. [PMID: 36772117 PMCID: PMC9921901 DOI: 10.3390/s23031080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/30/2022] [Revised: 01/10/2023] [Accepted: 01/15/2023] [Indexed: 06/18/2023]
Abstract
Current artificial intelligence systems for determining a person's emotions rely heavily on lip and mouth movement and other facial features such as eyebrows, eyes, and the forehead. Furthermore, low-light images are typically classified incorrectly because of the dark region around the eyes and eyebrows. In this work, we propose a facial emotion recognition method for masked facial images using low-light image enhancement and feature analysis of the upper features of the face with a convolutional neural network. The proposed approach employs the AffectNet image dataset, which includes eight types of facial expressions and 420,299 images. Initially, the facial input image's lower parts are covered behind a synthetic mask. Boundary and regional representation methods are used to indicate the head and upper features of the face. Secondly, we effectively adopt a facial landmark detection method-based feature extraction strategy using the partially covered masked face's features. Finally, the features, the coordinates of the landmarks that have been identified, and the histograms of the oriented gradients are then incorporated into the classification procedure using a convolutional neural network. An experimental evaluation shows that the proposed method surpasses others by achieving an accuracy of 69.3% on the AffectNet dataset.
Collapse
Affiliation(s)
- Mukhriddin Mukhiddinov
- Department of Computer Engineering, Gachon University, Seongnam 13120, Republic of Korea
| | - Oybek Djuraev
- Department of Hardware and Software of Control Systems in Telecommunication, Tashkent University of Information Technologies Named after Muhammad al-Khwarizmi, Tashkent 100084, Uzbekistan
| | - Farkhod Akhmedov
- Department of Computer Engineering, Gachon University, Seongnam 13120, Republic of Korea
| | - Abdinabi Mukhamadiyev
- Department of Computer Engineering, Gachon University, Seongnam 13120, Republic of Korea
| | - Jinsoo Cho
- Department of Computer Engineering, Gachon University, Seongnam 13120, Republic of Korea
| |
Collapse
|
5
|
Grundmann F, Epstude K, Scheibe S. Face masks reduce emotion-recognition accuracy and perceived closeness. PLoS One 2021; 16:e0249792. [PMID: 33891614 PMCID: PMC8064590 DOI: 10.1371/journal.pone.0249792] [Citation(s) in RCA: 124] [Impact Index Per Article: 41.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2021] [Accepted: 03/24/2021] [Indexed: 12/25/2022] Open
Abstract
Face masks became the symbol of the global fight against the coronavirus. While face masks' medical benefits are clear, little is known about their psychological consequences. Drawing on theories of the social functions of emotions and rapid trait impressions, we tested hypotheses on face masks' effects on emotion-recognition accuracy and social judgments (perceived trustworthiness, likability, and closeness). Our preregistered study with 191 German adults revealed that face masks diminish people's ability to accurately categorize an emotion expression and make target persons appear less close. Exploratory analyses further revealed that face masks buffered the negative effect of negative (vs. non-negative) emotion expressions on perceptions of trustworthiness, likability, and closeness. Associating face masks with the coronavirus' dangers predicted higher perceptions of closeness for masked but not for unmasked faces. By highlighting face masks' effects on social functioning, our findings inform policymaking and point at contexts where alternatives to face masks are needed.
Collapse
Affiliation(s)
- Felix Grundmann
- Department of Psychology, University of Groningen, Groningen, The Netherlands
| | - Kai Epstude
- Department of Psychology, University of Groningen, Groningen, The Netherlands
| | - Susanne Scheibe
- Department of Psychology, University of Groningen, Groningen, The Netherlands
| |
Collapse
|
6
|
Kinchella J, Guo K. Facial Expression Ambiguity and Face Image Quality Affect Differently on Expression Interpretation Bias. Perception 2021; 50:328-342. [PMID: 33709837 DOI: 10.1177/03010066211000270] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
We often show an invariant or comparable recognition performance for perceiving prototypical facial expressions, such as happiness and anger, under different viewing settings. However, it is unclear to what extent the categorisation of ambiguous expressions and associated interpretation bias are invariant in degraded viewing conditions. In this exploratory eye-tracking study, we systematically manipulated both facial expression ambiguity (via morphing happy and angry expressions in different proportions) and face image clarity/quality (via manipulating image resolution) to measure participants' expression categorisation performance, perceived expression intensity, and associated face-viewing gaze distribution. Our analysis revealed that increasing facial expression ambiguity and decreasing face image quality induced the opposite direction of expression interpretation bias (negativity vs. positivity bias, or increased anger vs. increased happiness categorisation), the same direction of deterioration impact on rating expression intensity, and qualitatively different influence on face-viewing gaze allocation (decreased gaze at eyes but increased gaze at mouth vs. stronger central fixation bias). These novel findings suggest that in comparison with prototypical facial expressions, our visual system has less perceptual tolerance in processing ambiguous expressions which are subject to viewing condition-dependent interpretation bias.
Collapse
|
7
|
Stevenson N, Guo K. Image Valence Modulates the Processing of Low-Resolution Affective Natural Scenes. Perception 2020; 49:1057-1068. [PMID: 32924858 DOI: 10.1177/0301006620957213] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
In natural vision, noisy and distorted visual inputs often change our perceptual strategy in scene perception. However, it is unclear the extent to which the affective meaning embedded in the degraded natural scenes modulates our scene understanding and associated eye movements. In this eye-tracking experiment by presenting natural scene images with different categories and levels of emotional valence (high-positive, medium-positive, neutral/low-positive, medium-negative, and high-negative), we systematically investigated human participants' perceptual sensitivity (image valence categorization and arousal rating) and image-viewing gaze behaviour to the changes of image resolution. Our analysis revealed that reducing image resolution led to decreased valence recognition and arousal rating, decreased number of fixations in image-viewing but increased individual fixation duration, and stronger central fixation bias. Furthermore, these distortion effects were modulated by the scene valence with less deterioration impact on the valence categorization of negatively valenced scenes and on the gaze behaviour in viewing of high emotionally charged (high-positive and high-negative) scenes. It seems that our visual system shows a valence-modulated susceptibility to the image distortions in scene perception.
Collapse
|
8
|
Guo K, Calver L, Soornack Y, Bourke P. Valence-dependent Disruption in Processing of Facial Expressions of Emotion in Early Visual Cortex—A Transcranial Magnetic Stimulation Study. J Cogn Neurosci 2020; 32:906-916. [DOI: 10.1162/jocn_a_01520] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Abstract
Our visual inputs are often entangled with affective meanings in natural vision, implying the existence of extensive interaction between visual and emotional processing. However, little is known about the neural mechanism underlying such interaction. This exploratory transcranial magnetic stimulation (TMS) study examined the possible involvement of the early visual cortex (EVC, Area V1/V2/V3) in perceiving facial expressions of different emotional valences. Across three experiments, single-pulse TMS was delivered at different time windows (50–150 msec) after a brief 10-msec onset of face images, and participants reported the visibility and perceived emotional valence of faces. Interestingly, earlier TMS at ∼90 msec only reduced the face visibility irrespective of displayed expressions, but later TMS at ∼120 msec selectively disrupted the recognition of negative facial expressions, indicating the involvement of EVC in the processing of negative expressions at a later time window, possibly beyond the initial processing of fed-forward facial structure information. The observed TMS effect was further modulated by individuals' anxiety level. TMS at ∼110–120 msec disrupted the recognition of anger significantly more for those scoring relatively low in trait anxiety than the high scorers, suggesting that cognitive bias influences the processing of facial expressions in EVC. Taken together, it seems that EVC is involved in structural encoding of (at least) negative facial emotional valence, such as fear and anger, possibly under modulation from higher cortical areas.
Collapse
|
9
|
Maza A, Moliner B, Ferri J, Llorens R. Visual Behavior, Pupil Dilation, and Ability to Identify Emotions From Facial Expressions After Stroke. Front Neurol 2020; 10:1415. [PMID: 32116988 PMCID: PMC7016192 DOI: 10.3389/fneur.2019.01415] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2019] [Accepted: 12/27/2019] [Indexed: 11/16/2022] Open
Abstract
Social cognition is the innate human ability to interpret the emotional state of others from contextual verbal and non-verbal information, and to self-regulate accordingly. Facial expressions are one of the most relevant sources of non-verbal communication, and their interpretation has been extensively investigated in the literature, using both behavioral and physiological measures, such as those derived from visual activity and visual responses. The decoding of facial expressions of emotion is performed by conscious and unconscious cognitive processes that involve a complex brain network that can be damaged after cerebrovascular accidents. A diminished ability to identify facial expressions of emotion has been reported after stroke, which has traditionally been attributed to impaired emotional processing. While this can be true, an alteration in visual behavior after brain injury could also negatively contribute to this ability. This study investigated the accuracy, distribution of responses, visual behavior, and pupil dilation of individuals with stroke while identifying emotional facial expressions. Our results corroborated impaired performance after stroke and exhibited decreased attention to the eyes, evidenced by a diminished time and number of fixations made in this area in comparison to healthy subjects and comparable pupil dilation. The differences in visual behavior reached statistical significance in some emotions when comparing individuals with stroke with impaired performance with healthy subjects, but not when individuals post-stroke with comparable performance were considered. The performance dependence of visual behavior, although not determinant, might indicate that altered visual behavior could be a negatively contributing factor for emotion recognition from facial expressions.
Collapse
Affiliation(s)
- Anny Maza
- Neurorehabilitation and Brain Research Group, Instituto de Investigación e Innovación en Bioingeniería, Universitat Politècnica de València, Valencia, Spain
| | - Belén Moliner
- NEURORHB, Servicio de Neurorrehabilitación de Hospitales Vithas, Valencia, Spain
| | - Joan Ferri
- NEURORHB, Servicio de Neurorrehabilitación de Hospitales Vithas, Valencia, Spain
| | - Roberto Llorens
- Neurorehabilitation and Brain Research Group, Instituto de Investigación e Innovación en Bioingeniería, Universitat Politècnica de València, Valencia, Spain.,NEURORHB, Servicio de Neurorrehabilitación de Hospitales Vithas, Valencia, Spain
| |
Collapse
|
10
|
Pollux PM, Craddock M, Guo K. Gaze patterns in viewing static and dynamic body expressions. Acta Psychol (Amst) 2019; 198:102862. [PMID: 31226535 DOI: 10.1016/j.actpsy.2019.05.014] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2019] [Revised: 05/09/2019] [Accepted: 05/26/2019] [Indexed: 11/25/2022] Open
Abstract
Evidence for the importance of bodily cues for emotion recognition has grown over the last two decades. Despite this growing literature, it is underspecified how observers view whole bodies for body expression recognition. Here we investigate to which extent body-viewing is face- and context-specific when participants are categorizing whole body expressions in static (Experiment 1) and dynamic displays (Experiment 2). Eye-movement recordings showed that observers viewed the face exclusively when visible in dynamic displays, whereas viewing was distributed over head, torso and arms in static displays and in dynamic displays with faces not visible. The strong face bias in dynamic face-visible expressions suggests that viewing of the body responds flexibly to the informativeness of facial cues for emotion categorisation. However, when facial expressions are static or not visible, observers adopt a viewing strategy that includes all upper body regions. This viewing strategy is further influenced by subtle viewing biases directed towards emotion-specific body postures and movements to optimise recruitment of diagnostic information for emotion categorisation.
Collapse
|
11
|
Oruc I, Balas B, Landy MS. Introduction to the special issue on face perception: Experience, models, and neural mechanisms. Vision Res 2019; 157:10-11. [PMID: 31173774 DOI: 10.1016/j.visres.2019.06.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Ipek Oruc
- Ophthalmology and Visual Sciences, University of British Columbia, Canada; Neuroscience, University of British Columbia, Canada
| | - Benjamin Balas
- Department of Psychology and Center for Visual and Cognitive Neuroscience, North Dakota State University, United States
| | - Michael S Landy
- Department of Psychology and Center for Neural Science, New York University, United States
| |
Collapse
|