1
|
Tessier MH, Mazet JP, Gagner E, Marcoux A, Jackson PL. Facial representations of complex affective states combining pain and a negative emotion. Sci Rep 2024; 14:11686. [PMID: 38777852 PMCID: PMC11111784 DOI: 10.1038/s41598-024-62423-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Accepted: 05/16/2024] [Indexed: 05/25/2024] Open
Abstract
Pain is rarely communicated alone, as it is often accompanied by emotions such as anger or sadness. Communicating these affective states involves shared representations. However, how an individual conceptually represents these combined states must first be tested. The objective of this study was to measure the interaction between pain and negative emotions on two types of facial representations of these states, namely visual (i.e., interactive virtual agents; VAs) and sensorimotor (i.e., one's production of facial configurations). Twenty-eight participants (15 women) read short written scenarios involving only pain or a combined experience of pain and a negative emotion (anger, disgust, fear, or sadness). They produced facial configurations representing these experiences on the faces of the VAs and on their face (own production or imitation of VAs). The results suggest that affective states related to a direct threat to the body (i.e., anger, disgust, and pain) share a similar facial representation, while those that present no immediate danger (i.e., fear and sadness) differ. Although visual and sensorimotor representations of these states provide congruent affective information, they are differently influenced by factors associated with the communication cycle. These findings contribute to our understanding of pain communication in different affective contexts.
Collapse
Affiliation(s)
- Marie-Hélène Tessier
- School of Psychology, Université Laval, Québec City, Canada
- Centre for Interdisciplinary Research in Rehabilitation and Social Integration (Cirris), Québec City, Canada
- CERVO Brain Research Centre, Québec City, Canada
| | - Jean-Philippe Mazet
- Department of Computer Science and Software Engineering, Université Laval, Québec City, Canada
| | - Elliot Gagner
- School of Psychology, Université Laval, Québec City, Canada
- Centre for Interdisciplinary Research in Rehabilitation and Social Integration (Cirris), Québec City, Canada
- CERVO Brain Research Centre, Québec City, Canada
| | - Audrey Marcoux
- School of Psychology, Université Laval, Québec City, Canada
- Centre for Interdisciplinary Research in Rehabilitation and Social Integration (Cirris), Québec City, Canada
- CERVO Brain Research Centre, Québec City, Canada
| | - Philip L Jackson
- School of Psychology, Université Laval, Québec City, Canada.
- Centre for Interdisciplinary Research in Rehabilitation and Social Integration (Cirris), Québec City, Canada.
- CERVO Brain Research Centre, Québec City, Canada.
| |
Collapse
|
2
|
Msika EF, Despres M, Piolino P, Narme P. Dynamic and/or multimodal assessments for social cognition in neuropsychology: Results from a systematic literature review. Clin Neuropsychol 2024; 38:922-962. [PMID: 37904259 DOI: 10.1080/13854046.2023.2266172] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 09/27/2023] [Indexed: 11/01/2023]
Abstract
Objective: Despite the prevalence of socio-cognitive disturbances, and their important diagnostic/therapeutic implications, the assessment of these disturbances remains scarce. This systematic review aims to identify available social cognition tools for adult assessment that use multimodal and/or dynamic social cues, specifying their strengths and limitations (e.g. from a methodological, psychometric, ecological, and clinical perspective). Method: An electronic search was conducted in Pubmed, PsychINFO, Embase and Scopus databases for articles published up to the 3th of January 2023 and the first 200 Google Scholar results on the same date. The PRISMA methodology was applied, 3884 studies were screened based on title and abstract and 329 full texts were screened. Articles using pseudo-dynamic methodologies (e.g. morphing), reported only subjective or self-reported measures, or investigated only physiological or brain activity responses were excluded. Results: In total, 149 works were included in this review, representing 65 assessment tools (i.e. 48% studying emotion recognition (n = 31), 32% Theory of Mind (n = 21), 5% empathy (n = 3), 1.5% moral cognition/social reasoning (n = 1), and 14% being multimodal (n = 9)). For each study, the tool's main characteristics, psychometric properties, ecological validity indicators and available norms are reported. The tools are presented according to social-cognitive process assessed and communication channels used. Conclusions: This study highlights the lack of validated and standardized tools. A few tools appear to partially meet some clinical needs. The development of methodologies using a first-person paradigm and taking into account the multidimensional nature of social cognition seems a relevant research endeavour for greater ecological validity.
Collapse
Affiliation(s)
- Eva-Flore Msika
- Laboratoire Mémoire, Cerveau et Cognition, Université Paris Cité, Boulogne-Billancourt, France
| | - Mathilde Despres
- Laboratoire Mémoire, Cerveau et Cognition, Université Paris Cité, Boulogne-Billancourt, France
| | - Pascale Piolino
- Laboratoire Mémoire, Cerveau et Cognition, Université Paris Cité, Boulogne-Billancourt, France
| | - Pauline Narme
- Laboratoire Mémoire, Cerveau et Cognition, Université Paris Cité, Boulogne-Billancourt, France
| |
Collapse
|
3
|
Gu X, Li S, Yi K, Yang X, Liu H, Wang G. Role-Exchange Playing: An Exploration of Role-Playing Effects for Anti-Bullying in Immersive Virtual Environments. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2023; 29:4215-4228. [PMID: 35727780 DOI: 10.1109/tvcg.2022.3184986] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Role-playing is widely used in many areas, such as psychotherapy and behavior change. However, few studies have explored the possible effects of playing multiple roles in a single role-playing process. We propose a new role-playing paradigm, called role-exchange playing, in which a user plays two opposite roles successively in the same simulated event for better cognitive enhancement. We designed an experiment with this novel role-exchange playing strategy in the immersive virtual environments; and school bullying was chosen as a scenario in this case. A total of 234 middle/high school students were enrolled in the mixed-design experiment. From the user study, we found that through role-exchange, students developed more morally correct opinions about bullying, as well as increased empathy and willingness to engage in supportive behavior. They also showed increased commitment to stopping bullying others. Our role-exchange paradigm could achieve a better effect than traditional role-playing methods in situations where participants have no prior experience associated with the roles they play. Therefore, using role-exchange playing in the immersive virtual environments to educate minors can help prevent them from bullying others in the real world. Our study indicates a positive significance in moral education of teenagers. Our role-exchange playing may have the potential to be extended to such applications as counseling, therapy, and crime prevention.
Collapse
|
4
|
Treal T, Jackson PL, Meugnot A. Biological postural oscillations during facial expression of pain in virtual characters modulate early and late ERP components associated with empathy: A pilot study. Heliyon 2023; 9:e18161. [PMID: 37560681 PMCID: PMC10407205 DOI: 10.1016/j.heliyon.2023.e18161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Revised: 06/21/2023] [Accepted: 07/10/2023] [Indexed: 08/11/2023] Open
Abstract
There is a surge in the use of virtual characters in cognitive sciences. However, their behavioural realism remains to be perfected in order to trigger more spontaneous and socially expected reactions in users. It was recently shown that biological postural oscillations (idle motion) were a key ingredient to enhance the empathic response to its facial pain expression. The objective of this study was to examine, using electroencephalography, whether idle motion would modulate the neural response associated with empathy when viewing a pain-expressing virtual character. Twenty healthy young adults were shown video clips of a virtual character displaying a facial expression of pain while its body was either static (Still condition) or animated with pre-recorded human postural oscillations (Idle condition). Participants rated the virtual human's facial expression of pain as significantly more intense in the Idle condition compared to the Still condition. Both the early (N2-N3) and the late (rLPP) event-related potentials (ERPs) associated with distinct dimensions of empathy, affective resonance and perspective-taking, respectively, were greater in the Idle condition compared to the Still condition. These findings confirm the potential of idle motion to increase empathy for pain expressed by virtual characters. They are discussed in line with contemporary empathy models in relation to human-machine interactions.
Collapse
Affiliation(s)
- Thomas Treal
- Université Paris-Saclay CIAMS, 91405, Orsay, France
- CIAMS, Université d'Orléans, 45067, Orléans, France
| | - Philip L. Jackson
- École de Psychologie, Université Laval, Québec, Canada
- Centre Interdisciplinaire de Recherche en Réadaptation et Intégration Sociale (CIRRIS), Québec, Canada
- CERVO Research Center, Québec, Canada
| | - Aurore Meugnot
- Université Paris-Saclay CIAMS, 91405, Orsay, France
- CIAMS, Université d'Orléans, 45067, Orléans, France
| |
Collapse
|
5
|
Treal T, Jackson PL, Jeuvrey J, Vignais N, Meugnot A. Natural human postural oscillations enhance the empathic response to a facial pain expression in a virtual character. Sci Rep 2021; 11:12493. [PMID: 34127724 PMCID: PMC8203793 DOI: 10.1038/s41598-021-91710-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Accepted: 05/21/2021] [Indexed: 01/10/2023] Open
Abstract
Virtual reality platforms producing interactive and highly realistic characters are being used more and more as a research tool in social and affective neuroscience to better capture both the dynamics of emotion communication and the unintentional and automatic nature of emotional processes. While idle motion (i.e., non-communicative movements) is commonly used to create behavioural realism, its use to enhance the perception of emotion expressed by a virtual character is critically lacking. This study examined the influence of naturalistic (i.e., based on human motion capture) idle motion on two aspects (the perception of other’s pain and affective reaction) of an empathic response towards pain expressed by a virtual character. In two experiments, 32 and 34 healthy young adults were presented video clips of a virtual character displaying a facial expression of pain while its body was either static (still condition) or animated with natural postural oscillations (idle condition). The participants in Experiment 1 rated the facial pain expression of the virtual human as more intense, and those in Experiment 2 reported being more touched by its pain expression in the idle condition compared to the still condition, indicating a greater empathic response towards the virtual human’s pain in the presence of natural postural oscillations. These findings are discussed in relation to the models of empathy and biological motion processing. Future investigations will help determine to what extent such naturalistic idle motion could be a key ingredient in enhancing the anthropomorphism of a virtual human and making its emotion appear more genuine.
Collapse
Affiliation(s)
- Thomas Treal
- CIAMS, Université Paris-Saclay, 91405, Orsay, France.,CIAMS, Université D'Orléans, 45067, Orléans, France
| | - Philip L Jackson
- École de Psychologie, Université Laval, Québec, Canada.,Centre Interdisciplinaire de Recherche en Réadaptation Et Intégration Sociale (CIRRIS), Québec, Canada.,CERVO Research Center, Québec, Canada
| | - Jean Jeuvrey
- CIAMS, Université Paris-Saclay, 91405, Orsay, France.,CIAMS, Université D'Orléans, 45067, Orléans, France
| | - Nicolas Vignais
- CIAMS, Université Paris-Saclay, 91405, Orsay, France.,CIAMS, Université D'Orléans, 45067, Orléans, France
| | - Aurore Meugnot
- CIAMS, Université Paris-Saclay, 91405, Orsay, France. .,CIAMS, Université D'Orléans, 45067, Orléans, France.
| |
Collapse
|
6
|
Treal T, Jackson PL, Meugnot A. Combining trunk movement and facial expression enhances the perceived intensity and believability of an avatar's pain expression. COMPUTERS IN HUMAN BEHAVIOR 2020. [DOI: 10.1016/j.chb.2020.106451] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
7
|
Grondin F, Lomanowska AM, Jackson PL. Empathy in computer-mediated interactions: A conceptual framework for research and clinical practice. CLINICAL PSYCHOLOGY-SCIENCE AND PRACTICE 2019. [DOI: 10.1111/cpsp.12298] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
|
8
|
Grabowski K, Rynkiewicz A, Lassalle A, Baron-Cohen S, Schuller B, Cummins N, Baird A, Podgórska-Bednarz J, Pieniążek A, Łucka I. Emotional expression in psychiatric conditions: New technology for clinicians. Psychiatry Clin Neurosci 2019; 73:50-62. [PMID: 30565801 DOI: 10.1111/pcn.12799] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/11/2017] [Revised: 09/24/2018] [Accepted: 11/11/2018] [Indexed: 12/24/2022]
Abstract
AIM Emotional expressions are one of the most widely studied topics in neuroscience, from both clinical and non-clinical perspectives. Atypical emotional expressions are seen in various psychiatric conditions, including schizophrenia, depression, and autism spectrum conditions. Understanding the basics of emotional expressions and recognition can be crucial for diagnostic and therapeutic procedures. Emotions can be expressed in the face, gesture, posture, voice, and behavior and affect physiological parameters, such as the heart rate or body temperature. With modern technology, clinicians can use a variety of tools ranging from sophisticated laboratory equipment to smartphones and web cameras. The aim of this paper is to review the currently used tools using modern technology and discuss their usefulness as well as possible future directions in emotional expression research and treatment strategies. METHODS The authors conducted a literature review in the PubMed, EBSCO, and SCOPUS databases, using the following key words: 'emotions,' 'emotional expression,' 'affective computing,' and 'autism.' The most relevant and up-to-date publications were identified and discussed. Search results were supplemented by the authors' own research in the field of emotional expression. RESULTS We present a critical review of the currently available technical diagnostic and therapeutic methods. The most important studies are summarized in a table. CONCLUSION Most of the currently available methods have not been adequately validated in clinical settings. They may be a great help in everyday practice; however, they need further testing. Future directions in this field include more virtual-reality-based and interactive interventions, as well as development and improvement of humanoid robots.
Collapse
Affiliation(s)
- Karol Grabowski
- Department of Psychiatry, Adult Psychiatry Clinic, Faculty of Medicine, Medical University of Gdansk, Gdansk, Poland
| | - Agnieszka Rynkiewicz
- Neurodevelopmental Disorders Research Lab, Institute of Experimental and Clinical Medicine, Faculty of Medicine, University of Rzeszow, Rzeszow, Poland.,Center for Diagnosis, Therapy and Education SPECTRUM ASC-MED, Gdansk & Rzeszow, Poland
| | - Amandine Lassalle
- Department of Psychology, Brain & Cognition, University of Amsterdam, Amsterdam, Netherlands
| | - Simon Baron-Cohen
- Autism Research Centre, Department of Psychiatry, University of Cambridge, Cambridge, UK
| | - Björn Schuller
- Department of Computing, GLAM - Group on Language, Audio, and Music, Imperial College London, London, UK
| | - Nicholas Cummins
- Department of Computing, GLAM - Group on Language, Audio, and Music, Imperial College London, London, UK
| | - Alice Baird
- Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany
| | - Justyna Podgórska-Bednarz
- Institute of Physiotherapy, Faculty of Medicine, University of Rzeszow, Rzeszow, Poland.,Association for Children with Attention Deficit Hyperactivity Disorder in Rzeszow, Rzeszow, Poland
| | - Agata Pieniążek
- Institute of Physiotherapy, Faculty of Medicine, University of Rzeszow, Rzeszow, Poland.,SOLIS RADIUS Association for People with Disabilities and Autism Spectrum Disorders in Rzeszow, Rzeszow, Poland.,Medical Center for Children with Autism Spectrum Disorders in Rzeszow, Rzeszow, Poland
| | - Izabela Łucka
- Developmental Psychiatry, Psychotic and Geriatric Disorders Clinic, Department of Psychiatry, Faculty of Medicine, Medical University of Gdansk, Gdansk, Poland
| |
Collapse
|
9
|
Fusaro M, Tieri G, Aglioti S. Influence of cognitive stance and physical perspective on subjective and autonomic reactivity to observed pain and pleasure: An immersive virtual reality study. Conscious Cogn 2019; 67:86-97. [DOI: 10.1016/j.concog.2018.11.010] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2018] [Revised: 11/09/2018] [Accepted: 11/27/2018] [Indexed: 12/20/2022]
|
10
|
Affiliation(s)
- Simone Shamay-Tsoory
- Department of Psychology, University of Haifa, Mount Carmel, Haifa 31905, Israel.
| | - Claus Lamm
- Department of Basic Psychological Research and Research Methods, Faculty of Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria.
| |
Collapse
|
11
|
Robitaille N, Jackson PL, Hébert LJ, Mercier C, Bouyer LJ, Fecteau S, Richards CL, McFadyen BJ. A Virtual Reality avatar interaction (VRai) platform to assess residual executive dysfunction in active military personnel with previous mild traumatic brain injury: proof of concept. Disabil Rehabil Assist Technol 2016; 12:758-764. [DOI: 10.1080/17483107.2016.1229048] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Nicolas Robitaille
- Centre interdisciplinaire de recherche en réadaptation et intégration sociale*, Québec, QC, Canada
| | - Philip L. Jackson
- Centre interdisciplinaire de recherche en réadaptation et intégration sociale*, Québec, QC, Canada
- School of Psychology, Laval University, Québec, QC, Canada
| | - Luc J. Hébert
- Centre interdisciplinaire de recherche en réadaptation et intégration sociale*, Québec, QC, Canada
- Department of Rehabilitation, Faculty of Medicine, Laval University, Québec, QC, Canada
- Department of Radiology, Faculty of Medicine, Laval University, Québec, QC, Canada
| | - Catherine Mercier
- Centre interdisciplinaire de recherche en réadaptation et intégration sociale*, Québec, QC, Canada
- Department of Rehabilitation, Faculty of Medicine, Laval University, Québec, QC, Canada
| | - Laurent J. Bouyer
- Centre interdisciplinaire de recherche en réadaptation et intégration sociale*, Québec, QC, Canada
- Department of Rehabilitation, Faculty of Medicine, Laval University, Québec, QC, Canada
| | - Shirley Fecteau
- Centre interdisciplinaire de recherche en réadaptation et intégration sociale*, Québec, QC, Canada
- Department of Rehabilitation, Faculty of Medicine, Laval University, Québec, QC, Canada
- Centre de recherche de l’institut universitaire en santé mentale de Québec*, Québec, QC, Canada
| | - Carol L. Richards
- Centre interdisciplinaire de recherche en réadaptation et intégration sociale*, Québec, QC, Canada
- Department of Rehabilitation, Faculty of Medicine, Laval University, Québec, QC, Canada
| | - Bradford J. McFadyen
- Centre interdisciplinaire de recherche en réadaptation et intégration sociale*, Québec, QC, Canada
- Department of Rehabilitation, Faculty of Medicine, Laval University, Québec, QC, Canada
| |
Collapse
|
12
|
Davies H, Wolz I, Leppanen J, Fernandez-Aranda F, Schmidt U, Tchanturia K. Facial expression to emotional stimuli in non-psychotic disorders: A systematic review and meta-analysis. Neurosci Biobehav Rev 2016; 64:252-71. [PMID: 26915928 DOI: 10.1016/j.neubiorev.2016.02.015] [Citation(s) in RCA: 73] [Impact Index Per Article: 9.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2015] [Revised: 02/01/2016] [Accepted: 02/17/2016] [Indexed: 02/07/2023]
Abstract
Facial expression of emotion is crucial to social interaction and emotion regulation; therefore, altered facial expressivity can be a contributing factor in social isolation, difficulties with emotion regulation and a target for therapy. This article provides a systematic review and meta-analysis of the literature on automatic emotional facial expression in people with non-psychotic disorders compared to healthy comparison groups. Studies in the review used an emotionally salient visual induction method, and reported on automatic facial expression in response to congruent stimuli. A total of 39 studies show alterations in emotional facial expression across all included disorders, except anxiety disorders. In depression, decreases in facial expression are mainly evident for positive affect. In eating disorders, a meta-analysis showed decreased facial expressivity in response to positive and negative stimuli. Studies in autism partially support generally decreased facial expressivity in this group. The data included in this review point towards decreased facial emotional expressivity in individuals with different non-psychotic disorders. This is the first review to synthesise facial expression studies across clinical disorders.
Collapse
Affiliation(s)
- H Davies
- King's College London, Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neurosciences, London SE5 8AF, UK
| | - I Wolz
- Department of Psychiatry, University Hospital of Bellvitge-IDIBELL, Barcelona, Spain; CIBER Fisiopatologia Obesidad y Nutrición (CIBERobn), Instituto de Salud Carlos III, Barcelona, Spain
| | - J Leppanen
- King's College London, Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neurosciences, London SE5 8AF, UK
| | - F Fernandez-Aranda
- Department of Psychiatry, University Hospital of Bellvitge-IDIBELL, Barcelona, Spain; CIBER Fisiopatologia Obesidad y Nutrición (CIBERobn), Instituto de Salud Carlos III, Barcelona, Spain; Clinical Sciences Department, School of Medicine, University of Barcelona, Spain
| | - U Schmidt
- King's College London, Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neurosciences, London SE5 8AF, UK
| | - K Tchanturia
- King's College London, Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neurosciences, London SE5 8AF, UK; Illia University, Department of Psychology, Tbilisi, Georgia.
| |
Collapse
|