Effects of virtual human animation on emotion contagion in simulated inter-personal experiences

Effects of Small Talk With a Crowd of Virtual Humans on Users' Emotional and Behavioral Responses

In this contribution, we empirically investigated the effect of small talk on the users' non-verbal behaviors and emotions when users interacted with a crowd of virtual humans (VHs) with positive behavioral dispositions. Users were tasked with collecting items in a virtual marketplace via natural speech-based dialogue with a crowd of virtual pedestrians and vendors. The users were able to engage in natural speech-based conversation in a predefined corpus of small talk content that covered various commonplace small talk topics such as conversations about the weather, general concerns, and entertainment based on similar real-life situations. For instance, the VHs with the small talk ability would ask the users some simple questions to make small talk or remind the users of their belongings. We conducted a between-subjects empirical evaluation to investigate whether the user behaviors and emotions were different between a small talk condition and a non-small talk condition, and examined gender effects of the participants. We collected objective and subjective measures of the users to analyze users' emotions and social interaction behaviors, when in conversation with VHs that either possessed small-talk capability or not, besides task or goal oriented dialogue capabilities. Our result revealed that the VHs with small talk capability could alter the emotions and non-verbal behaviors of the users. Furthermore, the non-verbal behaviors between female and male participants differed greatly in the presence or absence of small talk.

On the Plausibility of Virtual Body Animation Features in Virtual Reality

We present two experiments to assess the relative impact of different levels of body animation fidelity on plausibility illusion (Psi). The first experiment presents a virtual character that is not controlled by the user ( n=13), while the second experiment presents a user-controlled virtual avatar ( n=24, all male). Psi concerns how realistic and coherent the events in a virtual environment look and feel and is part of Slater's proposition of two orthogonal components of presence in virtual reality (VR). In the experiments, the face, hands, upper and lower bodies of the character or self-avatar were manipulated to present different degrees of animation fidelity, such as no animation, procedural animation, and motion captured animation. Participants started the experiment experiencing the best animation configuration. Then, animation features were reduced to limit the amount of captured information made available to the system. Participants had to move from this basic animation configuration towards a more complete one, and declare when the avatar animation realism felt equivalent to the initial and most complete configuration, which could happen before all animation features were maxed out. Participants in the self-avatar experiment were also asked to rate how each animation feature affected their sense of control of the virtual body. We found that a virtual body with upper and lower body animated using eight tracked rigid bodies and inverse kinematics (IK) was often perceived as equivalent to a professional capture pipeline relying on 53 markers. Compared to what standard VR kits in the market are offering, i.e., a tracked headset and two hand controllers, we found that foot tracking, followed by mouth animation and finger tracking, were the features that added the most to the sense of control of a self-representing avatar. In addition, these features were often among the first to be improved in both experiments.

Designing Empathic Virtual Agents: Manipulating Animation, Voice, Rendering, and Empathy to Create Persuasive Agents

Designers of virtual agents have a combinatorically large space of choices for the look and behavior of their characters. We conducted two between-subjects studies to explore the systematic manipulation of animation quality, speech quality, rendering style, and simulated empathy, and its impact on perceptions of virtual agents in terms of naturalness, engagement, trust, credibility, and persuasion within a health counseling domain. In the first study, animation was varied between manually created, procedural, or no animations; voice quality was varied between recorded audio and synthetic speech; and rendering style was varied between realistic and toon-shaded. In the second study, simulated empathy of the agent was varied between no empathy, verbal-only empathic responses, and full empathy involving verbal, facial, and immediacy feedback. Results show that natural animations and recorded voice are more appropriate for the agent's general acceptance, trust, credibility, and appropriateness for the task. However, for a brief health counseling task, animation might actually be distracting from the persuasive message, with the highest levels of persuasion found when the amount of agent animation is minimized. Further, consistent and high levels of empathy improve agent perception but may interfere with forming a trusting bond with the agent.

VR Realism Scale-Revalidation of contemporary VR headsets on a Polish sample

This paper presents validation of the VR Simulation Realism Scale on a Polish sample. The scale enables a self-report measurement of perceived realism of a virtual environment in four main aspects of such realism-scene realism, audience behavior realism, audience appearance realism and sound realism. However, since the development of the original scale, the VR technology significantly changed. We aimed to respond to that change and revalidate the original measure in the contemporary setting. For the purpose of scale validation, data was gathered from six studies with 720 participants in total. Five experiments and one online survey were conducted to examine psychometric properties of the scale in accordance with the Standards for Educational and Psychological Testing. Evidence based on internal structure, relations to other variables and test content was obtained. The factorial structure of the original scale was tested and confirmed. The connections between realism and immersion, presence, aesthetics were verified. A suppressed relationship between realism and positive affect was discovered. Moreover, it was confirmed that scale result is dependent on the quality of VR graphics. Results of the analyses provide the evidence that the VR Simulation Realism Scale is a well-established tool that might be used both in science and in VR development. However, further research needs to be done to increase external validity and predictive power of the scale.

Virtual environments to study emotional responses to clinical communication: A scoping review

OBJECTIVE

This scoping review explores the potential for virtual environments (VE) to evaluate emotional outcomes in clinical communication research. Authors representing multiple disciplines use review results to propose potential research opportunities and considerations.

METHODS

We utilized a structured framework for scoping reviews. We searched four literature databases for relevant articles. We applied multidisciplinary perspectives to synthesize relevant potential opportunities for emotion-focused communications research using VE.

RESULTS

Twenty-one articles met inclusion criteria. They applied different methodological approaches, including a range of VE technologies and diverse emotional outcome measures, such as psychophysiological arousal, emotional valence, or empathy. Major research topics included use of virtual reality to provoke and measure emotional responses, train clinicians in communication skills, and increase clinician empathy.

CONCLUSION

Researchers may leverage VE technologies to ethically and systematically examine how characteristics of clinical interactions, environments, and communication impact emotional reactions and responses among patients and clinicians. Variability exists in how VE technologies are employed and reported in published literature, and this may limit the internal and external validity of the research. However, virtual reality can provide a low-cost, low-risk, experimentally controlled, and ecologically valid approach for studying clinician-patient communication.

PRACTICE IMPLICATIONS

Future research should leverage psychophysiological measures to further examine emotional responses during clinical communication scenarios and clearly report virtual environment characteristics to support evaluation of study conclusions, study replicability, and meta-analyses.

Natural human postural oscillations enhance the empathic response to a facial pain expression in a virtual character

Virtual reality platforms producing interactive and highly realistic characters are being used more and more as a research tool in social and affective neuroscience to better capture both the dynamics of emotion communication and the unintentional and automatic nature of emotional processes. While idle motion (i.e., non-communicative movements) is commonly used to create behavioural realism, its use to enhance the perception of emotion expressed by a virtual character is critically lacking. This study examined the influence of naturalistic (i.e., based on human motion capture) idle motion on two aspects (the perception of other’s pain and affective reaction) of an empathic response towards pain expressed by a virtual character. In two experiments, 32 and 34 healthy young adults were presented video clips of a virtual character displaying a facial expression of pain while its body was either static (still condition) or animated with natural postural oscillations (idle condition). The participants in Experiment 1 rated the facial pain expression of the virtual human as more intense, and those in Experiment 2 reported being more touched by its pain expression in the idle condition compared to the still condition, indicating a greater empathic response towards the virtual human’s pain in the presence of natural postural oscillations. These findings are discussed in relation to the models of empathy and biological motion processing. Future investigations will help determine to what extent such naturalistic idle motion could be a key ingredient in enhancing the anthropomorphism of a virtual human and making its emotion appear more genuine.

Customizing Your Demons: Anxiety Reduction via Anthropomorphizing and Destroying an "Anxiety Avatar"

Character customization is a prominent feature in digital games, affording users the ability to tailor one’s virtual self-representation (avatar) to match aspects of their actual or ideal self, influencing psychological well-being. The mental health implications of character customization can be partially explained by self-discrepancy theory, which argues that achieving congruence with one’s avatar reduces cognitive dissonance. However, the role of undesirable self-concepts such as mental health ailments have largely been overlooked in this context despite forming part of one’s identity. In theory, customization of an avatar representing undesirable self-concepts presents a self-regulatory paradox: individuals desire to reduce discrepancies with a self-representation, yet they also desire to enlarge discrepancies with a disliked-self. To reconcile this, two experiments explored the psychological implications of imbuing avatars with undesirable self-concepts. In Study 1 (N = 90), participants customized an avatar to represent anxiety within themselves (i.e., an anxiety avatar). Customization significantly reduced state anxiety compared to a control group, supporting the proposed discrepancy-reduction mechanism. Study 2 (N = 122) employed a 2 (customization: yes, no) × 2 (destruction: yes, no) between-subjects design, with participants either destroying or observing an anxiety avatar. Destruction of customized anxiety avatars resulted in the largest reduction in anxiety among all conditions, supporting the proposed discrepancy-enlargement mechanism. Theoretical and practical implications for the use of avatar-based e-mental health interventions are discussed.

Universality vs. Cultural Specificity in the Relations Among Emotional Contagion, Emotion Regulation, and Mood State: An Emotion Process Perspective

To investigate the universality and cultural specificity of emotion processing in children from three different ethnic groups (Han, Jingpo, and Dai), we administered three questionnaires, including the emotional contagion scale, emotion regulation scale, and the Chinese mood adjective check list, to 1,362 ethnic Han, Dai, and Jingpo participants (M_age = 13.78 years). Structural equation modeling was used to examine the universality and cultural specificity in the relations among emotional contagion, emotion regulation, and mood state. The results revealed that emotion regulation mediated the relation between emotional contagion and mood state: positive emotional contagion increased positive mood state and decreased negative mood state by the mediated role of reappraisal, negative emotional contagion decreased positive contagion and increased negative mood state by the inconsistent mediated role of reappraisal; negative contagion increased negative mood state by the mediated role of suppression. We found both universality and cultural specificity in the relations among emotional contagion, emotion regulation, and mood state. Regarding cultural specificity, among Dai and Jingpo participants, negative contagion positively predicted reappraisal, while for Han participants, it did not; Jingpo participants demonstrated a weaker negative relation between reappraisal and negative mood state, and a stronger positive relation between negative contagion and suppression; and Dai participants were the only ethnic group that showed a negative connection between negative contagion and positive mood state. Regarding emotion universality, the three ethnic groups all showed positive relations between negative contagion and negative mood, and between suppression and negative mood; additionally, positive contagion positively predicted positive mood state, mediated by reappraisal. Thus, some emotion processes are universal and others more specific. In this paper, we discuss universal emotion processes and ethnic cultural differences in these emotion processes.

A Systematic Review of Social Presence: Definition, Antecedents, and Implications

Social presence, or the feeling of being there with a "real" person, is a crucial component of interactions that take place in virtual reality. This paper reviews the concept, antecedents, and implications of social presence, with a focus on the literature regarding the predictors of social presence. The article begins by exploring the concept of social presence, distinguishing it from two other dimensions of presence-telepresence and self-presence. After establishing the definition of social presence, the article offers a systematic review of 233 separate findings identified from 152 studies that investigate the factors (i.e., immersive qualities, contextual differences, and individual psychological traits) that predict social presence. Finally, the paper discusses the implications of heightened social presence and when it does and does not enhance one's experience in a virtual environment.

Virtual Antenatal Encounter and Standardized Simulation Assessment (VANESSA): Pilot Study

Background

Prenatal counseling at the limits of newborn viability involves sensitive interactions between neonatal providers and families. Empathetic discussions are currently learned through practice in times of high stress. Decision aids may help improve provider communication but have not been universally adopted. Virtual standardized patients are increasingly recognized as a modality for education, but prenatal counseling simulations have not been described. To be valuable as a tool, a virtual patient would need to accurately portray emotions and elicit a realistic response from the provider.

Objective

To determine if neonatal providers can accurately identify a standardized virtual prenatal patient’s emotional states and examine the frequency of empathic responses to statements made by the patient.

Methods

A panel of Neonatologists, Simulation Specialists, and Ethicists developed a dialogue and identified empathic responses. Virtual Antenatal Encounter and Standardized Simulation Assessment (VANESSA), a screen-based simulation of a woman at 23 weeks gestation, was capable of displaying anger, fear, sadness, and happiness through animations. Twenty-four neonatal providers, including a subgroup with an ethics interest, were asked to identify VANESSA’s emotions 28 times, respond to statements, and answer open-ended questions. The emotions were displayed in different formats: without dialogue, with text dialogue, and with audio dialogue. Participants completed a post-encounter survey describing demographics and experience. Data were reported using descriptive statistics. Qualitative data from open ended questions (eg, “What would you do?”) were examined using thematic analysis.

Results

Half of our participants had over 10 years of clinical experience. Most participants reported using medical research (18/23, 78%) and mortality calculators (17/23, 74%). Only the ethics-interested subgroup (10/23, 43%) listed counseling literature (7/10, 70%). Of 672 attempts, participants accurately identified VANESSA’s emotions 77.8% (523/672) of the time, and most (14/23, 61%) reported that they were confident in identifying these emotions. The ethics interest group was more likely to choose empathic responses (P=.002). Participants rated VANESSA as easy to use (22/23, 96%) and reported that she had realistic dialogue (15/23, 65%).

Conclusions

This pilot study shows that a prenatal counseling simulation is feasible and can yield useful data on prenatal counseling communication. Our participants showed a high rate of emotion recognition and empathy in their responses.

Virtual Avatar for Emotion Recognition in Patients with Schizophrenia: A Pilot Study

Persons who suffer from schizophrenia have difficulties in recognizing emotions in others’ facial expressions, which affects their capabilities for social interaction and hinders their social integration. Photographic images have traditionally been used to explore emotion recognition impairments in schizophrenia patients, but they lack of the dynamism that is inherent to facial expressiveness. In order to overcome those inconveniences, over the last years different authors have proposed the use of virtual avatars. In this work, we present the results of a pilot study that explored the possibilities of using a realistic-looking avatar for the assessment of emotion recognition deficits in patients who suffer from schizophrenia. In the study, 20 subjects with schizophrenia of long evolution and 20 control subjects were invited to recognize a set of facial expressions of emotions showed by both the said virtual avatar and static images. Our results show that schizophrenic patients exhibit recognition deficits in emotion recognition from facial expressions regardless the type of stimuli (avatar or images), and that those deficits are related with the psychopathology. Finally, some improvements in recognition rates (RRs) for the patient group when using the avatar were observed for sadness or surprise expressions, and they even outperform the control group in the recognition of the happiness expression. This leads to conclude that, apart from the dynamism of the shown expression, the RRs for schizophrenia patients when employing animated avatars may depend on other factors which need to be further explored.

The Effects of Avatars, Stereo Vision and Display Size on Reaching and Motion Reproduction

Thanks to recent advances on motion capture devices and stereoscopic consumer displays, animated virtual characters can now realistically interact with users in a variety of applications. We investigate in this paper the effect of avatars, stereo vision and display size on task execution in immersive virtual environments. We report results obtained with three experiments in varied configurations that are commonly used in rehabilitation applications. The first experiment analyzes the accuracy of reaching tasks under different system configurations: with and without an avatar, with and without stereo vision, and employing a 2D desktop monitor versus a large multi-tile visualization display. The second experiment analyzes the use of avatars and user-perspective stereo vision on the ability to perceive and subsequently reproduce motions demonstrated by an autonomous virtual character. The third experiment evaluates the overall user experience with a complete immersive user interface for motion modeling by direct demonstration. Our experiments expose and quantify the benefits of using stereo vision and avatars, and show that the use of avatars improve the quality of produced motions and the resemblance of replicated msotions; however, direct interaction in user-perspective leads to tasks executed in less time and to targets more accurately reached. These and additional tradeoffs are important for the effective design of avatar-based training systems.

Effects of Virtual Human Appearance Fidelity on Emotion Contagion in Affective Inter-Personal Simulations

Realistic versus stylized depictions of virtual humans in simulated inter-personal situations and their ability to elicit emotional responses in users has been an open question for artists and researchers alike. We empirically evaluated the effects of near visually realistic vs. non-realistic stylized appearance of virtual humans on the emotional response of participants in a medical virtual reality system that was designed to educate users in recognizing the signs and symptoms of patient deterioration. In a between-subjects experiment protocol, participants interacted with one of three different appearances of a virtual patient, namely visually realistic, cartoon-shaded and charcoal-sketch like conditions in a mixed reality simulation. Emotional impact were measured via a combination of quantitative objective measures were gathered using skin Electrodermal Activity (EDA) sensors, and quantitative subjective measures such as the Differential Emotion Survey (DES IV), Positive and Negative Affect Schedule (PANAS), and Social Presence questionnaire. The emotional states of the participants were analyzed across four distinct time steps during which the medical condition of the virtual patient deteriorated (an emotionally stressful interaction), and were contrasted to a baseline affective state. Objective EDA results showed that in all three conditions, male participants exhibited greater levels of arousal as compared to female participants. We found that negative affect levels were significantly lower in the visually realistic condition, as compared to the stylized appearance conditions. Furthermore, in emotional dimensions of interest-excitement, surprise, anger, fear and guilt participants in all conditions responded similarly. However, in social emotional constructs of shyness, presence, perceived personality, and enjoyment-joy, we found that participants responded differently in the visually realistic condition as compared to the cartoon and sketch conditions. Our study suggests that virtual human appearance can affect not only critical emotional reactions in affective inter-personal training scenarios. but also users' perceptions of personality and social characteristic of the virtual interlocutors.