From Robot to Virtual Doppelganger: Impact of Visual Fidelity of Avatars Controlled in Third-Person Perspective on Embodiment and Behavior in Immersive Virtual Environments

The use of virtual reality as a perspective-taking manipulation to improve self-awareness in Alzheimer's disease

Lack of awareness of symptoms or having a condition referred to as anosognosia is a common feature of individuals with Alzheimer's Disease (AD). Previous literature on AD reported difficulties in evaluating self-abilities, often showing underestimation of limitations. There is increasing evidence that the perspective through which information is presented may moderate the performance appraisal and that anosognosia in AD might be a consequence of a deficit in assuming a third-person perspective. In this context, some studies showed that subjects may better recognize self-and other-difficulties when exposed to a third-person perspective. Considering the variety of approaches aiming to investigate the lack of awareness, there is still a scarcity of methods that provide great ecological validity and consider more than one facet of awareness, thus failing to offer more accurate evaluations of daily experiences. The present paper primarily addresses the theme of the multidimensional character of awareness of abilities in AD and the effect of perspective-taking on its trajectories. The focus turns to virtual reality as a promising tool for a greater evaluation of perspective-taking and self-awareness. Particularly, these systems offer the possibility to involve users in cognitive and sensorimotor tasks that simulate daily life conditions within immersive and realistic environments, and a great sense of embodiment. We propose that virtual reality might allow a great level of complexity, veracity, and safety that is needed for individuals with AD to behave according to their actual abilities and enable to explore the liaison between the subject's viewpoint, performance, and self-evaluation. In addition, we suggest promising clinical implications of virtual reality-based methods for individualized assessments, investigating specific impacts on subjects' life and possible improvements in their awareness.

Virtually simulated interpersonal touch negatively affects perceived closeness and social affiliation to an avatar partner

Interpersonal touch is an essential component of human non-verbal communication, facilitating social affiliation and bonding. With the widespread use of digital interfaces and online platforms in all realms of human interactions, there are fewer opportunities for communicating through touch. Popular online platforms that virtually simulate human interactions rely primarily on visual and auditory modalities, providing limited or no capacity for the exchange of tactile cues. Previous studies of virtual interactions have explored the simulation of social touch using haptic devices, but little is known about how the visual representation of interpersonal touch is perceived and integrated into a virtual social experience. In two studies we examined how the exchange of virtual touch mediated by simulated 3-dimensional human characters, or avatars, within an online virtual environment influenced affiliation towards an unfamiliar interaction partner. Surprisingly, the exchange of virtual touch negatively affected the perceived closeness and affiliation to the partner and the social evaluation of the interaction but did not affect the level of physiological arousal during the interaction. These results indicate that the visual representation of social touch is sufficient to virtually communicate touch-related cues that impact social affiliation, but the influence of touch may be dependent on the interaction context.

I am a Genius! Influence of Virtually Embodying Leonardo da Vinci on Creative Performance

Virtual reality (VR) provides users with the ability to substitute their physical appearance by embodying virtual characters (avatars) using head-mounted displays and motion-capture technologies. Previous research demonstrated that the sense of embodiment toward an avatar can impact user behavior and cognition. In this paper, we present an experiment designed to investigate whether embodying a well-known creative genius could enhance participants' creative performance. Following a preliminary online survey ( N = 157) to select a famous character suited to the purpose of this study, we developed a VR application allowing participants to embody Leonardo da Vinci or a self-avatar. Self-avatars were approximately matched with participants in terms of skin tone and morphology. 40 participants took part in three tasks seamlessly integrated in a virtual workshop. The first task was based on a Guilford's Alternate Uses test (GAU) to assess participants' divergent abilities in terms of fluency and originality. The second task was based on a Remote Associates Test (RAT) to evaluate convergent abilities. Lastly, the third task consisted in designing potential alternative uses of an object displayed in the virtual environment using a 3D sketching tool. Participants embodying Leonardo da Vinci demonstrated significantly higher divergent thinking abilities, with a substantial difference in fluency between the groups. Conversely, participants embodying a self-avatar performed significantly better in the convergent thinking task. Taken together, these results promote the use of our virtual embodiment approach, especially in applications where divergent creativity plays an important role, such as design and innovation.

A Systematic Review on the Visualization of Avatars and Agents in AR & VR displayed using Head-Mounted Displays

Augmented Reality (AR) and Virtual Reality (VR) are pushing from the labs towards consumers, especially with social applications. These applications require visual representations of humans and intelligent entities. However, displaying and animating photo-realistic models comes with a high technical cost while low-fidelity representations may evoke eeriness and overall could degrade an experience. Thus, it is important to carefully select what kind of avatar to display. This article investigates the effects of rendering style and visible body parts in AR and VR by adopting a systematic literature review. We analyzed 72 papers that compare various avatar representations. Our analysis includes an outline of the research published between 2015 and 2022 on the topic of avatars and agents in AR and VR displayed using head-mounted displays, covering aspects like visible body parts (e.g., hands only, hands and head, full-body) and rendering style (e.g., abstract, cartoon, realistic); an overview of collected objective and subjective measures (e.g., task performance, presence, user experience, body ownership); and a classification of tasks where avatars and agents were used into task domains (physical activity, hand interaction, communication, game-like scenarios, and education/training). We discuss and synthesize our results within the context of today's AR and VR ecosystem, provide guidelines for practitioners, and finally identify and present promising research opportunities to encourage future research of avatars and agents in AR/VR environments.

The Open Virtual Mirror Framework for enfacement illusions : Enhancing the sense of agency with avatars that imitate facial expressions

Enfacement illusions are traditionally elicited by visuo-tactile stimulation, but more active paradigms become possible through the usage of virtual reality techniques. For instance, virtual mirrors have been recently proposed to induce enfacement by visuo-motor stimulation. In a virtual mirror experiment, participants interact with an avatar that imitates their facial movements. The active control over the avatar greatly enhances the sense of agency, which is an important ingredient for successful enfacement illusion induction. Due to technological challenges, most virtual mirrors so far were limited to the imitation of the participant's head pose, i.e., its location and rotation. However, stronger experiences of agency can be expected by an increase in the avatar's mimicking abilities. We here present a new open-source framework for virtual mirror experiments, which we call the Open Virtual Mirror Framework (OVMF). The OVMF can track and imitate a large range of facial movements, including pose and expressions. It has been designed to run on standard computer hardware and easily interfaces with existing toolboxes for psychological experimentation, while satisfying the requirement of a tightly controlled experimental setup. Further, it is designed to enable convenient extension of its core functionality such that it can be flexibly adjusted to many different experimental paradigms. We demonstrate the usage of the OVMF and experimentally validate its ability to elicit experiences of agency over an avatar, concluding that the OVMF can serve as a reference for future experiments and that it provides high potential to stimulate new directions in enfacement research and beyond.

Impact of Personalized Avatars and Motion Synchrony on Embodiment and Users’ Subjective Experience: Empirical Study

Background

Embodiment through a virtual avatar is a key element for people to feel that they are in the virtual world.

Objective

This study aimed to elucidate the interaction between 2 methods of eliciting embodiment through a virtual avatar: motion synchronization and appearance similarity between a human and avatar, to understand embodiment (agency, body ownership, and self-location) and subjective experience (presence, simulator sickness, and emotion) in virtual reality.

Methods

Using a full-body motion capture system, 24 participants experienced their virtual avatars with a 3D-scanned face and size-matched body from a first-person perspective. This study used a 2 (motion; sync and async) × 2 (appearance; personalized and generic) within-subject design.

Results

The results indicated that agency and body ownership increased when motion and appearance were matched, whereas self-location, presence, and emotion were affected by motion only. Interestingly, if the avatar’s appearance was similar to the participants (personalized avatar), they formed an agency toward the avatar’s motion that was not performed by themselves.

Conclusions

Our findings would be applicable in the field of behavioral therapy, rehabilitation, and entertainment applications, by eliciting higher agency with a personalized avatar.

What is it like to be a bot? Variable perspective embodied telepresence for crowdsourcing robot movements

Movement and embodiment are communicative affordances central to social robotics, but designing embodied movements for robots often requires extensive knowledge of both robotics and movement theory. More accessible methods such as learning from demonstration often rely on physical access to the robot which is usually limited to research settings. Machine learning (ML) algorithms can complement hand-crafted or learned movements by generating new behaviors, but this requires large and diverse training datasets, which are hard to come by. In this work, we propose an embodied telepresence system for remotely crowdsourcing emotive robot movement samples that can serve as ML training data. Remote users control the robot through the internet using the motion sensors in their smartphones and view the movement either from a first-person or a third-person perspective. We evaluated the system in an online study where users created emotive movements for the robot and rated their experience. We then utilized the user-crafted movements as inputs to a neural network to generate new movements. We found that users strongly preferred the third-person perspective and that the ML-generated movements are largely comparable to the user-crafted movements. This work supports the usability of telepresence robots as a movement crowdsourcing platform.

Augmenting Immersive Telepresence Experience with a Virtual Body

We propose augmenting immersive telepresence by adding a virtual body, representing the user's own arm motions, as realized through a head-mounted display and a 360-degree camera. Previous research has shown the effectiveness of having a virtual body in simulated environments; however, research on whether seeing one's own virtual arms increases presence or preference for the user in an immersive telepresence setup is limited. We conducted a study where a host introduced a research lab while participants wore a head-mounted display which allowed them to be telepresent at the host's physical location via a 360-degree camera, either with or without a virtual body. We first conducted a pilot study of 20 participants, followed by a pre-registered 62 participant confirmatory study. Whereas the pilot study showed greater presence and preference when the virtual body was present, the confirmatory study failed to replicate these results, with only behavioral measures suggesting an increase in presence. After analyzing the qualitative data and modeling interactions, we suspect that the quality and style of the virtual arms, and the contrast between animation and video, led to individual differences in reactions to the virtual body which subsequently moderated feelings of presence.

A comparison of natural user interface and graphical user interface for narrative in HMD-based augmented reality

Over the years, the various mediums available for storytelling have progressively expanded, from spoken to written word, then to film, and now to Virtual Reality (VR) and Augmented Reality (AR). In 2016, the cutting-edge Head-Mounted Display (HMD) AR Microsoft HoloLens was released. However, though it has been several years, the quality of the user experience with narration using HMD-based AR technology has been rarely discussed. The present study explored interactive narrative in HMD-based AR regarding different user interfaces and their influence on users' presence, narrative engagement and reflection. Inspired by an existing exhibition at the National Holocaust Centre and Museum in the U.K., a HoloLens narrative application, entitled The AR Journey, was developed by the authors using two different interaction methods, Natural User Interface (NUI) and Graphical User Interface (GUI), which were used to perform an empirical study. As revealed from the results of the between-subject design experiment, NUI exhibited statistically significant advantages in creating presence for users without 3D Role Playing Game (RPG) experience, and GUI was superior in creating presence and increasing narrative engagement for users with 3D RPG experience. As indicated by the results of the interviews, the overall narrative experience in HMD-based AR was acceptable, and the branching narrative design was engaging. However, HoloLens hardware issues, as well as virtuality and reality mismatch, adversely affected user experience. Design guidelines were proposed according to the qualitative results.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s11042-021-11723-0.

Exploring Virtual Doppelgangers as Movement Models to Enhance Voluntary Imitation

Virtual Reality (VR) setups offer the possibility to investigate interactions between model and observer characteristics in imitation behavior, such as in the chameleon effect of automatic mimicry. We tested the hypothesis that perceived affiliative characteristics of a virtual model, such as similarity to the observer and likability, will facilitate observers' engagement in voluntary motor imitation. In a within-subjects design, participants were exposed to four virtual characters of different degrees of realism and observer similarity (avatar numbers AN=1-4), ranging from an abstract stickperson to a personalized doppelganger avatar designed from 3d scans of the observer. The characters performed different trunk movements and participants were asked to imitate these. We defined functional ranges of motion (ROM) for spinal extension (bending backward, BB), lateral flexion (bending sideward, BS) and rotation in the horizontal plane (RH) based on shoulder marker trajectories as behavioral indicators of imitation. Participants' ratings on avatar appearance, characteristics and embodiment/ enfacement were recorded in an Autonomous Avatar Questionnaire (AAQ), factorized into three sum scales based on our explorative analysis. Linear mixed effects models revealed that for lateral flexion (BS), a facilitating influence of avatar type on ROM was mediated by perceived identificatory avatar properties such as avatar likability, avatar-observer-similarity and other affiliative characteristics (AAQ1). This suggests that maximization of model-observer similarity with a virtual doppelganger may be useful in observational modeling and this could be used to modify maladaptive motor behaviors in patients with chronic back pain.

Psychological benefits of using social virtual reality platforms during the covid-19 pandemic: The role of social and spatial presence

Social virtual reality (VR) platforms are an emergent phenomenon, with growing numbers of users utilizing them to connect with others while experiencing feelings of presence (“being there”). This article examines the associations between feelings of presence and the activities performed by users, and the psychological benefits obtained in terms of relatedness, self-expansion, and enjoyment, in the context of the covid-19 pandemic. The results of a survey conducted among users (N = 220) indicate that feelings of spatial presence predict these three outcomes, while social presence predicts relatedness and enjoyment, but not self-expansion. Socialization activities like meeting friends in VR are associated with relatedness and enjoyment, while playful and creative activities allow for self-expansion. Moreover, the perceived impact of social distancing measures was associated with an increase in use, suggesting the utility of these platforms to help users meeting particularly frustrated psychological needs. These results provide a first quantitative account of the potential positive effects of social VR platforms on users’ wellbeing and encourage further research on the topic.

Virtual Reality as a Reflection Technique for Public Speaking Training

Video recording is one of the most commonly used techniques for reflection, because video allows people to know what they look like to others and how they could improve their performance, but it is problematic because some people easily fall into negative emotions and worry about their performance, resulting in a low benefit. In this study, the possibility of applying a simple VR-based reflection method was explored. This method uses virtual reality (VR) and a head-mounted display (HMD) to allow presenters to watch their own presentations from the audience’s perspective and uses an avatar, which hides personal appearance, which has low relevance to the quality of presentation, to help reduce self-awareness during reflection. An experimental study was carried out, considering four personal characteristics—gender, personal anxiety, personal confidence and self-bias. The goal of this study is to discuss which populations can benefit more from this system and to assess the impact of the avatar and HMD-based VR. According to the results, the individuals with low self-confidence in their public speaking skills could benefit more on self-evaluation from VR reflection with HMD, while individuals with negative self-bias could reduce more anxiety by using an avatar.

Avatar and Sense of Embodiment: Studying the Relative Preference Between Appearance, Control and Point of View

In Virtual Reality, a number of studies have been conducted to assess the influence of avatar appearance, avatar control and user point of view on the Sense of Embodiment (SoE) towards a virtual avatar. However, such studies tend to explore each factor in isolation. This paper aims to better understand the inter-relations among these three factors by conducting a subjective matching experiment. In the presented experiment ( n=40), participants had to match a given "optimal" SoE avatar configuration (realistic avatar, full-body motion capture, first-person point of view), starting by a "minimal" SoE configuration (minimal avatar, no control, third-person point of view), by iteratively increasing the level of each factor. The choices of the participants provide insights about their preferences and perception over the three factors considered. Moreover, the subjective matching procedure was conducted in the context of four different interaction tasks with the goal of covering a wide range of actions an avatar can do in a VE. The paper also describes a baseline experiment ( n=20) which was used to define the number and order of the different levels for each factor, prior to the subjective matching experiment (e.g. different degrees of realism ranging from abstract to personalised avatars for the visual appearance). The results of the subjective matching experiment show that point of view and control levels were consistently increased by users before appearance levels when it comes to enhancing the SoE. Second, several configurations were identified with equivalent SoE as the one felt in the optimal configuration, but vary between the tasks. Taken together, our results provide valuable insights about which factors to prioritize in order to enhance the SoE towards an avatar in different tasks, and about configurations which lead to fulfilling SoE in VE.

Toward Enhanced Teleoperation Through Embodiment

Telerobotics aims to transfer human manipulation skills and dexterity over an arbitrary distance and at an arbitrary scale to a remote workplace. A telerobotic system that is transparent enables a natural and intuitive interaction. We postulate that embodiment (with three sub-components: sense of ownership, agency, and self-location) of the robotic system leads to optimal perceptual transparency and increases task performance. However, this has not yet been investigated directly. We reason along four premises and present findings from the literature that substantiate each of them: (1) the brain can embody non-bodily objects (e.g., robotic hands), (2) embodiment can be elicited with mediated sensorimotor interaction, (3) embodiment is robust against inconsistencies between the robotic system and the operator's body, and (4) embodiment positively correlates to dexterous task performance. We use the predictive encoding theory as a framework to interpret and discuss the results reported in the literature. Numerous previous studies have shown that it is possible to induce embodiment over a wide range of virtual and real extracorporeal objects (including artificial limbs, avatars, and android robots) through mediated sensorimotor interaction. Also, embodiment can occur for non-human morphologies including for elongated arms and a tail. In accordance with the predictive encoding theory, none of the sensory modalities is critical in establishing ownership, and discrepancies in multisensory signals do not necessarily lead to loss of embodiment. However, large discrepancies in terms of multisensory synchrony or visual likeness can prohibit embodiment from occurring. The literature provides less extensive support for the link between embodiment and (dexterous) task performance. However, data gathered with prosthetic hands do indicate a positive correlation. We conclude that all four premises are supported by direct or indirect evidence in the literature, suggesting that embodiment of a remote manipulator may improve dexterous performance in telerobotics. This warrants further implementation testing of embodiment in telerobotics. We formulate a first set of guidelines to apply embodiment in telerobotics and identify some important research topics.