A fully immersive set-up for remote interaction and neurorehabilitation based on virtual body ownership

Perception and control of a virtual body in immersive virtual reality for rehabilitation

PURPOSE OF REVIEW

This review explores recent advances in using immersive virtual reality to improve bodily perception and motor control in rehabilitation across musculoskeletal and neurological conditions, examining how virtual reality's unique capabilities can address the challenges of traditional approaches. The potential in this area of the emerging metaverse and the integration of artificial intelligence in virtual reality are discussed.

RECENT FINDINGS

In musculoskeletal rehabilitation, virtual reality shows promise in enhancing motivation, adherence, improving range of motion, and reducing kinesiophobia, particularly postsurgery. For neurological conditions like stroke and spinal cord injury, virtual reality's ability to manipulate bodily perceptions offers significant therapeutic potential, with reported improvements in upper limb function and gait performance. Balance and gait rehabilitation, especially in older adults, have also seen positive outcomes. The integration of virtual reality with brain-computer interfaces presents exciting possibilities for severe speech and motor impairments.

SUMMARY

Current research is limited by small sample sizes, short intervention durations, and variability in virtual reality systems. Future studies should focus on larger, long-term trials to confirm findings and explore underlying mechanisms. As virtual reality technology advances, its integration into rehabilitation programs could revolutionize treatment approaches, personalizing treatments, facilitating home training, and potentially improving patient outcomes across a wide variety of conditions.

The Role of Virtual Reality in the Management of Irritable Bowel Syndrome

PURPOSE OF REVIEW

Irritable bowel syndrome (IBS) is a disorder of gut-brain interaction that significantly impacts health-related quality of life (HRQOL). This article explores the potential role of virtual reality (VR)-based cognitive behavioral therapy (CBT) in treating patients with IBS.

RECENT FINDINGS

While CBT is a proven, skills-based therapy approach that modifies behaviors and alters dysfunctional thinking patterns to influence the gut-brain axis and improve IBS symptoms, it is rarely prescribed given a paucity of CBT-trained clinicians. We developed a novel VR program that delivers a standardized CBT program over an 8-week period to help patients manage their symptoms. In initial qualitative validation testing, patients expressed positive perceptions about using VR CBT for IBS. Home-based, standardized VR CBT has the potential to be an effective and scalable treatment option for patients with IBS. While initial studies have shown proof-of-concept definitive randomized controlled trials are needed to demonstrate the efficacy of self-administered VR CBT in IBS.

Immersive virtual reality in orthopaedics-a narrative review

PURPOSE

This narrative review explores the applications and benefits of immersive virtual reality (VR) in orthopaedics, with a focus on surgical training, patient functional recovery, and pain management.

METHODS

The review examines existing literature and research studies on immersive VR in orthopaedics, analyzing both experimental and clinical studies.

RESULTS

Immersive VR provides a realistic simulation environment for orthopaedic surgery training, enhancing surgical skills, reducing errors, and improving overall performance. In post-surgical recovery and rehabilitation, immersive VR environments can facilitate motor learning and functional recovery through virtual embodiment, motor imagery during action observation, and virtual training. Additionally VR-based functional recovery programs can improve patient adherence and outcomes. Moreover, VR has the potential to revolutionize pain management, offering a non-invasive, drug-free alternative. Virtual reality analgesia acts by a variety of means including engagement and diverting patients' attention, anxiety reduction, and specific virtual-body transformations.

CONCLUSION

Immersive virtual reality holds significant promise in orthopaedics, demonstrating potential for improved surgical training, patient functional recovery, and pain management but further research is needed to fully exploit the benefits of VR technology in these areas.

Bill-EVR: An Embodied Virtual Reality Framework for Reward-and-Error-Based Motor Rehab-Learning

VR rehabilitation is an established field by now, however, it often refers to computer screen-based interactive rehabilitation activities. In recent years, there was an increased use of VR-headsets, which can provide an immersive virtual environment for real-world tasks, but they are lacking any physical interaction with the task objects and any proprioceptive feedback. Here, we focus on Embodied Virtual Reality (EVR), an emerging field where not only the visual input via VR-headset but also the haptic feedback is physically correct. This happens because subjects interact with physical objects that are veridically aligned in Virtual Reality. This technology lets us manipulate motor performance and motor learning through visual feedback perturbations. Bill-EVR is a framework that allows interventions in the performance of real-world tasks, such as playing pool billiard, engaging end-users in motivating life-like situations to trigger motor (re)learning - subjects see in VR and handle the real-world cue stick, the pool table and shoot physical balls. Specifically, we developed our platform to isolate and evaluate different mechanisms of motor learning to investigate its two main components, error-based and reward-based motor adaptation. This understanding can provide insights for improvements in neurorehabilitation: indeed, reward-based mechanisms are putatively impaired by degradation of the dopaminergic system, such as in Parkinson's disease, while error-based mechanisms are essential for recovering from stroke-induced movement errors. Due to its fully customisable features, our EVR framework can be used to facilitate the improvement of several conditions, providing a valid extension of VR-based implementations and constituting a motor learning tool that can be completely tailored to the individual needs of patients.

Using virtual reality to assess gesture performance deficits in schizophrenia patients

Introduction

Gesture performance deficits are prevalent in schizophrenia patients and are strongly associated with poor social communication skills and community functioning, affecting their overall quality of life. Currently, video-recording technology is widely used in clinical settings to assess gesture production deficits in schizophrenia patients. Nevertheless, the subjective evaluation of video-recordings can encumber task assessment. The present study will aim to use virtual reality to examine its potential use as an alternative tool to objectively measure gesture performance accuracy in schizophrenia patients and healthy controls.

Methods

Gesture performance in the virtual reality setting will be based on the well-established Test of Upper Limb Apraxia. Participants will be immersed in a virtual environment where they will experience themselves being embodied in a collocated virtual body seen from a first-person perspective. Motion trackers will be placed on participants' hands and elbows to track upper body movements in real-time, and to record gesture movement for later analysis. Participants will see a virtual agent sitting across from them, with a virtual table in between. The agent will perform various types of gestures and the participants' task will be to imitate those gestures as accurately as possible. Measurements from the tracking devices will be stored and analyzed to address gesture performance accuracy across groups.

Discussion

This study aims to provide objective measurements of gesture performance accuracy in schizophrenia patients. If successful, the results will provide new knowledge to the gesture literature and offer the potential for novel therapeutic interventions using virtual reality technologies. Such interventions can improve gesturing and thus advance social communication skills in schizophrenia patients.

Survey of Movement Reproduction in Immersive Virtual Rehabilitation

Virtual reality (VR) has emerged as a powerful tool for rehabilitation. Many effective VR applications have been developed to support motor rehabilitation of people affected by motor issues. Movement reproduction, which transfers users' movements from the physical world to the virtual environment, is commonly used in VR rehabilitation applications. Three major components are required for movement reproduction in VR: (1) movement input, (2) movement representation, and (3) movement modulation. Until now, movement reproduction in virtual rehabilitation has not yet been systematically studied. This article aims to provide a state-of-the-art review on this subject by focusing on existing literature on immersive motor rehabilitation using VR. In this review, we provided in-depth discussions on the rehabilitation goals and outcomes, technology issues behind virtual rehabilitation, and user experience regarding movement reproduction. Similarly, we present good practices and highlight challenges and opportunities that can form constructive suggestions for the design and development of fit-for-purpose VR rehabilitation applications and can help frame future research directions for this emerging area that combines VR and health.

Effect of immersive visualization technologies on cognitive load, motivation, usability, and embodiment

Virtual reality (VR) is a promising tool to promote motor (re)learning in healthy users and brain-injured patients. However, in current VR-based motor training, movements of the users performed in a three-dimensional space are usually visualized on computer screens, televisions, or projection systems, which lack depth cues (2D screen), and thus, display information using only monocular depth cues. The reduced depth cues and the visuospatial transformation from the movements performed in a three-dimensional space to their two-dimensional indirect visualization on the 2D screen may add cognitive load, reducing VR usability, especially in users suffering from cognitive impairments. These 2D screens might further reduce the learning outcomes if they limit users' motivation and embodiment, factors previously associated with better motor performance. The goal of this study was to evaluate the potential benefits of more immersive technologies using head-mounted displays (HMDs). As a first step towards potential clinical implementation, we ran an experiment with 20 healthy participants who simultaneously performed a 3D motor reaching and a cognitive counting task using: (1) (immersive) VR (IVR) HMD, (2) augmented reality (AR) HMD, and (3) computer screen (2D screen). In a previous analysis, we reported improved movement quality when movements were visualized with IVR than with a 2D screen. Here, we present results from the analysis of questionnaires to evaluate whether the visualization technology impacted users' cognitive load, motivation, technology usability, and embodiment. Reports on cognitive load did not differ across visualization technologies. However, IVR was more motivating and usable than AR and the 2D screen. Both IVR and AR rea ched higher embodiment level than the 2D screen. Our results support our previous finding that IVR HMDs seem to be more suitable than the common 2D screens employed in VR-based therapy when training 3D movements. For AR, it is still unknown whether the absence of benefit over the 2D screen is due to the visualization technology per se or to technical limitations specific to the device.

[Virtual reality technology as a promising direction in neurorehabilitation]

Nervous system damage affects more than a billion people worldwide and is one of the leading causes of cognitive impairment. An urgent issue in modern medicine is the neurorehabilitation of this particular group of patients. The purpose of this article is to search for new approaches to achieve more effective recovery of cognitive functions, precisely by using virtual reality technology as a promising direction in neurorehabilitation. It has been shown that neurobiological effects of virtual reality have a positive effect on the plasticity of neurons, improve cognitive functions and positively affect the psychoemotional state. A case of the positive impact of being in the virtual environment «Outer Space» in a female patient with cognitive impairment and chronic pain is presented.

Altered bodily perceptions in chronic neuropathic pain conditions and implications for treatment using immersive virtual reality

Chronic neuropathic pain is highly disabling and difficult to treat and manage. Patients with such conditions often report altered bodily perceptions that are thought to be associated with maladaptive structural and functional alterations in the somatosensory cortex. Manipulating these altered perceptions using body illusions in virtual reality is being investigated and may have positive clinical implications for the treatment of these conditions. Here, we have conducted a narrative review of the evidence for the types of bodily distortions associated with a variety of peripheral and central neuropathic pain conditions. In addition, we summarize the experimental and clinical studies that have explored embodiment and body transformation illusions in immersive virtual reality for neuropathic pain relief, which are thought to target these maladaptive changes, as well as suggesting directions for future research.

Adversarial Autoencoder and Multi-Armed Bandit for Dynamic Difficulty Adjustment in Immersive Virtual Reality for Rehabilitation: Application to Hand Movement

Motor rehabilitation is used to improve motor control skills to improve the patient’s quality of life. Regular adjustments based on the effect of therapy are necessary, but this can be time-consuming for the clinician. This study proposes to use an efficient tool for high-dimensional data by considering a deep learning approach for dimensionality reduction of hand movement recorded using a wireless remote control embedded with the Oculus Rift S. This latent space is created as a visualization tool also for use in a reinforcement learning (RL) algorithm employed to provide a decision-making framework. The data collected consists of motions drawn with wireless remote control in an immersive VR environment for six different motions called “Cube”, “Cylinder”, “Heart”, “Infinity”, “Sphere”, and “Triangle”. From these collected data, different artificial databases were created to simulate variations of the data. A latent space representation is created using an adversarial autoencoder (AAE), taking into account unsupervised (UAAE) and semi-supervised (SSAAE) training. Then, each test point is represented by a distance metric and used as a reward for two classes of Multi-Armed Bandit (MAB) algorithms, namely Boltzmann and Sibling Kalman filters. The results showed that AAE models can represent high-dimensional data in a two-dimensional latent space and that MAB agents can efficiently and quickly learn the distance evolution in the latent space. The results show that Sibling Kalman filter exploration outperforms Boltzmann exploration with an average cumulative weighted probability error of 7.9 versus 19.9 using the UAAE latent space representation and 8.0 versus 20.0 using SSAAE. In conclusion, this approach provides an effective approach to visualize and track current motor control capabilities regarding a target in order to reflect the patient’s abilities in VR games in the context of DDA.

Impact of virtual embodiment and exercises on functional ability and range of motion in orthopedic rehabilitation

Recent evidence supports the use of immersive virtual reality (immersive VR) as a means of applying visual feedback techniques in neurorehabilitation. In this study, we investigated the benefits of an embodiment-based immersive VR training program for orthopedic upper limb rehabilitation, with the aim of improving the motor functional ability of the arm and accelerating the rehabilitation process in patients with a conservatively managed distal radius fracture. We designed a rehabilitation program based on developing ownership over a virtual arm and then exercising it in immersive VR. We carried out a between 3-group controlled trial with 54 patients (mean age = 61.80 ± 14.18): 20 patients were assigned to the experimental training group (immersive VR), 20 to the conventional digit mobilization (CDM) training control group, and 14 to a non-immersive (non-immersive VR) training control group. We found that functional recovery of the arm in the immersive VR group was correlated with the ownership and agency scores over the virtual arm. We also found larger range of joint movements and lower disability of the fractured arm compared with patients in the Non-immersive VR and CDM groups. Feeling embodied in a virtual body can be used as a rehabilitation tool to speed up and improve motor functional recovery of a fractured arm after the immobilization period.

Design, development, and testing of a virtual reality device for upper limb training in people with multiple sclerosis: a feasibility study (Preprint)

Background

Multiple sclerosis (MS) is a common nontraumatic, neurological, disabling disease that often presents with upper limb dysfunction. Exercise training has resulted in improvement for patients; however, there can be a lack of compliance due to access because of location and lack of MS experts. Virtual reality (VR) is a promising technology that can offer exercise therapy/rehabilitation at a distance. This type of remote training can be motivational and effective for patients with MS and can improve range of motion and muscle strength for those with upper limb dysfunction.

Objective

The aim of this study is to evaluate the safety and feasibility of the XRHealth software and the Oculus Rift Station for patients with MS with upper limb motor dysfunction.

Methods

A single-center, prospective, feasibility study was conducted with patients with MS who had upper limb motor dysfunction. Patients participated in a single 45-minute digital environment session with VR and completed a questionnaire about the quality of the training and fatigability. The clinician also completed a questionnaire to evaluate the suitability and safety of the training.

Results

Overall, 30 patients were enrolled between the ages of 20 and 81 years. Patients reported that the training sessions within the digital environment were helpful, challenging, fun, and simple to understand, and that they would be willing to repeat the sessions again. The physical therapist that oversaw the patients reported that the training was suitable for 87% (n=26) of the patients. Anticipated adverse events were fatigue, temporary dizziness, and temporary nausea. The operator complications included that the cable of the head-mounted display interrupted the training (n=2, 7%) and fatigue that caused cessation of the VR training session (n=2, 7%). No serious adverse events were reported.

Conclusions

These preliminary results demonstrated that the use of the XRHealth software and Oculus Rift Station platform is feasible, safe, and engaging for patients, and has the potential to improve the functionality of the upper limbs in patients with MS. This study provides support for future studies of implementing a series of training sessions with virtual reality in a home-based environment.

Telemedicine and Virtual Reality at Time of COVID-19 Pandemic: An Overview for Future Perspectives in Neurorehabilitation

In catastrophic situations such as pandemics, patients' healthcare including admissions to hospitals and emergency services are challenged by the risk of infection and by limitations of healthcare resources. In such a setting, the use of telemedicine interventions has become extremely important. New technologies have proved helpful in pandemics as a solution to improve the quality of life in vulnerable patients such as persons with neurological diseases. Moreover, telemedicine interventions provide at-home solutions allowing clinicians to telemonitor and assess patients remotely, thus minimizing risk of infection. After a review of different studies using telemedicine in neurological patients, we propose a telemedicine process flow for healthcare of subjects with chronic neurological disease to respond to the new challenges for delivering quality healthcare during the transformation of public and private healthcare organizations around the world forced by COVID-19 pandemic contingency. This telemedicine process flow represents a replacement for in-person treatment and thereby the provision equitable access to the care of vulnerable people. It is conceptualized as comprehensive service including (1) teleassistance with patient counseling and medical treatment, (2) telemonitoring of patients' health conditions and any changes over time, as well as (3) telerehabilitation, i.e., interventions to assess and promote body functions, activities, and consecutively participation. The hereby proposed telemedicine process flow could be adopted on a large scale to improve the public health response during healthcare crises like the COVID-19 pandemic but could equally promote equitable health care independent of people's mobility or location with respect to the specialized health care center.

Virtual Reality as a Technological-Aided Solution to Support Communication in Persons With Neurodegenerative Diseases and Acquired Brain Injury During COVID-19 Pandemic

The COVID-19 poses an ongoing threat to lives around the world and challenges the existing public health and medical service delivery. The lockdown or quarantine measures adopted to prevent the spread of COVID-19 has caused the interruption in ongoing care and access to medical care including to patients with existing neurological conditions. Besides the passivity, isolation, and withdrawal, patients with neurodegenerative diseases experience difficulties in communication due to a limited access to leisure opportunities and interaction with friends and relatives. The communication difficulties may exacerbate the burden on the caregivers. Therefore, assistive-technologies may be a useful strategy in mitigating challenges associated with remote communication. The current paper presents an overview of the use of assistive technologies using virtual reality and virtual body ownership in providing communication opportunities to isolated patients, during COVID-19, with neurological diseases and moderate-to-severe communication difficulties. We postulate that the assistive technologies-based intervention may improve social interactions in patients with neurodegenerative diseases and acquired brain injury-thereby reducing isolation and improving their quality of life and mental well-being.

Using Virtual Reality as a Tool in the Rehabilitation of Movement Abnormalities in Schizophrenia

Movement abnormalities are prevalent across all stages of schizophrenia contributing to poor social functioning and reduced quality of life. To date, treatments are scarce, often involving pharmacological agents, but none have been shown to improve movement abnormalities effectively. Virtual reality (VR) is a tool used to simulate virtual environments where behavioral performance can be quantified safely across different tasks while exerting control over stimulus delivery, feedback and measurement in real time. Sensory information is transmitted via a head mounted display allowing users to directly interact with virtual objects and bodies using gestures and body movements in the real world to perform different actions, permitting a sense of immersion in the simulated virtual environment. Although, VR has been widely used for successful motor rehabilitation in a variety of different neurological domains, none have been exploited for motor rehabilitation in schizophrenia. The objectives of this article are to review movement abnormalities specific to schizophrenia, and how VR can be utilized to restore and improve motor functioning in patients with schizophrenia. Constructing VR-mediated motor-cognitive interventions that can help in retaining and transferring the learned outcomes to real life are also discussed.

Being the Victim of Intimate Partner Violence in Virtual Reality: First- Versus Third-Person Perspective

Immersive virtual reality is widely used for research and clinical purposes. Here we explored the impact of an immersive virtual scene of intimate partner violence experienced from the victim’s perspective (first person), as opposed to witnessing it as an observer (third person). We are ultimately interested in the potential of this approach to rehabilitate batterers and in understanding the mechanisms underlying this process. For this, non-offender men experienced the scene either from the perspective of the victim’s virtual body (a female avatar), which moved synchronously with the participants’ real movements, or from the perspective of an observer, while we recorded their behavior and physiological responses. We also evaluated through questionnaires, interviews and implicit association tests their subjective impressions and potential pre/post changes in implicit gender bias following the experience. We found that in all participants, regardless of perspective, the magnitude of the physiological reactions to virtual threatening stimuli was related to how vulnerable they felt for being a woman, the sensation that they could be assaulted, how useful the scene could be for batterer rehabilitation, and how different it would have been to experience the scenario on TV. Furthermore, we found that their level of identification with the female avatar correlated with the decrease in prejudice against women. Although the first-person perspective (1PP) facilitated taking the scene personally, generated a sensation of fear, helplessness, and vulnerability, and tended to induce greater behavioral and physiological reactions, we show that the potential for batterer rehabilitation originates from presence and identification with the victim, which in turn is more easily, but not exclusively, achieved through 1PP. This study is relevant for the development of advanced virtual reality tools for clinical purposes.

Brain-computer interfaces and virtual reality for neurorehabilitation

Brain-computer interfaces (BCIs) and virtual reality (VR) are two technologic advances that are changing our way of interacting with the world. BCIs can be used to influence and can serve as a control mechanism in navigation tasks, communication, or other assistive functions. VR can create ad hoc interactive scenarios that involve all our senses, stimulate the brain in a multisensory fashion, and increase the motivation and fun with game-like environments. VR and motion tracking enable natural human-computer interaction at cognitive and physical levels. This includes both brain and body in the design of meaningful VR experiences; these cases in which participants feel naturally present could help augment the benefits of BCIs for assistive and neurorehabilitation applications for the relearning of motor and cognitive skills. VR technology is now available at the consumer level thanks to the proliferation of affordable head-mounted displays (HMDs). Merging both technologies into simplified, practical devices may help democratize these technologies.

The Effects of Motion Artifacts on Self-Avatar Agency

One way of achieving self-agency in virtual environments is by using a motion capture system and retargeting user’s motion to the virtual avatar. In this study, we investigated whether the self-agency is affected when motion artifacts appear on top of the baseline motion capture data assigned to the self-avatar. For this experiment, we implemented four artifacts: noise, latency, motion jump, and offset rotation of joints. The data provided directly from the motion capture system formed the baseline of the study. We developed three observation tasks to assess self-agency: self-observation, observation through a virtual mirror, and observation during locomotion. A questionnaire was adopted and used to capture the self-agency of participants. We analyzed the collected responses of participants to determine whether the motion artifacts significantly altered the participants’ sense of self-agency. The obtained results indicated that participants are not always sensitive to the motion artifacts assigned to the self-avatar, but the sense of self-agency is dependent on the observation task they were asked to perform. Implications for further research are discussed.

Virtual reality experiences, embodiment, videogames and their dimensions in neurorehabilitation

Background

In the context of stroke rehabilitation, new training approaches mediated by virtual reality and videogames are usually discussed and evaluated together in reviews and meta-analyses. This represents a serious confounding factor that is leading to misleading, inconclusive outcomes in the interest of validating these new solutions.

Main body

Extending existing definitions of virtual reality, in this paper I put forward the concept of virtual reality experience (VRE), generated by virtual reality systems (VRS; i.e. a group of variable technologies employed to create a VRE). Then, I review the main components composing a VRE, and how they may purposely affect the mind and body of participants in the context of neurorehabilitation. In turn, VRS are not anymore exclusive from VREs but are currently used in videogames and other human-computer interaction applications in different domains. Often, these other applications receive the name of virtual reality applications as they use VRS. However, they do not necessarily create a VRE. I put emphasis on exposing fundamental similarities and differences between VREs and videogames for neurorehabilitation. I also recommend describing and evaluating the specific features encompassing the intervention rather than evaluating virtual reality or videogames as a whole.

Conclusion

This disambiguation between VREs, VRS and videogames should help reduce confusion in the field. This is important for databases searches when looking for specific studies or building metareviews that aim at evaluating the efficacy of technology-mediated interventions.

Virtual Reality as a Vehicle to Empower Motor-Cognitive Neurorehabilitation

In this paper, we advocate the combination of four key ingredients that we believe are necessary to design long-lasting effective treatments for neurorehabilitation: (i) motor-cognitive training, (ii) evidence-based neuroscience principles, in particular those related to body perception, (iii) motivational games, and (iv) empowerment techniques. Then, we propose virtual reality (VR) as the appropriate medium to encompass all the requirements mentioned above. VR is arguably one of the most suitable technologies for neurorehabilitation able to integrate evidence-based neurorehabilitation techniques and neuroscience principles into motivating training approaches that promote self-management by empowering patients to own their recovery process. We discuss the advantages and challenges of such an approach on several exemplary applications and outline directions for future developments. We strongly believe that the combination of positive psychology and positive technology mediated by VR-based interventions can heavily impact the rehabilitation outcomes of motor-cognitive functions along all the stages of the rehabilitation path.

Classification of Mental Tasks from EEG Signals Using Spectral Analysis, PCA and SVM

Signals provided by the ElectroEncephaloGraphy (EEG) are widely used in Brain-Computer Interface (BCI) applications. They can be further analyzed and used for thinking activity recognition. In this paper we proposed an algorithm that is able to recognize five mental tasks using 6 channel EEG data. The main idea is to separate the raw EEG signals into several frames and compute their spectrums. Next, a second-order derivative of Gaussian is applied to extract features and an optimum Gaussian kernel parameters grid search is performed with the help of cross-validation. The extracted features are further reduced by Principal Component Analysis. The processed data is utilized to train SVM classifier which is used for mental tasks recognition afterwards. The performance of the algorithm is estimated on publically available dataset. In terms of 5 folds cross-validation we obtained an average of 82.7% recognition rate (accuracy). Additional experiments were conducted using leave-one-out cross-validation where 67.2% correct classification was reported. Comparison to several state-of-the art methods reveals the advantages of the proposed algorithm.

Beaming into the News: A System for and Case Study of Tele-Immersive Journalism

We show how a combination of virtual reality and robotics can be used to beam a physical representation of a person to a distant location, and describe an application of this system in the context of journalism. Full body motion capture data of a person is streamed and mapped in real time, onto the limbs of a humanoid robot present at the remote location. A pair of cameras in the robots eyes stream stereoscopic video back to the HMD worn by the visitor, and a two-way audio connection allows the visitor to talk to people in the remote destination. By fusing the multisensory data of the visitor with the robot, the visitors consciousness is transformed to the robots body. This system was used by a journalist to interview a neuroscientist and a chef 900 miles distant, about food for the brain, resulting in an article published in the popular press.

The Effects of Avatars, Stereo Vision and Display Size on Reaching and Motion Reproduction

Thanks to recent advances on motion capture devices and stereoscopic consumer displays, animated virtual characters can now realistically interact with users in a variety of applications. We investigate in this paper the effect of avatars, stereo vision and display size on task execution in immersive virtual environments. We report results obtained with three experiments in varied configurations that are commonly used in rehabilitation applications. The first experiment analyzes the accuracy of reaching tasks under different system configurations: with and without an avatar, with and without stereo vision, and employing a 2D desktop monitor versus a large multi-tile visualization display. The second experiment analyzes the use of avatars and user-perspective stereo vision on the ability to perceive and subsequently reproduce motions demonstrated by an autonomous virtual character. The third experiment evaluates the overall user experience with a complete immersive user interface for motion modeling by direct demonstration. Our experiments expose and quantify the benefits of using stereo vision and avatars, and show that the use of avatars improve the quality of produced motions and the resemblance of replicated msotions; however, direct interaction in user-perspective leads to tasks executed in less time and to targets more accurately reached. These and additional tradeoffs are important for the effective design of avatar-based training systems.

Assessment and rehabilitation of neglect using virtual reality: a systematic review

After experiencing a stroke in the right hemisphere, almost 50% of patients showed Unilateral Spatial Neglect (USN). In recent decades, Virtual Reality (VR) has been used as an effective tool both for the assessment and rehabilitation of USN. Indeed, this advanced technology allows post-stroke patients to interact with ecological and engaging environments similar to real ones, but in a safe and controlled way. To provide an overview of the most recent VR applications for the assessment and rehabilitation of USN, a systematic review has been carried out. Since 2010, 13 studies have proposed and tested innovative VR tools for USN. After a wide description of the selected studies, we discuss the main features of these VR tools in order to provide crucial indications for future studies, neurorehabilitation interventions, and clinical practice.

Studying social interactions through immersive virtual environment technology: virtues, pitfalls, and future challenges

The goal of the present review is to explain how immersive virtual environment technology (IVET) can be used for the study of social interactions and how the use of virtual humans in immersive virtual environments can advance research and application in many different fields. Researchers studying individual differences in social interactions are typically interested in keeping the behavior and the appearance of the interaction partner constant across participants. With IVET researchers have full control over the interaction partners, can standardize them while still keeping the simulation realistic. Virtual simulations are valid: growing evidence shows that indeed studies conducted with IVET can replicate some well-known findings of social psychology. Moreover, IVET allows researchers to subtly manipulate characteristics of the environment (e.g., visual cues to prime participants) or of the social partner (e.g., his/her race) to investigate their influences on participants’ behavior and cognition. Furthermore, manipulations that would be difficult or impossible in real life (e.g., changing participants’ height) can be easily obtained with IVET. Beside the advantages for theoretical research, we explore the most recent training and clinical applications of IVET, its integration with other technologies (e.g., social sensing) and future challenges for researchers (e.g., making the communication between virtual humans and participants smoother).

Beaming: an asymmetric telepresence system

The Beaming project recreates, virtually, a real environment; using immersive VR, remote participants can visit the virtual model and interact with the people in the real environment. The real environment doesn't need extensive equipment and can be a space such as an office or meeting room, domestic environment, or social space.

Beaming into the rat world: enabling real-time interaction between rat and human each at their own scale

Immersive virtual reality (IVR) typically generates the illusion in participants that they are in the displayed virtual scene where they can experience and interact in events as if they were really happening. Teleoperator (TO) systems place people at a remote physical destination embodied as a robotic device, and where typically participants have the sensation of being at the destination, with the ability to interact with entities there. In this paper, we show how to combine IVR and TO to allow a new class of application. The participant in the IVR is represented in the destination by a physical robot (TO) and simultaneously the remote place and entities within it are represented to the participant in the IVR. Hence, the IVR participant has a normal virtual reality experience, but where his or her actions and behaviour control the remote robot and can therefore have physical consequences. Here, we show how such a system can be deployed to allow a human and a rat to operate together, but the human interacting with the rat on a human scale, and the rat interacting with the human on the rat scale. The human is represented in a rat arena by a small robot that is slaved to the human’s movements, whereas the tracked rat is represented to the human in the virtual reality by a humanoid avatar. We describe the system and also a study that was designed to test whether humans can successfully play a game with the rat. The results show that the system functioned well and that the humans were able to interact with the rat to fulfil the tasks of the game. This system opens up the possibility of new applications in the life sciences involving participant observation of and interaction with animals but at human scale.