The Science Behind Virtual Reality Displays

Lightness constancy in reality, in virtual reality, and on flat-panel displays

Virtual reality (VR) displays are being used in an increasingly wide range of applications. However, previous work shows that viewers often perceive scene properties very differently in real and virtual environments and so realistic perception of virtual stimuli should always be a carefully tested conclusion, not an assumption. One important property for realistic scene perception is surface color. To evaluate how well virtual platforms support realistic perception of achromatic surface color, we assessed lightness constancy in a physical apparatus with real lights and surfaces, in a commercial VR headset, and on a traditional flat-panel display. We found that lightness constancy was good in all three environments, though significantly better in the real environment than on the flat-panel display. We also found that variability across observers was significantly greater in VR and on the flat-panel display than in the physical environment. We conclude that these discrepancies should be taken into account in applications where realistic perception is critical but also that in many cases VR can be used as a flexible alternative to flat-panel displays and a reasonable proxy for real environments.

Linear perspective cues have a greater effect on the perceptual rescaling of distant stimuli than textures in the virtual environment

The presence of pictorial depth cues in virtual environments is important for minimising distortions driven by unnatural viewing conditions (e.g., vergence-accommodation conflict). Our aim was to determine how different pictorial depth cues affect size constancy in virtual environments under binocular and monocular viewing conditions. We systematically removed linear perspective cues and textures of a hallway in a virtual environment. The experiment was performed using the method of constant stimuli. The task required participants to compare the size of 'far' (10 m) and 'near' (5 m) circles displayed inside a virtual environment with one or both or none of the pictorial depth cues. Participants performed the experiment under binocular and monocular viewing conditions while wearing a virtual reality headset. ANOVA revealed that size constancy was greater for both the far and the near circles in the virtual environment with pictorial depth cues compared to the one without cues. However, the effect of linear perspective cues was stronger than textures, especially for the far circle. We found no difference between the binocular and monocular viewing conditions across the different virtual environments. We conclude that linear perspective cues exert a stronger effect than textures on the perceptual rescaling of far stimuli placed in the virtual environment, and that this effect does not vary between binocular and monocular viewing conditions.

Exploring the potential role for extended reality in Mohs micrographic surgery

Mohs micrographic surgery (MMS) is a cornerstone of dermatological practice. Virtual reality (VR) and augmented reality (AR) technology, initially used for entertainment, have entered healthcare, offering real-time data overlaying a surgeon's view. This paper explores potential applications of VR and AR in MMS, emphasizing their advantages and limitations. We aim to identify research gaps to facilitate innovation in dermatological surgery. We conducted a PubMed search using the following: "augmented reality" OR "virtual reality" AND "Mohs" or "augmented reality" OR "virtual reality" AND "surgery." Inclusion criteria were peer-reviewed articles in English discussing these technologies in medical settings. We excluded non-peer-reviewed sources, non-English articles, and those not addressing these technologies in a medical context. VR alleviates patient anxiety and enhances patient satisfaction while serving as an educational tool. It also aids physicians by providing realistic surgical simulations. On the other hand, AR assists in real-time lesion analysis, optimizing incision planning, and refining margin control during surgery. Both of these technologies offer remote guidance for trainee residents, enabling real-time learning and oversight and facilitating synchronous teleconsultations. These technologies may transform dermatologic surgery, making it more accessible and efficient. However, further research is needed to validate their effectiveness, address potential challenges, and optimize seamless integration. All in all, AR and VR enhance real-world environments with digital data, offering real-time surgical guidance and medical insights. By exploring the potential integration of these technologies in MMS, our study identifies avenues for further research to thoroughly understand the role of these technologies to redefine dermatologic surgery, elevating precision, surgical outcomes, and patient experiences.

Evaluation of a Virtual Reality-Based Open Educational Resource Software

OBJECTIVES

Virtual reality (VR) teaching methods have potential to support medical students acquire increasing amounts of knowledge. EVENT (Easy VR EducatioN Tool) is an open educational resource software for immersive VR environments, which is designed for use without programming skills. In this work, EVENT was used in a medical student VR course on pancreatic cancer.

METHODS

Medical students were invited to participate in the course. Before and after VR simulation, participants completed a multiple-choice knowledge assessment, with a maximum score of 10, and a VR experience questionnaire. The primary endpoint compared pre- and post-VR simulation test scores. Secondary endpoints included usability and factors that could affect learning growth and test results.

RESULTS

Data from 117 of the 135 participating students was available for analysis. Student test scores improved by an average of 3.4 points (95% CI 3.1-3.7, P < 0.001) after VR course. The secondary endpoints of gender, age, prior knowledge regarding the medical subject, professional training completed in the medical field, video game play, three-dimensional imagination skills, or cyber-sickness had no major impact on test scores or final ranking (top or bottom 25%). The 27 students whose post-VR simulation test scores ranked in the top 25% had no prior experience with VR. The average System Usability Scale score was 86.1, which corresponds to an excellent outcome for user-friendliness. Questionnaire responses post-VR simulation show students (81.2% [95/117]) interest in more VR options in medical school.

CONCLUSIONS

We present a freely available software that allows for the development of VR teaching lessons without programming skills.

Patient experience of a virtual reality calm room in a psychiatric inpatient care setting in Sweden: a qualitative study with inpatients

OBJECTIVE

Calm rooms have been developed and implemented in psychiatric inpatient care settings to offer patients a dedicated space for relaxation in a convenient and safe environment. Recent technology developments have enabled virtual reality (VR) equivalents of calm rooms that can be feasibly deployed in psychiatric care settings. While research has shown VR environments to be efficacious in inducing relaxation, little is known how these virtual calm rooms are perceived by patients. The aim of this study was to elucidate patient experiences of using a VR calm room in a psychiatric inpatient setting.

DESIGN

Qualitative interview study. Semi-structured interviews were analysed using qualitive inductive content analysis, which focuses on the interpretation of texts for making replicable and valid inferences.

SETTING

Swedish hospital psychiatric inpatient care setting with a wireless, three degrees-of-freedom VR head-mounted display running a calm room application simulating nature environment.

PARTICIPANTS

20 adult patients (12 women) with bipolar disorder (n=18) or unipolar depression (n=2).

RESULTS

Participants experienced the use of the VR calm room as having a positive impact on them, inducing awareness, calmness and well-being. They were thankful to be offered a non-pharmacological alternative for anxiety relief. Participants also expressed that they had some concerns about how they would react emotionally before using the VR device. However, after use, they highlighted that their overall experience was positive. They also expressed that they could see potential for further development of VR technology in psychiatric care.

CONCLUSIONS

VR technology has the potential to solve pressing logistic issues in offering calm rooms in psychiatric inpatient care. VR calm rooms appear to be appreciated by psychiatric inpatients, who value their accessibility, convenience and variety of modalities offered. Participants perceived an increase in their well-being after use.

Measuring motion-to-photon latency for sensorimotor experiments with virtual reality systems

Consumer virtual reality (VR) systems are increasingly being deployed in research to study sensorimotor behaviors, but properties of such systems require verification before being used as scientific tools. The 'motion-to-photon' latency (the lag between a user making a movement and the movement being displayed within the display) is a particularly important metric as temporal delays can degrade sensorimotor performance. Extant approaches to quantifying this measure have involved the use of bespoke software and hardware and produce a single measure of latency and ignore the effect of the motion prediction algorithms used in modern VR systems. This reduces confidence in the generalizability of the results. We developed a novel, system-independent, high-speed camera-based latency measurement technique to co-register real and virtual controller movements, allowing assessment of how latencies change through a movement. We applied this technique to measure the motion-to-photon latency of controller movements in the HTC Vive, Oculus Rift, Oculus Rift S, and Valve Index, using the Unity game engine and SteamVR. For the start of a sudden movement, all measured headsets had mean latencies between 21 and 42 ms. Once motion prediction could account for the inherent delays, the latency was functionally reduced to 2-13 ms, and our technique revealed that this reduction occurs within ~25-58 ms of movement onset. Our findings indicate that sudden accelerations (e.g., movement onset, impacts, and direction changes) will increase latencies and lower spatial accuracy. Our technique allows researchers to measure these factors and determine the impact on their experimental design before collecting sensorimotor data from VR systems.

No pain, no gain? The effects of adding a pain stimulus in virtual training for police officers

Virtual training systems provide highly realistic training environments for police. This study assesses whether a pain stimulus can enhance the training responses and sense of the presence of these systems. Police officers (n = 219) were trained either with or without a pain stimulus in a 2D simulator (VirTra V-300) and a 3D virtual reality (VR) system. Two (training simulator) × 2 (pain stimulus) ANOVAs revealed a significant interaction effect for perceived stress (p = .010, ηp2 = .039). Post-hoc pairwise comparisons showed that VR provokes significantly higher levels of perceived stress compared to VirTra when no pain stimulus is used (p = .009). With a pain stimulus, VirTra training provokes significantly higher levels of perceived stress compared to VirTra training without a pain stimulus (p < .001). Sense of presence was unaffected by the pain stimulus in both training systems. Our results indicate that VR training appears sufficiently realistic without adding a pain stimulus. Practitioner summary: Virtual police training benefits from highly realistic training environments. This study found that adding a pain stimulus heightened perceived stress in a 2D simulator, whereas it influenced neither training responses nor sense of presence in a VR system. VR training appears sufficiently realistic without adding a pain stimulus.

Design guidelines for limiting and eliminating virtual reality-induced symptoms and effects at work: a comprehensive, factor-oriented review

Virtual reality (VR) can induce side effects known as virtual reality-induced symptoms and effects (VRISE). To address this concern, we identify a literature-based listing of these factors thought to influence VRISE with a focus on office work use. Using those, we recommend guidelines for VRISE amelioration intended for virtual environment creators and users. We identify five VRISE risks, focusing on short-term symptoms with their short-term effects. Three overall factor categories are considered: individual, hardware, and software. Over 90 factors may influence VRISE frequency and severity. We identify guidelines for each factor to help reduce VR side effects. To better reflect our confidence in those guidelines, we graded each with a level of evidence rating. Common factors occasionally influence different forms of VRISE. This can lead to confusion in the literature. General guidelines for using VR at work involve worker adaptation, such as limiting immersion times to between 20 and 30 min. These regimens involve taking regular breaks. Extra care is required for workers with special needs, neurodiversity, and gerontechnological concerns. In addition to following our guidelines, stakeholders should be aware that current head-mounted displays and virtual environments can continue to induce VRISE. While no single existing method fully alleviates VRISE, workers' health and safety must be monitored and safeguarded when VR is used at work.

vexptoolbox: A software toolbox for human behavior studies using the Vizard virtual reality platform

Virtual reality (VR) is a powerful tool for researchers due to its potential to study dynamic human behavior in highly naturalistic environments while retaining full control over the presented stimuli. Due to advancements in consumer hardware, VR devices are now very affordable and have also started to include technologies such as eye tracking, further extending potential research applications. Rendering engines such as Unity, Unreal, or Vizard now enable researchers to easily create complex VR environments. However, implementing the experimental design can still pose a challenge, and these packages do not provide out-of-the-box support for trial-based behavioral experiments. Here, we present a Python toolbox, designed to facilitate common tasks when developing experiments using the Vizard VR platform. It includes functionality for common tasks like creating, randomizing, and presenting trial-based experimental designs or saving results to standardized file formats. Moreover, the toolbox greatly simplifies continuous recording of eye and body movements using any hardware supported in Vizard. We further implement and describe a simple goal-directed reaching task in VR and show sample data recorded from five volunteers. The toolbox, example code, and data are all available on GitHub under an open-source license. We hope that our toolbox can simplify VR experiment development, reduce code duplication, and aid reproducibility and open-science efforts.

Enhancing Operational Police Training in High Stress Situations with Virtual Reality: Experiences, Tools and Guidelines

Virtual Reality (VR) provides great opportunities for police officers to train decision-making and acting (DMA) in cognitively demanding and stressful situations. This paper presents a summary of findings from a three-year project, including requirements collected from experienced police trainers and industry experts, and quantitative and qualitative results of human factor studies and field trials. Findings include advantages of VR training such as the possibility to safely train high-risk situations in controllable and reproducible training environments, include a variety of avatars that would be difficult to use in real-life training (e.g., vulnerable populations or animals) and handle dangerous equipment (e.g., explosives) but also highlight challenges such as tracking, locomotion and intelligent virtual agents. The importance of strong alignment between training didactics and technical possibilities is highlighted and potential solutions presented. Furthermore training outcomes are transferable to real-world police duties and may apply to other domains that would benefit from simulation-based training.

Familiar size affects perception differently in virtual reality and the real world

The promise of virtual reality (VR) as a tool for perceptual and cognitive research rests on the assumption that perception in virtual environments generalizes to the real world. Here, we conducted two experiments to compare size and distance perception between VR and physical reality (Maltz et al. 2021 J. Vis. 21, 1-18). In experiment 1, we used VR to present dice and Rubik's cubes at their typical sizes or reversed sizes at distances that maintained a constant visual angle. After viewing the stimuli binocularly (to provide vergence and disparity information) or monocularly, participants manually estimated perceived size and distance. Unlike physical reality, where participants relied less on familiar size and more on presented size during binocular versus monocular viewing, in VR participants relied heavily on familiar size regardless of the availability of binocular cues. In experiment 2, we demonstrated that the effects in VR generalized to other stimuli and to a higher quality VR headset. These results suggest that the use of binocular cues and familiar size differs substantially between virtual and physical reality. A deeper understanding of perceptual differences is necessary before assuming that research outcomes from VR will generalize to the real world. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Virtual Reality for Vision Science

Virtual reality (VR) allows us to create visual stimuli that are both immersive and reactive. VR provides many new opportunities in vision science. In particular, it allows us to present wide field-of-view, immersive visual stimuli; for observers to actively explore the environments that we create; and for us to understand how visual information is used in the control of behaviour. In contrast with traditional psychophysical experiments, VR provides much greater flexibility in creating environments and tasks that are more closely aligned with our everyday experience. These benefits of VR are of particular value in developing our theories of the behavioural goals of the visual system and explaining how visual information is processed to achieve these goals. The use of VR in vision science presents a number of technical challenges, relating to how the available software and hardware limit our ability to accurately specify the visual information that defines our virtual environments and the interpretation of data gathered in experiments with a freely moving observer in a responsive environment.

[The application of virtual reality in the treatment of mental disorders]

BACKGROUND

Virtual reality (VR) enables immersion in an interactive digital world with realistic experiences, that can be applied for controlled and personalized interventions. This review summarizes the current research on VR in the treatment of mental disorders.

METHODS

Selective literature search in PubMed and Google Scholar.

RESULTS

An increasing number of publications report the therapeutic application of VR for the treatment of mental disorders. Most VR applications are based on established therapy approaches, such as exposure therapy. According to meta-analytic data, virtual exposure therapy (VRET) for specific phobia and agoraphobia with panic disorder is as effective as traditional in vivo exposure therapy. VRET for the treatment of social phobia is significantly more effective than waitlist and placebo control groups with, however, currently inconsistent metanalytic results when compared to in vivo exposure therapy. VRET for the treatment of posttraumatic stress disorder (PTSD) is similar in effectiveness compared to active psychotherapy. For psychosis, positive results have been reported for the VR-based treatment of auditory verbal hallucinations. For patients with a substance use disorder, VR can induce craving, with still unverified diagnostic and therapeutic relevance.

CONCLUSIONS

VRET can broaden the psychotherapy options for anxiety disorders. Encouraging results of VR-based treatments for psychosis and PTSD indicate the need for further research concerning its effectiveness and safety. In the field of substance use disorders, evaluation of clinical-orientated VR applications is needed.

The Treatment of Substance Use Disorders: Recent Developments and New Perspectives

Substance-related disorders are complex psychiatric disorders that are characterized by continued consumption in spite of harmful consequences. Addiction affects various brain networks critically involved in learning, reward, and motivation, as well as inhibitory control. Currently applied therapeutic approaches aim at modification of behavior that ultimately leads to decrease of consumption or abstinence in individuals with substance use disorders. However, traditional treatment methods might benefit from recent neurobiological and cognitive neuroscientific research findings. Novel cognitive-behavioral approaches in the treatment of addictive behavior aim at enhancement of strategies to cope with stressful conditions as well as craving-inducing cues and target erroneous learning mechanisms, including cognitive bias modification, reconsolidation-based interventions, mindfulness-based interventions, virtual-reality-based cue exposure therapy as well as pharmacological augmentation strategies. This review discusses therapeutic strategies that target dysregulated neurocognitive processes associated with the development and maintenance of disordered substance use and may hold promise as effective treatments for substance-related disorders.

Multiple spatial frames for immersive working memory

As we move around, relevant information that disappears from sight can still be held in working memory to serve upcoming behaviour. How we maintain and select visual information as we move through the environment remains poorly understood because most laboratory tasks of working memory rely on removing visual material while participants remain still. We used virtual reality to study visual working memory following self-movement in immersive environments. Directional biases in gaze revealed the recruitment of more than one spatial frame for maintaining and selecting memoranda following self-movement. The findings bring the important realization that multiple spatial frames support working memory in natural behaviour. The results also illustrate how virtual reality can be a critical experimental tool to characterize this core memory system.

Stereoscopic depth constancy for physical objects and their virtual counterparts

Stereopsis plays an important role in depth perception; if so, disparity-defined depth should not vary with distance. However, studies of stereoscopic depth constancy often report systematic distortions in depth judgments over distance, particularly for virtual stimuli. Our aim was to understand how depth estimation is impacted by viewing distance and display-based cue conflicts by replicating physical objects in virtual counterparts. To this end, we measured perceived depth using virtual textured half-cylinders and identical three-dimensional (3D) printed versions at two viewing distances under monocular and binocular conditions. Virtual stimuli were viewed using a mirror stereoscope and an Oculus Rift head-mounted display (HMD), while physical stimuli were viewed in a controlled test environment. Depth judgments were similar in both virtual apparatuses, which suggests that variations in the viewing geometry and optics of the HMD have little impact on perceived depth. When viewing physical stimuli binocularly, judgments were accurate and exhibited stereoscopic depth constancy. However, in all cases, depth was underestimated for virtual stimuli and failed to achieve depth constancy. It is clear that depth constancy is only complete for cue-rich physical stimuli and that the failure of constancy in virtual stimuli is due to the presence of the vergence-accommodation conflict. Further, our post hoc analysis revealed that prior experience with virtual and physical environments had a strong effect on depth judgments. That is, performance in virtual environments was enhanced by limited exposure to a related task using physical objects.

Visual-haptic integration, action and embodiment in virtual reality

The current generation of virtual reality (VR) technologies has improved substantially from legacy systems, particularly in the resolution and latency of their visual display. The presentation of haptic cues remains challenging, however, because haptic systems do not generalise well over the range of stimuli (both tactile and proprioceptive) normally present when interacting with objects in the world. This study investigated whether veridical tactile and proprioceptive cues lead to more efficient interaction with a virtual environment. Interaction in the world results in spatial and temporal correlation of tactile, proprioceptive and visual cues. When cues in VR are similarly correlated, observers experience a sense of embodiment and agency of their avatars. We investigated whether sensorimotor performance was mediated by embodiment of the avatar hands. Participants performed a Fitts' tapping task in different conditions (VR with no haptics, active haptics, passive haptics, and on a real touchscreen). The active-haptic condition provided abstract tactile cues and the passive haptic condition provided veridical tactile and proprioceptive cues. An additional (hybrid haptics) condition simulated an ideal passive haptic system. Movement efficiency (throughput) and embodiment were higher for the passive than for the active and no-haptics conditions. However, components of embodiment (perceived agency and ownership) did not predict unique variance in throughput. Improved sensorimotor performance and ratings of presence and realism support the use of passive haptics in VR environments where objects are in known and stable locations, regardless of whether performance was mediated by the sense of embodiment.

Patients Prefer a Virtual Reality Approach Over a Similarly Performing Screen-Based Approach for Continuous Oculomotor-Based Screening of Glaucomatous and Neuro-Ophthalmological Visual Field Defects

Standard automated perimetry (SAP) is the gold standard for evaluating the presence of visual field defects (VFDs). Nevertheless, it has requirements such as prolonged attention, stable fixation, and a need for a motor response that limit application in various patient groups. Therefore, a novel approach using eye movements (EMs) - as a complementary technique to SAP - was developed and tested in clinical settings by our group. However, the original method uses a screen-based eye-tracker which still requires participants to keep their chin and head stable. Virtual reality (VR) has shown much promise in ophthalmic diagnostics - especially in terms of freedom of head movement and precise control over experimental settings, besides being portable. In this study, we set out to see if patients can be screened for VFDs based on their EM in a VR-based framework and if they are comparable to the screen-based eyetracker. Moreover, we wanted to know if this framework can provide an effective and enjoyable user experience (UX) compared to our previous approach and the conventional SAP. Therefore, we first modified our method and implemented it on a VR head-mounted device with built-in eye tracking. Subsequently, 15 controls naïve to SAP, 15 patients with a neuro-ophthalmological disorder, and 15 glaucoma patients performed three tasks in a counterbalanced manner: (1) a visual tracking task on the VR headset while their EM was recorded, (2) the preceding tracking task but on a conventional screen-based eye tracker, and (3) SAP. We then quantified the spatio-temporal properties (STP) of the EM of each group using a cross-correlogram analysis. Finally, we evaluated the human-computer interaction (HCI) aspects of the participants in the three methods using a user-experience questionnaire. We find that: (1) the VR framework can distinguish the participants according to their oculomotor characteristics; (2) the STP of the VR framework are similar to those from the screen-based eye tracker; and (3) participants from all the groups found the VR-screening test to be the most attractive. Thus, we conclude that the EM-based approach implemented in VR can be a user-friendly and portable companion to complement existing perimetric techniques in ophthalmic clinics.

Virtual Reality-Based Treatment Approaches in the Field of Substance Use Disorders

Purpose

Substance use disorders (SUD) are burdening chronic conditions characterized by high relapse rates despite severe negative consequences. Substance-related cues that elicit craving by means of automatic physiological and behavioural responses have long been suggested to predict relapse. One major mechanism contributing to relapse behaviour are cue-induced behavioural approach tendencies towards the addictive agent. Recently, there has been an emerging interest in virtual reality (VR)-based approaches to assess and modify craving and its related responses. This review aims at elucidating (1) VR techniques applied in the field of SUD, (2) VR as an induction/assessment tool for biopsychological correlates of craving and (3) VR-based therapeutic approaches.

Findings

There is an emerging number of studies focusing on different substances of abuse incorporating VR in craving induction/assessment as well as therapy. Despite some limitations as missing of randomized controlled clinical trials with large samples and missing data on the long-term effects of VR treatment, the VR approach showed consistent results in eliciting and reducing craving across different substances.

Summary

This review suggests virtual reality as a promising tool for the assessment and treatment of craving among individuals with substance use disorders. Because of its ecological validity, VR unifies the benefits of a laboratory setting with the advantages of a realistic environment. Further studies with large samples and randomized controlled clinical trials using more homogenous VR techniques as well as assessment of objective biophysiological craving markers are required.

Taking aim at the perceptual side of motor learning: exploring how explicit and implicit learning encode perceptual error information through depth vision

Motor learning in visuomotor adaptation tasks results from both explicit and implicit processes, each responding differently to an error signal. Although the motor output side of these processes has been extensively studied, the visual input side is relatively unknown. We investigated if and how depth perception affects the computation of error information by explicit and implicit motor learning. Two groups of participants made reaching movements to bring a virtual cursor to a target in the frontoparallel plane. The Delayed group was allowed to reaim and their feedback was delayed to emphasize explicit learning, whereas the camped group received task-irrelevant clamped cursor feedback and continued to aim straight at the target to emphasize implicit adaptation. Both groups played this game in a highly detailed virtual environment (depth condition), leveraging a cover task of playing darts in a virtual tavern, and in an empty environment (no-depth condition). The delayed group showed an increase in error sensitivity under depth relative to no-depth. In contrast, the clamped group adapted to the same degree under both conditions. The movement kinematics of the delayed participants also changed under the depth condition, consistent with the target appearing more distant, unlike the Clamped group. A comparison of the delayed behavioral data with a perceptual task from the same individuals showed that the greater reaiming in the depth condition was consistent with an increase in the scaling of the error distance and size. These findings suggest that explicit and implicit learning processes may rely on different sources of perceptual information.NEW & NOTEWORTHY We leveraged a classic sensorimotor adaptation task to perform a first systematic assessment of the role of perceptual cues in the estimation of an error signal in the 3-D space during motor learning. We crossed two conditions presenting different amounts of depth information, with two manipulations emphasizing explicit and implicit learning processes. Explicit learning responded to the visual conditions, consistent with perceptual reports, whereas implicit learning appeared to be independent of them.

A Methodological Framework for Assessing Social Presence in Music Interactions in Virtual Reality

Virtual reality (VR) brings radical new possibilities to the empirical study of social music cognition and interaction. In the present article, we consider the role of VR as a research tool, based on its potential to create a sense of "social presence": the illusory feeling of being, and socially interacting, inside a virtual environment. This makes VR promising for bridging ecological validity ("research in the wild") and experimental control ("research in the lab") in empirical music research. A critical assumption however is the actual ability of VR to simulate real-life social interactions, either via human-embodied avatars or computer-controlled agents. The mediation of social musical interactions via VR is particularly challenging due to their embodied, complex, and emotionally delicate nature. In this article, we introduce a methodological framework to operationalize social presence by a combination of factors across interrelated layers, relating to the performance output, embodied co-regulation, and subjective experiences. This framework provides the basis for the proposal of a pragmatic approach to determine the level of social presence in virtual musical interactions, by comparing the outcomes across the multiple layers with the outcomes of corresponding real-life musical interactions. We applied and tested this pragmatic approach via a case-study of piano duet performances of the piece Piano Phase composed by Steve Reich. This case-study indicated that a piano duet performed in VR, in which the real-time interaction between pianists is mediated by embodied avatars, might lead to a strong feeling of social presence, as reflected in the measures of performance output, embodied co-regulation, and subjective experience. In contrast, although a piano duet in VR between an actual pianist and a computer-controlled agent led to a relatively successful performance output, it was inadequate in terms of both embodied co-regulation and subjective experience.

Combining cues to judge distance and direction in an immersive virtual reality environment

When we move, the visual direction of objects in the environment can change substantially. Compared with our understanding of depth perception, the problem the visual system faces in computing this change is relatively poorly understood. Here, we tested the extent to which participants' judgments of visual direction could be predicted by standard cue combination rules. Participants were tested in virtual reality using a head-mounted display. In a simulated room, they judged the position of an object at one location, before walking to another location in the room and judging, in a second interval, whether an object was at the expected visual direction of the first. By manipulating the scale of the room across intervals, which was subjectively invisible to observers, we put two classes of cue into conflict, one that depends only on visual information and one that uses proprioceptive information to scale any reconstruction of the scene. We find that the sensitivity to changes in one class of cue while keeping the other constant provides a good prediction of performance when both cues vary, consistent with the standard cue combination framework. Nevertheless, by comparing judgments of visual direction with those of distance, we show that judgments of visual direction and distance are mutually inconsistent. We discuss why there is no need for any contradiction between these two conclusions.

EEG Cortical Activities and Networks Altered by Watching 2D/3D Virtual Reality Videos

Abstract. Virtual reality (VR), which can represent real-life events and situations, is being increasingly applied to many fields, such as education, entertainment, and medical rehabilitation. Correspondingly, the neural information processing of VR has attracted attention. However, the underlying neural mechanisms of VR environments have not yet been fully revealed. The purpose of this study was to examine the possible differences in brain activities and networks between the less immersive 2D and the fully immersive 3D VR environments. 3D VR videos and the same 2D scenes were presented to the participants and the scalp electroencephalogram (EEG) was recorded, respectively. Power spectral density (PSD) and the functional connectivity of these EEG signals were analyzed. The results showed that 3D VR videos significantly enhanced the PSD of θ rhythm (4–7 Hz) in the frontal lobe; decreased the PSD of α rhythm (8–13 Hz) in the parietal and the occipital lobes; increased the PSD of β rhythm (14–30 Hz) in the frontal, the parietal, the temporal, and the occipital lobes, relative to 2D VR watching. Furthermore, 3D versus 2D VR-induced alterations in the patterns of brain networks were similar to the patterns of PSD. Specifically, for the θ rhythm, 3D VR significantly enhanced the frontal and the temporal brain functional connectivity; for the α rhythm, 3D VR increased the parietal and the occipital networks; for the β rhythm, 3D VR remarkably increased the frontal, the occipital, the frontal-temporal and the frontal-occipital brain functional connectivity, relative to 2D VR. These significant differences between 3D and 2D VR video-watching suggest that the neural information processing of cortical activities and networks is correlated to the degree of immersion. The present results, collected with previous researches, implicate that some visual-related information processes, such as visual attention, visual perception, and visual immersion are more robust in 3D VR environments.

What do users think about Virtual Reality relaxation applications? A mixed methods study of online user reviews using natural language processing

The advent of affordable Virtual Reality (VR) technology has spurred consumer and commercial interest in VR relaxation applications, which has quickly grown into a popular non-gaming genre on digital marketplaces. While laboratory studies have demonstrated efficacy of VR relaxation for mental health purposes, little is known about how users experience this type of intervention and no study has examined the reception of consumer versions among regular users in everyday life. Studying published user reviews offers a unique window into naturalistic user experiences that complements traditional qualitative methods by circumventing the sampling bias of interview studies, and allowing analyses on full samples, unconstrained by coding resources. Using an innovative, semi-automated Natural Language Processing technique, the current study analyzed 1379 published reviews (including star ratings) of 30 different VR relaxation applications available for the Oculus Go and Gear VR. The uncovered topic structure and sentiment analysis thereof suggests that users have an overall positive view of VR relaxation applications, describing them as successful in inducing immersion and relaxation, and having appreciated gamification elements. However, perceived quality varied substantially between applications that explained more variance in star ratings than specific features. Critical issues raised were both technical (e.g. "overheating") in nature and related to specific design elements and use. Implications for the design of consumer VR applications and future research are discussed.

Size and shape constancy in consumer virtual reality

With the increase in popularity of consumer virtual reality headsets, for research and other applications, it is important to understand the accuracy of 3D perception in VR. We investigated the perceptual accuracy of near-field virtual distances using a size and shape constancy task, in two commercially available devices. Participants wore either the HTC Vive or the Oculus Rift and adjusted the size of a virtual stimulus to match the geometric qualities (size and depth) of a physical stimulus they were able to refer to haptically. The judgments participants made allowed for an indirect measure of their perception of the egocentric, virtual distance to the stimuli. The data show under-constancy and are consistent with research from carefully calibrated psychophysical techniques. There was no difference in the degree of constancy found in the two headsets. We conclude that consumer virtual reality headsets provide a sufficiently high degree of accuracy in distance perception, to allow them to be used confidently in future experimental vision science, and other research applications in psychology.

Is Continued Improvement After Automated Virtual Reality Exposure Therapy for Spider Phobia Explained by Subsequent in-vivo Exposure? A First Test of the Lowered Threshold Hypothesis

Consumer Virtual Reality (VR) technology offers a powerful, immersive medium for scalable dissemination of mental health interventions. Decades of research has shown VR exposure therapy to be efficacious in the treatment of anxiety disorders and that the fear reduction generalizes to real-world stimuli. Many studies also report continued improvement over time, after discontinuing VR use. The lowered threshold hypothesis states that this continued improvement is moderated by lowering the threshold to conduct subsequent in-vivo exposure. The current study is the first to formally test this hypothesis, using data from a recent trial on automated VR exposure therapy for spider phobia, in which participants (n = 49) were followed for 1 year, completing assessments 1 week, 3 and 12 months post-treatment. The assessment included validated self-report of phobia symptoms, a standardized behavioral approach test featuring a real spider, and a questionnaire for self-reporting frequency of in-vivo exposures since last assessment. Number of in-vivo exposures was found to be independently associated with greater symptom decrease in longitudinal outcome models. In sequential structural equation models, immediate post-treatment symptom reduction was associated with subsequent in-vivo exposures, which in turn was associated with continued symptom reduction. However, this applied only to self-reported phobia symptoms (not behavioral avoidance) and no associations were found past 3 months. Our findings offer preliminary, partial support for the lowered threshold hypothesis, suggesting that VR exposure interventions may benefit from including explicit in-virtuo to in-vivo transitioning components.

Better, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy

Virtual reality (VR) is an immersive technology capable of creating a powerful, perceptual illusion of being present in a virtual environment. VR technology has been used in cognitive behavior therapy since the 1990s and accumulated an impressive evidence base, yet with the recent release of consumer VR platforms came a true paradigm shift in the capabilities and scalability of VR for mental health. This narrative review summarizes the past, present, and future of the field, including milestone studies and discussions on the clinical potential of alternative embodiment, gamification, avatar therapists, virtual gatherings, immersive storytelling, and more. Although the future is hard to predict, clinical VR has and will continue to be inherently intertwined with what are now rapid developments in technology, presenting both challenges and exciting opportunities to do what is not possible in the real world.

bmlTUX: Design and Control of Experiments in Virtual Reality and Beyond

Advances in virtual reality technology have made it a valuable new tool for vision and perception researchers. Coding virtual reality experiments from scratch can be difficult and time-consuming, so researchers rely on software such as Unity game engine to create and edit virtual scenes. However, Unity lacks built-in tools for controlling experiments. Existing third-party add-ins requires complicated scripts to define experiments. This can be difficult and requires advanced coding knowledge, especially for multifactorial experimental designs. In this article, we describe a new free and open-source tool called the BiomotionLab Toolkit for Unity Experiments (bmlTUX) that provides a simple interface for controlling experiments in Unity. In contrast to existing tools, bmlTUX provides a graphical interface to automatically handle combinatorics, counterbalancing, randomization, mixed designs, and blocking of trial order. The toolbox works out-of-the-box since simple experiments can be created with almost no coding. Furthermore, multiple design configurations can be swapped with a drag-and-drop interface allowing researchers to test new configurations iteratively while maintaining the ability to easily revert to previous configurations. Despite its simplicity, bmlTUX remains highly flexible and customizable, catering to coding novices and experts alike.