The reality of virtual reality

The presence of fear: How subjective fear, not physiological changes, shapes the experience of presence

When we become engrossed in novels, films, games, or even our own wandering thoughts, we can feel present in a reality distinct from the real world. Although this subjective sense of presence is, presumably, a ubiquitous aspect of conscious experience, the mechanisms that produce it are unknown. Correlational studies conducted in virtual reality have shown that we feel more present when we are afraid, motivating claims that physiological changes contribute to presence; however, such causal claims remain to be evaluated. Here, we report two experiments that test the causal role of subjective and physiological components of fear (i.e., activation of the sympathetic nervous system) in generating presence. In Study 1, we validated a virtual reality simulation capable of inducing fear. Participants rated their emotions while they crossed a wooden plank that appeared to be suspended above a city street; at the same time, we recorded heart rate and skin conductance levels. Height exposure increased ratings of fear, presence, and both measures of sympathetic activation. Although presence and fear ratings were correlated during height exposure, presence and sympathetic activation were unrelated. In Study 2, we manipulated whether the plank appeared at height or at ground level. We also captured participants' movements, which revealed that alongside increases in subjective fear, presence, and sympathetic activation, participants also moved more slowly at height relative to controls. Using a mediational approach, we found that the relationship between height exposure and presence on the plank was fully mediated by self-reported fear, and not by sympathetic activation. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Electrophysiological correlates of face and object perception: A comparative analysis of 2D laboratory and virtual reality conditions

Human face perception is a specialized visual process with inherent social significance. The neural mechanisms reflecting this intricate cognitive process have evolved in spatially complex and emotionally rich environments. Previous research using VR to transfer an established face perception paradigm to realistic conditions has shown that the functional properties of face-sensitive neural correlates typically observed in the laboratory are attenuated outside the original modality. The present study builds on these results by comparing the perception of persons and objects under conventional laboratory (PC) and realistic conditions in VR. Adhering to established paradigms, the PC- and VR modalities both featured images of persons and cars alongside standard control images. To investigate the individual stages of realistic face processing, response times, the typical face-sensitive N170 component, and relevant subsequent components (L1, L2; pre-, post-response) were analyzed within and between modalities. The between-modality comparison of response times and component latencies revealed generally faster processing under realistic conditions. However, the obtained N170 latency and amplitude differences showed reduced discriminative capacity under realistic conditions during this early stage. These findings suggest that the effects commonly observed in the lab are specific to monitor-based presentations. Analyses of later and response-locked components showed specific neural mechanisms for identification and evaluation are employed when perceiving the stimuli under realistic conditions, reflected in discernible amplitude differences in response to faces and objects beyond the basic perceptual features. Conversely, the results do not provide evidence for comparable stimulus-specific perceptual processing pathways when viewing pictures of the stimuli under conventional laboratory conditions.

SHAP value-based ERP analysis (SHERPA): Increasing the sensitivity of EEG signals with explainable AI methods

Conventionally, event-related potential (ERP) analysis relies on the researcher to identify the sensors and time points where an effect is expected. However, this approach is prone to bias and may limit the ability to detect unexpected effects or to investigate the full range of the electroencephalography (EEG) signal. Data-driven approaches circumvent this limitation, however, the multiple comparison problem and the statistical correction thereof affect both the sensitivity and specificity of the analysis. In this study, we present SHERPA - a novel approach based on explainable artificial intelligence (XAI) designed to provide the researcher with a straightforward and objective method to find relevant latency ranges and electrodes. SHERPA is comprised of a convolutional neural network (CNN) for classifying the conditions of the experiment and SHapley Additive exPlanations (SHAP) as a post hoc explainer to identify the important temporal and spatial features. A classical EEG face perception experiment is employed to validate the approach by comparing it to the established researcher- and data-driven approaches. Likewise, SHERPA identified an occipital cluster close to the temporal coordinates for the N170 effect expected. Most importantly, SHERPA allows quantifying the relevance of an ERP for a psychological mechanism by calculating an "importance score". Hence, SHERPA suggests the presence of a negative selection process at the early and later stages of processing. In conclusion, our new method not only offers an analysis approach suitable in situations with limited prior knowledge of the effect in question but also an increased sensitivity capable of distinguishing neural processes with high precision.

Induced oscillatory brain responses under virtual reality conditions in the context of repetition priming

In the human electroencephalogram (EEG), induced oscillatory responses in various frequency bands are regarded as valuable indices to examine the neural mechanisms underlying human memory. While the advent of virtual reality (VR) drives the investigation of mnemonic processing under more lifelike settings, the joint application of VR and EEG methods is still in its infancy (e.g., due to technical limitations impeding the signal acquisition). The objective of the present EEG study was twofold. First, we examined whether the investigation of induced oscillations under VR conditions yields equivalent results compared to standard paradigms. Second, we aimed at obtaining further insights into basic memory-related brain mechanisms in VR. To these ends, we relied on a standard implicit memory design, namely repetition priming, for which the to-be-expected effects are well-documented for conventional studies. Congruently, we replicated a suppression of the evoked potential after stimulus onset. Regarding the induced responses, we observed a modulation of induced alphaband in response to a repeated stimulus. Importantly, our results revealed a repetition-related suppression of the high-frequency induced gammaband response (>30 Hz), indicating the sharpening of a cortical object representation fostering behavioral priming effects. Noteworthy, the analysis of the induced gammaband responses required a number of measures to minimize the influence of external and internal sources of artefacts (i.e., the electrical shielding of the technical equipment and the control for miniature eye movements). In conclusion, joint VR-EEG studies with a particular focus on induced oscillatory responses offer a promising advanced understanding of mnemonic processing under lifelike conditions.

Proposing immersive virtual reality scenarios for validating verbal content analysis methods in adult samples

Verbal content analyses to differentiate truthful and fabricated statements, such as the Criteria-Based Content Analysis (CBCA), are used in lie detection research as well as in practice to assess the credibility of statements in criminal court proceedings. Meta-analyses demonstrate validity of verbal content analyses above chance, but the traditional research paradigms usually lack either ecological or internal validity. The authors discuss the usage of immersive virtual reality scenarios to solve this dilemma, as both types of validity can be increased by this approach. In this integrative review of existing literature on the current use of virtual scenarios in forensic and victimology research, the authors extract strengths and limitations for possible VR studies in the context of verbal content analysis. Furthermore, novel ethical challenges involved are summarized and implications for future studies proposed. Overall, we argue in favor of using virtual reality scenarios to validate methods for verbal content analysis, but also urge to consider ethical limitations regarding unwanted short- and long-term aftereffects.

Confirmation of the viability of a metaverse yoga class and investigation into the impact on pain, anxiety, and depression associated with low back pain after engaging in virtual yoga sessions

This study aimed to explore the influence of metaverse technology (MT) factors like presence, usability, and enjoyment on patients' satisfaction, with a focus on examining potential mediating effects. In addition, it sought to assess whether the yoga practice as an intervention therapy in MT induces changes in the pain, anxiety, and depression levels of patients experiencing back pain. From the pool of 202 participants, this study chose participants who had reported enduring low back pain over 12 weeks, with a visual analogue scale (VAS) rating of 4 or higher. After completing the questionnaire, patients were randomly assigned to either the control group (COG, n=100) or the yoga exercise group (YEG, n=99). Results showed that the construct validity for questionnaires and a reasonable model fit were confirmed, and that presence showed a statistically significant effect on psychological satisfaction via the mediating path of enjoyment (β=0.592, P=0.001). Following 8 weeks of the yoga practice, the VAS increased for the COG, while it decreased significantly by ~29% for the YEG (P=0.001). YEG also exhibited a decrease in the Oswestry Disability Index by ~17%, anxiety by ~7%, and depression by ~10% (P=0.001). In conclusion, psychological satisfaction in a yoga practice using a metaverse cannot be achieved solely through the sense of presence; enjoyment is necessary for patients' satisfaction. Moreover, it was verified that virtual yoga practice is effective in ameliorating psychological factors resulting from back pain.

Self-Assessed Experience of Emotional Involvement in Sensory Analysis Performed in Virtual Reality

Virtual reality (VR) technology has gained significant attention in various fields, including education for health professionals, sensory science, psychology, and consumer research. The first aim of the paper is to explore the self-assessed experience of emotional involvement in sensory analysis performed in VR. The Positive and Negative Affect Schedule (PANAS) is a widely used self-report measure that assesses positive and negative affective states. VR sensory analysis involves the use of immersive, interactive, and multi-sensory environments to evaluate sensory perception and emotional responses. By synthesizing relevant literature, this paper provides insights into the impact of VR on affective states, the effectiveness of VR in eliciting emotions, and the potential applications of the PANAS in VR sensory analysis. Furthermore, the second aim of the paper is to uncover the effect of VR sensory evaluation on the participant's emotional states, as it has a significant effect on their evaluations. The results suggest an increase in the sum of positive effects and a decrease in the negative ones. Although these results are promising, the relationship between the PANAS and VR sensory analysis is still underexplored, with limited research investigating the specific effects of VR on affective states measured using the PANAS. Further research is needed to better understand the potential of the PANAS in assessing emotional responses in VR environments and its implications for sensory analysis.

Temporal precision and accuracy of audio-visual stimuli in mixed reality systems

Mixed Reality (MR) techniques, such as Virtual (VR) and Augmented Reality (AR), are gaining popularity as a new methodology for neuroscience and psychology research. In studies involving audiovisual stimuli, it is crucial to have MR systems that can deliver these bimodal stimuli with controlled timing between the onset of each modality. However, the extent to which modern MR setups can achieve the necessary precision and accuracy of audiovisual stimulus onset asynchronies (SOAs) remains largely unknown. The objective of this study is to systematically evaluate the lag and variability between the auditory and visual onset of audiovisual stimuli produced on popular modern MR head-mounted displays (HMDs) from Meta, Microsoft, HTC, and Varjo in conjunction with commonly used development environments such as Unity and the Unreal Engine. To accomplish this, we developed a low-cost measurement system that enabled us to measure the actual SOA and its associated jitter. Our findings revealed that certain MR systems exhibited significant SOAs, with one case averaging 156.63 ms, along with jitter of up to ±11.82 ms. Using our methodology, we successfully conducted experimental calibration of a headset, achieving SOAs of -3.89 ± 1.56 ms. This paper aims to raise awareness among neuroscience researchers regarding the limitations of MR systems in delivering audiovisual stimuli without prior calibration. Furthermore, we present cost-effective methods to calibrate these systems, thereby facilitating the replication of future results.

Virtual vs. real: exploring perceptual, cognitive and affective dimensions in design product experiences

BACKGROUND

Virtual Reality (VR) has already emerged as an effective instrument for simulating realistic interactions, across various domains. In the field of User Experience (UX), VR has been used to create prototypes of real-world products. Here, the question is to what extent the users' experience of a virtual prototype can be equivalent to that of its real counterpart (the real product). This issue particularly concerns the perceptual, cognitive and affective dimensions of users' experiences.

METHODS

This exploratory study aims to address this issue by comparing the users' experience of a well-known product, i.e., the Graziella bicycle, presented either in Sumerian or Sansar VR platform, or in a physical setting. Participants' Emotional Engagement, Sense of Presence, Immersion, and Perceived Product Quality were evaluated after being exposed to the product in all conditions (i.e., Sumerian, Sansar and Physical).

RESULTS

The findings indicated significantly higher levels of Engagement and Positive Affect in the virtual experiences when compared to their real-world counterparts. Additionally, the sole notable distinction among the VR platforms was observed in terms of Realism.

CONCLUSIONS

This study suggests the feasibility and potential of immersive VR environments as UX evaluation tools and underscores their effectiveness in replicating genuine real-world experiences.

Therapist perspectives on telehealth-based virtual reality exposure therapy

Virtual reality (VR) can enhance mental health care. In particular, the effectiveness of VR-based exposure therapy (VRET) has been well-demonstrated for treatment of anxiety disorders. However, most applications of VRET remain localized to clinic spaces. We aimed to explore mental health therapists' perceptions of telehealth-based VRET (tele-VRET) by conducting semi-structured, qualitative interviews with 18 telemental health therapists between October and December 2022. Interview topics included telehealth experiences, exposure therapy over telehealth, perceptions of VR in therapy, and perspectives on tele-VRET. Therapists described how telehealth reduced barriers (88.9%, 16/18), enhanced therapy (61.1%, 11/18), and improved access to clients (38.9%, 7/18), but entailed problems with technology (61.1%, 11/18), uncontrolled settings (55.6%, 10/18), and communication di culties (50%, 9/18). Therapists adapted exposure therapy to telehealth by using online resources (66.7%, 12/18), preparing client expectations (55.6%, 10/18), and adjusting workflows (27.8%, 5/18). Most therapists had used VR before (72.2%, 13/18) and had positive impressions (55.6%, 10/18), but none had used VR clinically. In response to tele-VRET, therapists requested interactive session activities (77.8%, 14/18) and customizable interventions components (55.6%, 10/18). Concerns about tele-VRET included risks with certain clients (77.8%, 14/18), costs (50%, 9/18), side effects and privacy (22.2%, 4/18), and inappropriateness for specific forms of exposure therapy (16.7%, 3/18). These results show how designing for telehealth may extend VRET and can help inform collaborative development of health technologies.

Real-life relevant face perception is not captured by the N170 but reflected in later potentials: A comparison of 2D and virtual reality stimuli

The perception of faces is one of the most specialized visual processes in the human brain and has been investigated by means of the early event-related potential component N170. However, face perception has mostly been studied in the conventional laboratory, i.e., monitor setups, offering rather distal presentation of faces as planar 2D-images. Increasing spatial proximity through Virtual Reality (VR) allows to present 3D, real-life-sized persons at personal distance to participants, thus creating a feeling of social involvement and adding a self-relevant value to the presented faces. The present study compared the perception of persons under conventional laboratory conditions (PC) with realistic conditions in VR. Paralleling standard designs, pictures of unknown persons and standard control images were presented in a PC- and a VR-modality. To investigate how the mechanisms of face perception differ under realistic conditions from those under conventional laboratory conditions, the typical face-specific N170 and subsequent components were analyzed in both modalities. Consistent with previous laboratory research, the N170 lost discriminatory power when translated to realistic conditions, as it only discriminated faces and controls under laboratory conditions. Most interestingly, analysis of the later component [230–420 ms] revealed more differentiated face-specific processing in VR, as indicated by distinctive, stimulus-specific topographies. Complemented by source analysis, the results on later latencies show that face-specific neural mechanisms are applied only under realistic conditions (A video abstract is available in the Supplementary material and via YouTube: https://youtu.be/TF8wiPUrpSY).