The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment

The Differential Effects of Multisensory Attentional Cues on Task Performance in VR Depending on the Level of Cognitive Load and Cognitive Capacity

As the utilization of VR is expanding across diverse fields, research on devising attentional cues that could optimize users' task performance in VR has become crucial. Since the cognitive load imposed by the context and the individual's cognitive capacity are representative factors that are known to determine task performance, we aimed to examine how the effects of multisensory attentional cues on task performance are modulated by the two factors. For this purpose, we designed a new experimental paradigm in which participants engaged in dual (N-back, visual search) tasks under different levels of cognitive load while an attentional cue (visual, tactile, or visuotactile) was presented to facilitate search performance. The results showed that multi-sensory attentional cues are generally more effective than uni-sensory cues in enhancing task performance, but the benefit of multi-sensory cues changes according to the level of cognitive load and the individual's cognitive capacity; the amount of benefit increases as the cognitive load is higher and the cognitive capacity is lower. The findings of this study provide practical implications for designing attentional cues to enhance VR task performance, considering both the complexity of the VR context and users' internal characteristics.

A study on the effect of different channel cues on learning in immersive 360° videos

Immersive 360° videos are of interest to educators because of their ability to provide immersive sensory experience and other features. This study examined the effects of four cue conditions on 360° video learning performance, attention, cognitive load, and mood using eye-tracking devices, brainwave meters, and subjective questionnaires. The randomly assigned participants (n = 62) did go to the experimental group (visual cues only, auditory cues only, and audiovisual cues) or the control group (no cues). The results showed that visual and audiovisual cues effectively guide learners' attention to the related learning content, reduce cognitive load during learning, and improve retention performance but have no significant effect on knowledge transfer or long-term memory. Auditory cues increase the number of times learners look at the related learning content but do not affect gaze duration and distract their attention, hindering the acquisition of relevant learning content. The study also found that visual cues effectively increase the number of times learners looked at the content. However, they do not affect gaze duration. The study also revealed that visual cues effectively increase learners' relaxation when viewing 360° videos. The study's findings can provide a reference for the instructional processing of information related to 360° video design and its practical application in teaching.

Study on the combination of virtual machine tools and wearable vibration devices for operators experiencing cutting forces in the milling process

The primary goal of this study is to develop a wearable system for providing CNC machine operators with visual and tactile perception of triaxial cutting forces, thereby assisting operators in industrial environments to enhance work efficiency and prevent mechanical failures. To achieve this goal, we successfully integrated a virtual machining tool simulator with the remote-control wearable system (RCWS). Using the 'King Path' milling parameters, we employed the simulation software developed by the AIM-HI team to calculate static and dynamic cutting forces, converting this data into vibrational commands for the RCWS to generate corresponding tactile feedback. Furthermore, we conducted extensive experiments, testing various data conversion methods, including three sampling techniques and two data compression strategies, aiming to provide accurate tactile feedback related to cutting forces under different operating conditions.

Haptic Feedback, Performance and Arousal: A Comparison Study in an Immersive VR Motor Skill Training Task

This article investigates the relationship between fine motor skill training in VR, haptic feedback, and physiological arousal. To do so, we present the design and development of a motor skill task (buzzwire), along with a custom vibrotactile feedback attachment for the Geomagic Touch haptic device. A controlled experiment following a between-subjects design was conducted with 73 participants, studying the role of three feedback conditions - visual/kinesthetic, visual/vibrotactile and visual only - on the learning and performance of the considered task and the arousal levels of the participants. Results indicate that performance improved in all three feedback conditions after the considered training session. However, participants reported no change in self-efficacy and in terms of presence and task load (NASA-TLX). All three feedback conditions also showed similar arousal levels. Further analysis revealed that positive changes in performance were linked to higher arousal levels. These results suggest the potential of haptic feedback to affect arousal levels and encourage further research into using this relationship to improve motor skill training in VR.

Extended reality to assess post-stroke manual dexterity: contrasts between the classic box and block test, immersive virtual reality with controllers, with hand-tracking, and mixed-reality tests

BACKGROUND

Recent technological advancements present promising opportunities to enhance the frequency and objectivity of functional assessments, aligning with recent stroke rehabilitation guidelines. Within this framework, we designed and adapted different manual dexterity tests in extended reality (XR), using immersive virtual reality (VR) with controllers (BBT-VR-C), immersive VR with hand-tracking (BBT-VR-HT), and mixed-reality (MD-MR).

OBJECTIVE

This study primarily aimed to assess and compare the validity of the BBT-VR-C, BBT-VR-HT and MD-MR to assess post-stroke manual dexterity. Secondary objectives were to evaluate reliability, usability and to define arm kinematics measures.

METHODS

A sample of 21 healthy control participants (HCP) and 21 stroke individuals with hemiparesis (IHP) completed three trials of the traditional BBT, the BBT-VR-C, BBT-VR-HT and MD-MR. Content validity of the different tests were evaluated by asking five healthcare professionals to rate the difficulty of performing each test in comparison to the traditional BBT. Convergent validity was evaluated through correlations between the scores of the traditional BBT and the XR tests. Test-retest reliability was assessed through correlations between the second and third trial and usability was assessed using the System Usability Scale (SUS). Lastly, upper limb movement smoothness (SPARC) was compared between IHP and HCP for both BBT-VR test versions.

RESULTS

For content validity, healthcare professionals rated the BBT-VR-HT (0[0-1]) and BBT-MR (0[0-1]) as equally difficult to the traditional BBT, whereas they rated BBT-VR-C as more difficult than the traditional BBT (1[0-2]). For IHP convergent validity, the Pearson tests demonstrated larger correlations between the scores of BBT and BBT-VR-HT (r = 0.94;p < 0.001), and BBT and MD-MR (r = 0.95;p < 0.001) than BBT and BBT-VR-C (r = 0.65;p = 0.001). BBT-VR-HT and MD-MR usability were both rated as excellent, with median SUS scores of 83[57.5-91.3] and 83[53.8-92.5] respectively. Excellent reliability was found for the BBT-VR-C (ICC = 0.96;p < 0.001), BBT-VR-HT (ICC = 0.96;p < 0.001) and BBT-MR (ICC = 0.99;p < 0.001). The usability of the BBT-VR-C was rated as good with a median SUS of 70[43.8-83.8]. Upper limb movements of HCP were significantly smoother than for IHP when completing either the BBT-VR-C (t = 2.05;p = 0.043) and the BBT-VR-HT (t = 5.21;p < 0.001).

CONCLUSION

The different XR manual tests are valid, short-term reliable and usable tools to assess post-stroke manual dexterity.

TRIAL REGISTRATION

https://clinicaltrials.gov/ct2/show/NCT04694833 ; Unique identifier: NCT04694833, Date of registration: 11/24/2020.

Virtual reality exposure with vibrotactile stimulation for the treatment of fear of flying: A pilot study

BACKGROUND AND OBJECTIVES

Virtual reality (VR) interventions are becoming more prevalent in treating fear of flying (FoF). Since multisensory stimulation can enhance the sense of presence in a virtual environment, the present study compared virtual reality exposure with and without vibrotactile cues to determine its contribution to the realism of the virtual experience.

METHODS

A repeated measures design was used. Thirty-one participants were exposed to two experimental conditions with a minimum of a one-week interval between them: one in which participants were exposed to the virtual environment with vibrotactile cues (smart chair, SC), and another in which participants were exposed to the virtual environment without vibrotactile cues (ordinary chair, OC). The administration order of both conditions was counterbalanced to avoid possible order effects.

RESULTS

Participants felt higher levels of sense of presence when using the SC than the OC. However, the addition of vibrotactile stimulation partially influenced experienced anxiety. Some personality traits were also associated with participants' sense of presence and anxiety responses during the exposure.

LIMITATIONS

The sample size was smaller than required. Moreover, only self-reported measures were used. Finally, a roller coaster instead of an airplane scenario was used for the exposure, which might not have been suitable enough for provoking anxiety in participants with FoF.

CONCLUSIONS

Vibrotactile cues enhanced the sense of presence. However, the addition of vibrotactile stimulation did not have a consistent effect on anxiety experienced during exposure. Therefore, the benefits of incorporating vibrotactile cues in virtual reality environments for exposure therapy are not clear.

Effects of virtual reality-based cognitive interventions on cognitive function and activity of daily living among stroke patients: Systematic review and meta-analysis

AIMS

To examine the effects of virtual reality-based cognitive interventions on cognitive function and activities of daily living among stroke patients, and to identify the optimal design for such intervention.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

Medline, EMBASE, Cochrane, CINANL, JBI-EBP and Web of Science from inception to October 2023.

METHODS

Methodological quality was assessed by Risk of Bias Tool. Meta-analyses were assessed by Review Manager 5.4. Subgroup analyses were conducted to explore the influence of study design. Grading of Recommendations Assessment, Development and Evaluation approach was adopted to assess the certainty of evidence.

RESULTS

Twenty-five randomized controlled trials (1178 participants) were included. Virtual reality-based cognitive interventions demonstrated moderate-to-large effects in improving global cognitive function (SMD = 0.43; 95% CI [0.01, 0.85]), executive function (SMD = 0.84; 95% CI [0.25, 1.43]) and memory (SMD = 0.65; 95% CI [0.15, 1.16]) compared to control treatments. No significant effects were found on language, visuospatial ability and activities of daily living. Subgroup analyses indicated one-on-one coaching, individualized design and dynamic difficulty adjustment, and interventions lasting ≥ 6 weeks had particularly enhanced effects, especially for executive function.

CONCLUSIONS

Virtual reality-based cognitive interventions improve global cognitive function, executive function and memory among stroke patients.

IMPLICATIONS FOR THE PATIENT CARE

This review underscores the broad cognitive advantages offered by virtual technology, suggesting its potential integration into standard stroke rehabilitation protocols for enhanced cognitive recovery.

IMPACT

The study identifies key factors in virtual technology interventions that effectively improve cognitive function among stroke patients, offering healthcare providers a framework for leveraging such technology to optimize cognitive outcomes in stroke rehabilitation.

REPORTING METHOD

PRISMA 2020 statement.

PROSPERO REGISTRATION NUMBER

CRD42022342668.

Effects of Augmented Reality-, Virtual Reality-, and Mixed Reality-Based Training on Objective Performance Measures and Subjective Evaluations in Manual Assembly Tasks: A Scoping Review

OBJECTIVE

The present scoping review aims to transform the diverse field of research on the effects of mixed reality-based training on performance in manual assembly tasks into comprehensive statements about industrial needs for and effects of mixed reality-based training.

BACKGROUND

Technologies such as augmented and virtual reality, referred to as mixed reality, are seen as promising media for training manual assembly tasks. Nevertheless, current literature shows partly contradictory results, which is due to the diversity of the hardware used, manual assembly tasks as well as methodological approaches to investigate the effects of mixed reality-based training.

METHOD

Following the methodological approach of a scoping review, we selected 24 articles according to predefined criteria and analyzed them concerning five key aspects: (1) the needs in the industry for mixed reality-based training, (2) the actual use and classification of mixed reality technologies, (3) defined measures for evaluating the outcomes of mixed reality-based training, (4) findings on objectively measured performance and subjective evaluations, as well as (5) identified research gaps.

RESULTS

Regarding the improvement of performance and effectiveness through mixed reality-based training, promising results were found particularly for augmented reality-based training, while virtual reality-based training is mostly-but not consistently-as good as traditional training.

APPLICATION

Mixed reality-based training is still not consistently better, but mostly at least as good as traditional training. However, depending on the use case and technology used, the training outcomes in terms of assembly performance and subjective evaluations show promising results of mixed reality-based training.

Enhancing learning outcomes through multisensory integration: A fMRI study of audio-visual training in virtual reality

The integration of information from different sensory modalities is a fundamental process that enhances perception and performance in real and virtual environments (VR). Understanding these mechanisms, especially during learning tasks that exploit novel multisensory cue combinations provides opportunities for the development of new rehabilitative interventions. This study aimed to investigate how functional brain changes support behavioural performance improvements during an audio-visual (AV) learning task. Twenty healthy participants underwent a 30 min daily VR training for four weeks. The task was an AV adaptation of a 'scanning training' paradigm that is commonly used in hemianopia rehabilitation. Functional magnetic resonance imaging (fMRI) and performance data were collected at baseline, after two and four weeks of training, and four weeks post-training. We show that behavioural performance, operationalised as mean reaction time reduction in VR, significantly improves. In separate tests in a controlled laboratory environment, we showed that the behavioural performance gains in the VR training environment transferred to a significant mean RT reduction for the trained AV voluntary task on a computer screen. Enhancements were observed in both the visual-only and AV conditions, with the latter demonstrating a faster response time supported by the presence of audio cues. The behavioural learning effect also transfers to two additional tasks that were tested: a visual search task and an involuntary visual task. Our fMRI results reveal an increase in functional activation (BOLD signal) in multisensory brain regions involved in early-stage AV processing: the thalamus, the caudal inferior parietal lobe and cerebellum. These functional changes were only observed for the trained, multisensory, task and not for unimodal visual stimulation. Functional activation changes in the thalamus were significantly correlated to behavioural performance improvements. This study demonstrates that incorporating spatial auditory cues to voluntary visual training in VR leads to augmented brain activation changes in multisensory integration, resulting in measurable performance gains across tasks. The findings highlight the potential of VR-based multisensory training as an effective method for enhancing cognitive function and as a potentially valuable tool in rehabilitative programmes.

An immersive multisensory virtual reality approach to the study of human-built environment interactions: Technical workflows

Virtual Reality technology has gained increased attention due to its capacity to provide immersive and interactive experiences to its users. Although increasing evidence has suggested that incorporating multisensory components in VR can promote the sense of presence and improve user performance, most of the current VR applications are limited to visual and auditory senses. In this article, a novel method of integrating thermal-related devices (heat lamps and fans) into Virtual Reality was developed. Automated interaction with the thermal-related devices was achieved using Arduino-based control module with its program embedded into the VR platform-Unity. The functions, hardware and software requirements of the multisensory Virtual Reality system as well as the step-by-step procedures are detailed to provide a reproducible workflow for future applications.•A practical workflow to integrate thermal apparatus into Virtual Reality.•Dynamic airflow and radiative heating incorporated into Virtual Reality.•Automated process to allow user interaction with the thermal components in Virtual Reality.

Sensorimotor Uncertainty of Immersive Virtual Reality Environments for People in Pain: Scoping Review

INTRODUCTION

Decision making and action execution both rely on sensory information, and their primary objective is to minimise uncertainty. Virtual reality (VR) introduces uncertainty due to the imprecision of perceptual information. The concept of "sensorimotor uncertainty" is a pivotal element in the interplay between perception and action within the VR environment. The role of immersive VR in the four stages of motor behaviour decision making in people with pain has been previously discussed. These four processing levels are the basis to understand the uncertainty that a patient experiences when using VR: sensory information, current state, transition rules, and the outcome obtained.

METHODS

This review examines the different types of uncertainty that a patient may experience when they are immersed in a virtual reality environment in a context of pain. Randomised clinical trials, a secondary analysis of randomised clinical trials, and pilot randomised clinical trials related to the scope of Sensorimotor Uncertainty in Immersive Virtual Reality were included after searching.

RESULTS

Fifty studies were included in this review. They were divided into four categories regarding the type of uncertainty the intervention created and the stage of the decision-making model.

CONCLUSIONS

Immersive virtual reality makes it possible to alter sensorimotor uncertainty, but studies of higher methodological quality are needed on this topic, as well as an exploration into the patient profile for pain management using immersive VR.

The role of sense of presence in expressing cognitive abilities in a virtual reality task: an initial validation study

There is a raised interest in literature to use Virtual Reality (VR) technology as an assessment tool for cognitive domains. One of the essential advantages of transforming tests in an immersive virtual environment is the possibility of automatically calculating the test's score, a time-consuming process under natural conditions. Although the characteristics of VR can deliver different degrees of immersion in a virtual environment, the sense of presence could jeopardize the evolution of these practices. The sense of presence results from a complex interaction between human, contextual factors, and the VR environment. The present study has two aims: firstly, it contributes to the validation of a virtual version of the naturalistic action test (i.e., virtual reality action test); second, it aims to evaluate the role of sense of presence as a critical booster of the expression of cognitive abilities during virtual reality tasks. The study relies on healthy adults tested in virtual and real conditions in a cross-over research design. The study's results support the validity of the virtual reality action test. Furthermore, two structural equation models are tested to comprehend the role of sense of presence as a moderator in the relationship between cognitive abilities and virtual task performance.

Walk the plank! Using mobile electroencephalography to investigate emotional lateralization of immersive fear in virtual reality

Most studies on emotion processing induce emotions through images or films. However, this method lacks ecological validity, limiting generalization to real-life emotion processing. More realistic paradigms using virtual reality (VR) may be better suited to investigate authentic emotional states and their neuronal correlates. This pre-registered study examines the neuronal underpinnings of naturalistic fear, measured using mobile electroencephalography (EEG). Seventy-five healthy participants walked across a virtual plank which extended from the side of a skyscraper-either 80 storeys up (the negative condition) or at street level (the neutral condition). Subjective ratings showed that the negative condition induced feelings of fear. Following the VR experience, participants passively viewed negative and neutral images from the international affective picture system (IAPS) outside of VR. We compared frontal alpha asymmetry between the plank and IAPS task and across valence of the conditions. Asymmetry indices in the plank task revealed greater right-hemispheric lateralization during the negative VR condition, relative to the neutral VR condition and to IAPS viewing. Within the IAPS task, no significant asymmetries were detected. In summary, our findings indicate that immersive technologies such as VR can advance emotion research by providing more ecologically valid ways to induce emotion.

Presence and simulator sickness predict the usability of a virtual reality attention task

Attention is the ability to actively process specific information within one's environment over longer periods of time while disregarding other details. Attention is an important process that contributes to overall cognitive performance from performing every day basic tasks to complex work activities. The use of virtual reality (VR) allows study of the attention processes in realistic environments using ecological tasks. To date, research has focused on the efficacy of VR attention tasks in detecting attention impairment, while the impact of the combination of variables such as mental workload, presence and simulator sickness on both self-reported usability and objective attention task performance in immersive VR has not been examined. The current study tested 87 participants on an attention task in a virtual aquarium using a cross-sectional design. The VR task followed the continuous performance test paradigm where participants had to respond to correct targets and ignore non-targets over 18 min. Performance was measured using three outcomes: omission (failing to respond to correct targets), commission errors (incorrect responses to targets) and reaction time to correct targets. Measures of self-reported usability, mental workload, presence and simulator sickness were collected. The results showed that only presence and simulator sickness had a significant impact on usability. For performance outcomes, simulator sickness was significantly and weakly associated with omission errors, but not with reaction time and commission errors. Mental workload and presence did not significantly predict performance. Our results suggest that usability is more likely to be negatively impacted by simulator sickness and lack of presence than performance and that usability and attention performance are linked. They highlight the importance of considering factors such as presence and simulator sickness in attention tasks as these variables can impact usability.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10055-023-00782-3.

Evoked sensory stimulation of the eating environment, impacts feeling of presence and food desires in an online environment

Online food choices are often made outside a regular food environment and suffer from sensory deprivation. The present study investigated if evoked multi-sensory stimulation can drive context specific food desires in an online environment. In a randomised between subject design, participants expressed their food desire on a visual analogue scale and feeling of presence (e.g., did you feel present on a beach) on a Likert scale, whilst looking online at a picture and reading a neutral description of a sensory laboratory (control condition), looking at a photo of a beach and reading a neutral description (beach condition), or looking at a photo of a beach and reading a sensory based description (beach + ). Participants (n = 725 participants, 622 females) who saw the beach photo increased their desire for cold, but not neutral foods (p < 0.05), those who were exposed to the sensory description in addition to the photo showed a higher desire for cold foods compared to those who just saw the beach photo (p < 0.001). These effects were modulated by an increased feeling of presence and how often participants visited the beach. Participants with a higher feeling of presence showed a higher desire for cold foods (p < 0.05). Food desires of those who visited the beach often were more impacted by the evoked sensory stimulation than food desires of those who visited the beach rarely. Food desires created in an online environment can be influenced by visual, and text based evoked sensory stimulation as long as consumers' feeling of presence in is high. The results can inform public health professionals how to impact healthy food choices in an online environment.

Metaverse!

This study aimed to synthesize the literature on Metaverse to highlight its current research, opportunities, and applications in e-healthcare and education to reduce inequalities and for delivering fair and equal opportunities and solutions. The authors employed preferred reporting items for systematic reviews and meta-analyses (PRISMA) to rapidly map the field of a metaverse in health and education. Two major application domains emerged in the literature from the 88 research publications, which include (1) Metaverse in healthcare and (2) Metaverse in education. This study will act as a road map to help academics who desire to continue their research work in the Metaverse for various healthcare and educational services. However, its implementation is required in the future to improve mental healthcare and the effectiveness of mental health services, particularly in low and medium-income (LMIC) and conflict-affected areas.

Evaluating Virtual Hand Illusion through Realistic Appearance and Tactile Feedback

We conducted a virtual reality study to explore virtual hand illusion through three levels of appearance (Appearance dimension: realistic vs. pixelated vs. toon hand appearances) and two levels of tactile feedback (Tactile dimension: no tactile vs. tactile feedback). We instructed our participants to complete a virtual assembly task in this study. Immediately afterward, we asked them to provide self-reported ratings on a survey that captured presence and five embodiment dimensions (hand ownership, touch sensation, agency and motor control, external appearance, and response to external stimuli). The results of our study indicate that (1) tactile feedback generated a stronger sense of presence, touch sensation, and response to external stimuli; (2) the pixelated hand appearance provided the least hand ownership and external appearance; and (3) in the presence of the pixelated hand, prior virtual reality experience of participants impacted their agency and motor control and their response to external stimuli ratings. This paper discusses our findings and provides design considerations for virtual reality applications with respect to the realistic appearance of virtual hands and tactile feedback.

"Sense of Presence" in Immersive Virtual Reality Environment: An Evolutionary Concept Analysis

There is no concept analysis in the literature about "sense of presence" in immersive virtual reality environment in the context of nursing clinical education. The purpose of this article is to explore the meaning and derive an operational definition of the concept of sense of presence within the context of immersive virtual reality environment in nursing clinical education. Using Rodgers' approach, this evolutionary concept analysis will clarify the concept's surrogate and related terms, attributes, antecedents, and consequences. A clearer conceptualization is paramount to help provide knowledge for nurses and other health care professionals, thus guiding their practice, theory, and research.

Vibrotactile feedback in virtual motor learning: A systematic review

Vibrotactile feedback can be effectively applied to motor (physical) learning in virtual environments, as it can provide task-intrinsic and augmented feedback to users, assisting them in enhancing their motor performance. This review investigates current uses of vibrotactile feedback systems in motor learning applications built upon virtual environments by systematically synthesizing 24 peer-reviewed studies. We aim to understand: (1) the current state of the science of using real-time vibrotactile feedback in virtual environments for aiding the acquisition (or improvement) of motor skills, (2) the effectiveness of using vibrotactile feedback in such applications, and (3) research gaps and opportunities in current technology. We used the Sensing-Analysis-Assessment-Intervention framework to assess the scientific literature in our review. The review identifies several research gaps in current studies, as well as potential design considerations that can improve vibrotactile feedback systems in virtual motor learning applications, including the selection and placement of feedback devices and feedback designs.

Vibrating Tilt Platform Enhancing Immersive Experience in VR

One of the disadvantages of virtual reality systems of the past was the fact that they had visual-only interfaces. However, with the development of haptic technology, peripheral solutions to enhance the VR experience are gaining momentum, and many haptic systems are being developed for deepening VR immersion. This paper deals with the concept of a vibrating tilt platform allowing the change of three angles of inclination to thus reinforce the VR experience. The proposed system is flexible and adaptable to different sports, health, and education applications. In this paper, we present the mechanical and mechatronics aspects of the platform and usability testing results based on an immersive geological experience application. The first tests were studied in terms of the cyber sickness, perceived realism, and anxiety through both subjective (questionnaires) and objective (electroencephalogram) measurements. The results indicate that our platform increased anxiety levels and was perceived as realistic. Adding vibrations and tilting had a considerable impact on the immersion level and brain activity.

Do Multisensory Stimuli Benefit the Virtual Reality Experience? A Systematic Review

The majority of virtual reality (VR) applications rely on audiovisual stimuli and do not exploit the addition of other sensory cues that could increase the potential of VR. This systematic review surveys the existing literature on multisensory VR and the impact of haptic, olfactory, and taste cues over audiovisual VR. The goal is to identify the extent to which multisensory stimuli affect the VR experience, which stimuli are used in multisensory VR, the type of VR setups used, and the application fields covered. An analysis of the 105 studies that met the eligibility criteria revealed that 84.8 percent of the studies show a positive impact of multisensory VR experiences. Haptics is the most commonly used stimulus in multisensory VR systems (86.6 percent). Non-immersive and immersive VR setups are preferred over semi-immersive setups. Regarding the application fields, a considerable part was adopted by health professionals and science and engineering professionals. We further conclude that smell and taste are still underexplored, and they can bring significant value to VR applications. More research is recommended on how to synthesize and deliver these stimuli, which still require complex and costly apparatus be integrated into the VR experience in a controlled and straightforward manner.

Influence of body visualization in VR during the execution of motoric tasks in different age groups

Virtual reality (VR) has become a common tool and is often considered for sport-specific purposes. Despite the increased usage, the transfer of VR-adapted skills into the real-world (RW) has not yet been sufficiently studied, and it is still unknown how much of the own body must be visible to complete motoric tasks within VR. In addition, it should be clarified whether older adults also need to perceive their body within VR scenarios to the same extent as younger people extending the usability. Therefore, younger (18-30 years old) and elderly adults (55 years and older) were tested (n = 42) performing a balance-, grasping- and throwing task in VR (HMD based) accompanied with different body visualization types in VR and in the RW having the regular visual input of body's components. Comparing the performances between the age groups, the time for completion, the number of steps (balance task), the subjective estimation of difficulty, the number of errors, and a rating system revealing movements' quality were considered as examined parameters. A one-way ANOVA/Friedmann with repeated measurements with factor [body visualization] was conducted to test the influence of varying body visualizations during task completion. Comparisons between the conditions [RW, VR] were performed using the t-Tests/Wilcoxon tests, and to compare both age groups [young, old], t-Tests for independent samples/Mann-Whitney-U-Test were used. The analyses of the effect of body visualization on performances showed a significant loss in movement's quality when no body part was visualized (p < .05). This did not occur for the elderly adults, for which no influence of the body visualization on their performance could be proven. Comparing both age groups, the elderly adults performed significantly worse than the young age group in both conditions (p < .05). In VR, both groups showed longer times for completion, a higher rating of tasks' difficulty in the balance and throwing task, and less performance quality in the grasping task. Overall, the results suggest using VR for the elderly with caution to the task demands, and the visualization of the body seemed less crucial for generating task completion. In summary, the actual task demands in VR could be successfully performed by elderly adults, even once one has to reckon with losses within movement's quality. Although more different movements should be tested, basic elements are also realizable for elderly adults expanding possible areas of VR applications.

Visual fatigue induced by watching virtual reality device and the effect of anisometropia

The effect of small anisometropia on visual fatigue when using virtual reality (VR) devices was investigated. Participants (n = 34) visited three times. In the first visit, VR exposure (10 min) was conducted with the full correction of the refractive error of both eyes. Experimental anisometropia was induced by adding a + 1.0 dioptre spherical lens either on the dominant eyes in the second visit or on the non-dominant eyes in the third visit. At each visit, the participants played a predetermined video game using a head-mounted display VR for 10 min. Visual fatigue was assessed before and after playing VR game using the Virtual Reality Symptom Questionnaire (VRSQ) and high-frequency component of accommodative microfluctuation. Results showed that watching VR induced significant increase of VRSQ score, significant decrease in the maximum accommodation power and objective increase in visual fatigue. Experimental anisometropia induction either on the dominant or non-dominant eyes did not aggravate visual fatigue. Practitioner summary: Mild differences in refractive error (up to 1.0 dioptre) between both eyes do not significantly increase ocular fatigue by viewing virtual reality device (10 min). The impact of small anisometropia may be limited in developing a virtual reality device. Abbreviations: VR: virtual reality; VRSQ: virtual reality symptom questionnaire; HMD: head-mounted display; HFC: high-frequency component.

Vision Impairment Provides New Insight Into Self-Motion Perception

Purpose

Leading causes of irreversible blindness such as age-related macular degeneration (AMD) and glaucoma can, respectively, lead to central or peripheral vision loss. The ability of sufferers to process visual motion information can be impacted even during early stages of eye disease. We used head-mounted display virtual reality as a tool to better understand how vision changes caused by eye diseases directly affect the processing of visual information critical for self-motion perception.

Methods

Participants with intermediate AMD or early manifest glaucoma with near-normal visual acuities and visual fields were recruited for this study. We examined their experiences of self-motion in depth (linear vection), spatial presence, and cybersickness when viewing radially expanding patterns of optic flow simulating different speeds of self-motion in depth. Viewing was performed with the head stationary (passive condition) or while making lateral-sway head movements (active conditions).

Results

Participants with AMD (i.e., central visual field loss) were found to have greater vection strength and spatial presence, compared to participants with normal visual fields. However, participants with glaucoma (i.e., peripheral visual field loss) were found to have lower vection strength and spatial presence, compared to participants with normal visual fields. Both AMD and glaucoma groups reported reduced severity in cybersickness compared to healthy normals.

Conclusions

These findings strongly support the view that perceived self-motion is differentially influenced by peripheral versus central vision loss, and that patients with different visual field defects are oppositely biased when processing visual cues to self-motion perception.

Exploring the Use of Virtual Reality for the Delivery and Practice of Stress-Management Exercises

Background

Mindfulness-based interventions may benefit healthcare professionals with burnout symptoms. Virtual reality (VR) may reduce initial difficulty of engaging in mindfulness exercises and increase participants’ engagement through immersion and presence.

Aim

The aim was to investigate how VR affects participants’ experience of engagement with mindfulness practice, and its impact on quality of practice and negative mood states.

Methods

Fifty-one healthcare professionals were randomized to receive either a visualization or non-visualization mindfulness practice, to compare the quality of practice through the use of audio only vs. with a virtual reality interface. Selected self-reported measures were collected during the session (immersion, quality and difficulty of practice, mood states and likelihood for future practice).

Results

Results showed that order instead of type of modality administered made a difference in quality of mindfulness practice. A greater sense of presence was reported with VR if administered after audio (F = 4.810, p = 0.033, Partial η² = 0.093). Further, participants described difficulty practicing with audio if administered after VR (F = 4.136, p = 0.048, Partial η² = 0.081). Additionally, lower mood disturbance was reported with VR if administered after audio (F = 8.116, p = 0.006, Partial η² = 0.147). Qualitative responses echoed a preference for VR to engage better, in addition to improved mood states after practice.

Conclusion

Findings suggest that VR has the potential to provide healthcare professionals with an alternative or a supplement to conventional mindfulness practice.

Assessing Visual Exploratory Activity of Athletes in Virtual Reality Using Head Motion Characteristics

Maximizing performance success in sports is about continuous learning and adaptation processes. Aside from physiological, technical and emotional performance factors, previous research focused on perceptual skills, revealing their importance for decision-making. This includes deriving relevant environmental information as a result of eye, head and body movement interaction. However, to evaluate visual exploratory activity (VEA), generally utilized laboratory settings have restrictions that disregard the representativeness of assessment environments and/or decouple coherent cognitive and motor tasks. In vivo studies, however, are costly and hard to reproduce. Furthermore, the application of elaborate methods like eye tracking are cumbersome to implement and necessitate expert knowledge to interpret results correctly. In this paper, we introduce a virtual reality-based reproducible assessment method allowing the evaluation of VEA. To give insights into perceptual-cognitive processes, an easily interpretable head movement-based metric, quantifying VEA of athletes, is investigated. Our results align with comparable in vivo experiments and consequently extend them by showing the validity of the implemented approach as well as the use of virtual reality to determine characteristics among different skill levels. The findings imply that the developed method could provide accurate assessments while improving the control, validity and interpretability, which in turn informs future research and developments.

Transfer of training-Virtual reality training with augmented multisensory cues improves user experience during training and task performance in the real world

Virtual reality (VR) can create safe, cost-effective, and engaging learning environments. It is commonly assumed that improvements in simulation fidelity lead to better learning outcomes. Some aspects of real environments, for example vestibular or haptic cues, are difficult to recreate in VR, but VR offers a wealth of opportunities to provide additional sensory cues in arbitrary modalities that provide task relevant information. The aim of this study was to investigate whether these cues improve user experience and learning outcomes, and, specifically, whether learning using augmented sensory cues translates into performance improvements in real environments. Participants were randomly allocated into three matched groups: Group 1 (control) was asked to perform a real tyre change only. The remaining two groups were trained in VR before performance was evaluated on the same, real tyre change task. Group 2 was trained using a conventional VR system, while Group 3 was trained in VR with augmented, task relevant, multisensory cues. Objective performance, time to completion and error number, subjective ratings of presence, perceived workload, and discomfort were recorded. The results show that both VR training paradigms improved performance for the real task. Providing additional, task-relevant cues during VR training resulted in higher objective performance during the real task. We propose a novel method to quantify the relative performance gains between training paradigms that estimates the relative gain in terms of training time. Systematic differences in subjective ratings that show comparable workload ratings, higher presence ratings and lower discomfort ratings, mirroring objective performance measures, were also observed. These findings further support the use of augmented multisensory cues in VR environments as an efficient method to enhance performance, user experience and, critically, the transfer of training from virtual to real environment scenarios.

The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence

Real-world experience is typically multimodal. Evidence indicates that the facilitation in the detection of multisensory stimuli is modulated by the perceptual load, the amount of information involved in the processing of the stimuli. Here, we used a realistic virtual reality environment while concomitantly acquiring Electroencephalography (EEG) and Galvanic Skin Response (GSR) to investigate how multisensory signals impact target detection in two conditions, high and low perceptual load. Different multimodal stimuli (auditory and vibrotactile) were presented, alone or in combination with the visual target. Results showed that only in the high load condition, multisensory stimuli significantly improve performance, compared to visual stimulation alone. Multisensory stimulation also decreases the EEG-based workload. Instead, the perceived workload, according to the "NASA Task Load Index" questionnaire, was reduced only by the trimodal condition (i.e., visual, auditory, tactile). This trimodal stimulation was more effective in enhancing the sense of presence, that is the feeling of being in the virtual environment, compared to the bimodal or unimodal stimulation. Also, we show that in the high load task, the GSR components are higher compared to the low load condition. Finally, the multimodal stimulation (Visual-Audio-Tactile-VAT and Visual-Audio-VA) induced a significant decrease in latency, and a significant increase in the amplitude of the P300 potentials with respect to the unimodal (visual) and visual and tactile bimodal stimulation, suggesting a faster and more effective processing and detection of stimuli if auditory stimulation is included. Overall, these findings provide insights into the relationship between multisensory integration and human behavior and cognition.

An intelligent virtual reality technology in the teaching of art creation and design in colleges and universities

The popularization of virtual reality technology has become a brand-new carrier of knowledge and information dissemination and a new tool for art creation and design teaching in Colleges and universities. In this paper, first of all, from the three aspects of the definition and characteristics of virtual reality technology, the integration of virtual reality technology into art design teaching, and the significance of using virtual reality technology in art design teaching, the reform of art creation and design teaching in Colleges and Universities under virtual reality technology is discussed. In order to carry out the application research, this paper then chooses to carry on the simulation modeling teaching of the bamboo forest in the intelligent virtual reality environment, making full use of the intelligent modeling tools, rendering tools and mapping tools to complete the teaching application examples of the art modeling design in Colleges and universities. In the specific teaching of art creation and design in Colleges and universities, the creation and design modeling of individual bamboos, the creation and design modeling of bamboos forest, and the creation and design modeling of different lighting effects and scene rendering are carried out respectively. Finally, from the advantages of intelligent virtual reality technology in art creation and design of colleges and universities, the expansion and promotion of teaching resources, and the synergistic effect of teaching results, we analyze the strategies of art creation and design teaching in Colleges and Universities under the background of intelligent virtual reality technology in detail.

The effect of water immersion on vection in virtual reality

Research about vection (illusory self-motion) has investigated a wide range of sensory cues and employed various methods and equipment, including use of virtual reality (VR). However, there is currently no research in the field of vection on the impact of floating in water while experiencing VR. Aquatic immersion presents a new and interesting method to potentially enhance vection by reducing conflicting sensory information that is usually experienced when standing or sitting on a stable surface. This study compares vection, visually induced motion sickness, and presence among participants experiencing VR while standing on the ground or floating in water. Results show that vection was significantly enhanced for the participants in the Water condition, whose judgments of self-displacement were larger than those of participants in the Ground condition. No differences in visually induced motion sickness or presence were found between conditions. We discuss the implication of this new type of VR experience for the fields of VR and vection while also discussing future research questions that emerge from our findings.

A Novel Virtual Reality Medical Image Display System for Group Discussions of Congenital Heart Disease: Development and Usability Testing

Background

The complex 3-dimensional (3D) nature of anatomical abnormalities in congenital heart disease (CHD) necessitates multidisciplinary group discussions centered around the review of medical images such as magnetic resonance imaging. Currently, group viewings of medical images are constrained to 2-dimensional (2D) cross-sectional displays of 3D scans. However, 2D display methods could introduce additional challenges since they require physicians to accurately reconstruct the images mentally into 3D anatomies for diagnosis, staging, and planning of surgery or other therapies. Virtual reality (VR) software may enhance diagnosis and care of CHD via 3D visualization of medical images. Yet, present-day VR developments for medicine lack the emphasis on multiuser collaborative environments, and the effect of displays and level of immersion for diagnosing CHDs have not been studied.

Objective

The objective of the study was to evaluate and compare the diagnostic accuracies and preferences of various display systems, including the conventional 2D display and a novel group VR software, in group discussions of CHD.

Methods

A total of 22 medical trainees consisting of 1 first-year, 10 second-year, 4 third-year, and 1 fourth-year residents and 6 medical students, who volunteered for the study, were formed into groups of 4 to 5 participants. Each group discussed three diagnostic cases of CHD with varying structural complexity using conventional 2D display and group VR software. A group VR software, Cardiac Review 3D, was developed by our team using the Unity engine. By using different display hardware, VR was classified into nonimmersive and full-immersive settings. The discussion time, diagnostic accuracy score, and peer assessment were collected to capture the group and individual diagnostic performances. The diagnostic accuracies for each participant were scored by two experienced cardiologists following a predetermined answer rubric. At the end of the study, all participants were provided a survey to rank their preferences of the display systems for performing group medical discussions.

Results

Diagnostic accuracies were highest when groups used the full-immersive VR compared with the conventional and nonimmersive VR (χ²₂=9.0, P=.01) displays. Differences between the display systems were more prominent with increasing case complexity (χ²₂=14.1, P<.001) where full-immersive VR had accuracy scores that were 54.49% and 146.82% higher than conventional and nonimmersive VR, respectively. The diagnostic accuracies provided by the two cardiologists for each participant did not statistically differ from each other (t=–1.01, P=.31). The full-immersive VR was ranked as the most preferred display for performing group CHD discussions by 68% of the participants.

Conclusions

The most preferred display system among medical trainees for visualizing medical images during group diagnostic discussions is full-immersive VR, with a trend toward improved diagnostic accuracy in complex anatomical abnormalities. Immersion is a crucial feature of displays of medical images for diagnostic accuracy in collaborative discussions.

Spatial Presence, Performance, and Behavior between Real, Remote, and Virtual Immersive Environments

Spatial presence encompasses the user's ability to experience a sense of "being there". While particular attention was given to assess spatial presence in real and virtual environments, few have been interested in measuring it in telepresence situations. To bridge this gap, the present work introduces a study that compares the execution of a task in three conditions: a real physical environment, a remote environment via a telepresence system, and a virtual simulation of the real environment. Following a within-subject design, 27 participants performed a navigation task consisting in following a route while avoiding obstacles. Spatial presence and five related factors (affordance, enjoyment, attention allocation, reality, and cybersickness) were evaluated using a presence questionnaire. In addition, performance measures were gathered regarding environment recollection and task execution. The evaluation also included a behavioral metric measured by obstacle avoidance distance extracted from participants' trajectories. Results indicated a higher presence in the real environment, along with the best performance measures. No difference was found in spatial presence between the remote and the virtual conditions, although a higher degree of affordance and enjoyment was attributed to the virtual environment, and a higher degree of reality was attributed to the remote environment. The number of collisions was found to be lower in the remote condition compared to the virtual condition. Similarly, the avoidance distance was also bigger (and almost similar) in the real and the remote environments compared to the virtual environment indicating a greater caution of participants. These cues highlight that the behavior of participants in the remote condition was closer to their behavior in the real situation than it was in the virtual condition. Furthermore, positive correlations were found between the reality factor and two of the three performance measures, as well as with the behavioral metric. This suggests that the degree of physical existence of the space in which participants operate can influence their performance and behavior.

Exergaming With Beat Saber: An Investigation of Virtual Reality Aftereffects

BACKGROUND

Virtual reality (VR) exergaming has the potential to target sedentary behavior. Immersive environments can distract users from the physical exertion of exercise and can motivate them to continue exergaming. Despite the recent surge in VR popularity, numerous users still experience VR sickness from using head-mounted displays (HMDs). Apart from the commonly assessed self-reported symptoms, depth perception and cognition may also be affected. Considering the potential benefits of VR exergaming, it is crucial to identify the adverse effects limiting its potential and continued uptake.

OBJECTIVE

This study aims to investigate the consequences of playing one of the most popular VR exergames for 10 and 50 min on aspects of vision, cognition, and self-reported VR sickness.

METHODS

A total of 36 participants played an exergame, called Beat Saber, using an HMD. A repeated measures within-subject design was conducted to assess changes in vision, cognition, and well-being after short (10 min) and long (50 min) durations of VR exposure. We measured accommodation, convergence, decision speed, movement speed, and self-reported sickness at 3 test periods-before VR, immediately after VR, and 40 min after VR (late).

RESULTS

Beat Saber was well tolerated, as there were no dropouts due to sickness. For most participants, any immediate aftereffects were short-lived and returned to baseline levels after 40 min of exiting VR. For both short and long exposures, there were changes in accommodation (F1,35=8.424; P=.006) and convergence (F1,35=7.826; P=.008); however, in the late test period, participants returned to baseline levels. Measures on cognition revealed no concern. The total simulator sickness questionnaire (SSQ) scores increased immediately after VR (F1,35=26.515; P<.001) and were significantly higher for long compared with short exposures (t35=2.807; P=.03), but there were no differences in exposure duration in the late test period, with scores returning to baseline levels. Although at a group level, participants' sickness levels returned to baseline 40 min after VR exposure, approximately 14% of the participants still reported high levels of sickness in the late test period after playing 50 min of Beat Saber. We also showed that the participants who experienced a high level of sickness after a short exposure were almost certain to experience a high level of symptoms after a longer exposure.

CONCLUSIONS

Irrespective of the duration of exposure, this study found no strong evidence for adverse symptoms 40 min after exiting VR; however, some individuals still reported high levels of VR sickness at this stage. We recommend that users commit to a waiting period after exiting VR to ensure that any aftereffects have deteriorated. Exergames in HMDs have the potential to encourage people to exercise but are understudied, and the aftereffects of exergaming need to be closely monitored to ensure that VR exergames can reach their full potential.

Effectiveness of Virtual Versus Physical Training: The Case of Assembly Tasks, Trainer's Verbal Assistance, and Task Complexity

Virtual immersive training (VIT) systems based on gamification of tasks are increasingly employed to train assembly workers. In this article, we present a study that compares the effectiveness of virtual and physical training for teaching a bimanual assembly task and in a novel approach, we introduce task complexity (T$_{\mathrm{CXB}}$ CXB ) as an indicator of assembly errors during final assembly. In a between-subjects experiment, 100 participants were trained to assemble a 3-D cube in one of the four conditions (physical, virtual and with trainer's verbal assistance or not). The results demonstrate that the best-performing conditions, both in terms of successful assemblies and time performance, are the ones that the physical objects are included in the training, whereas no significant difference is found when the trainer's verbal assistance is present or absent during training. Additionally, we address the validity of a practical T$_{\mathrm{CXB}}$ CXB list as a tool for supporting the design of VIT systems.

A Mixed Reality system for the simulation of emergency and first-aid scenarios

Simulation is a powerful learning tool, as it allows gaining direct experience in a controlled and repeatable way. However, the simulation is effective when it is able to reproduce the real conditions and when the user feels him/herself immersed and present in the situation. With the aim of improving these critical points, we propose an immersive virtual reality system for first-aid handling. Specifically, we increase the visual realism of medical mannequins and the contextualization, and we add the touch feedback by mapping the real mannequin into its virtual representation. Moreover, the interaction is performed by using a virtual representation of the users own hands by allowing a more realistic execution of tasks. The results show a good accuracy in the mapping between the real and the virtual mannequin, and a high degree of presence for both the control group and the medical one. These results and the low values of simulator sickness reported during the experiment are a good starting point for the use of the proposed mixed reality system in simulation scenarios.

Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review

In order to take advantage of the potential offered by the medium of virtual reality (VR), it will be essential to develop an understanding of how to maximize the desirable experience of "presence" in a virtual space ("being there"), and how to minimize the undesirable feeling of "cybersickness" (a constellation of discomfort symptoms experienced in VR). Although there have been frequent reports of a possible link between the observer's sense of presence and the experience of bodily discomfort in VR, the amount of literature that discusses the nature of the relationship is limited. Recent research has underlined the possibility that these variables have shared causes, and that both factors may be manipulated with a single approach. This review paper summarizes the concepts of presence and cybersickness and highlights the strengths and gaps in our understanding about their relationship. We review studies that have measured the association between presence and cybersickness, and conclude that the balance of evidence favors a negative relationship between the two factors which is driven principally by sensory integration processes. We also discuss how system immersiveness might play a role in modulating both presence and cybersickness. However, we identify a serious absence of high-powered studies that aim to reveal the nature of this relationship. Based on this evidence we propose recommendations for future studies investigating presence, cybersickness, and other related factors.