Comparison of Oculus Rift and HTC Vive: Feasibility for Virtual Reality-Based Exploration, Navigation, Exergaming, and Rehabilitation

Using immersive virtual reality to remotely examine performance differences between dominant and non-dominant hands

Circle drawing may be a useful task to study upper-limb function in patient populations. However, previous studies rely on expensive and bulky robotics to measure performance. For clinics or hospitals with limited budgets and space, this may be unfeasible. Virtual reality (VR) provides a portable and low-cost tool with integrated motion capture. It offers potentially a more feasible medium by which to assess upper-limb motor function. Prior to use with patient populations, it is important to validate and test the capabilities of VR with healthy users. This study examined whether a VR-based circle drawing task, completed remotely using participant's own devices, could capture differences between movement kinematics of the dominant and non-dominant hands in healthy individuals. Participants (n = 47) traced the outline of a circle presented on their VR head-mounted displays with each hand, while the positions of the hand-held controllers were continuously recorded. Although there were no differences observed in the size or roundness of circles drawn with each hand, consistent with prior literature our results did show that the circles drawn with the dominant hand were completed faster than those with the non-dominant hand. This provides preliminary evidence that a VR-based circle drawing task may be a feasible method for detecting subtle differences in function in clinical populations.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10055-023-00794-z.

Quantification and Rehabilitation of Unilateral Spatial Neglect in Immersive Virtual Reality: A Validation Study in Healthy Subjects

Unilateral spatial neglect is a common sensorimotor disorder following the occurrence of a stroke, for which prismatic adaptation is a promising rehabilitation method. However, the use of prisms for rehabilitation often requires the use of specific equipment that may not be available in clinics. To address this limitation, we developed a new software package that allows for the quantification and rehabilitation of unilateral spatial neglect using immersive virtual reality. In this study, we compared the effects of virtual and real prisms in healthy subjects and evaluated the performance of our virtual reality tool (HTC Vive) against a validated motion capture tool. Ten healthy subjects were randomly exposed to virtual and real prisms, and measurements were taken before and after exposure. Our findings indicate that virtual prisms are at least as effective as real prisms in inducing aftereffects (4.39° ± 2.91° with the virtual prisms compared to 4.30° ± 3.49° with the real prisms), but that these effects were not sustained beyond 2 h regardless of exposure modality. The virtual measurements obtained with our software showed excellent metrological qualities (ICC = 0.95, error = 0.52° ± 1.18°), demonstrating its validity and reliability for quantifying deviation during pointing movements. Overall, our results suggest that our virtual reality software (Virtualis, Montpellier, France) could provide an easy and reliable means of quantifying and rehabilitating spatial neglect. Further validation of these results is required in individuals with unilateral spatial neglect.

Dynamic spectral cues do not affect human sound localization during small head movements

Natural listening involves a constant deployment of small head movement. Spatial listening is facilitated by head movements, especially when resolving front-back confusions, an otherwise common issue during sound localization under head-still conditions. The present study investigated which acoustic cues are utilized by human listeners to localize sounds using small head movements (below ±10° around the center). Seven normal-hearing subjects participated in a sound localization experiment in a virtual reality environment. Four acoustic cue stimulus conditions were presented (full spectrum, flattened spectrum, frozen spectrum, free-field) under three movement conditions (no movement, head rotations over the yaw axis and over the pitch axis). Localization performance was assessed using three metrics: lateral and polar precision error and front-back confusion rate. Analysis through mixed-effects models showed that even small yaw rotations provide a remarkable decrease in front-back confusion rate, whereas pitch rotations did not show much of an effect. Furthermore, MSS cues improved localization performance even in the presence of dITD cues. However, performance was similar between stimuli with and without dMSS cues. This indicates that human listeners utilize the MSS cues before the head moves, but do not rely on dMSS cues to localize sounds when utilizing small head movements.

Gymnastic skills on a balance beam with simulated height

Virtual reality (VR) is a valuable tool for simulating dangerous situations and training under these conditions with a reduced possibility of injury. This could be beneficial within different sports scenarios. In gymnastics, the height of the balance beam can be dangerous and frightening, especially for beginners. Here, a simulated height can reduce the participant's risk of injury and facilitate entry to balance beam gymnastics. However, the investigation of sports performance in a virtual environment is rare. Therefore, the current study aims to compare beginners' performance in balance beam tasks between the real world (RW) and VR. 34 sports students executed gymnastic balancing forward, backward and the extension of one leg to the front, the side and the back on a regular balance beam in the RW and on a balance beam with simulated height in VR (using a head-mounted display). We analyzed beginners' performance in both conditions (RW and VR) regarding different criteria for each balance beam task. Statistical analyses show significant differences in performance with better results in RW (p < .05). Especially the body tension and the upper body posture were superior in RW. One explanation can be the more difficult regulation of the balance in VR. For the tasks where the motion took part in a fixed position on the beam, the further performance aspects were similar for RW and VR. When the participants moved along the beam, further performance aspects, like the leg extension, were better in RW. In total, the participants executed the VR balance beam tasks with decreased danger of injury due to the simulated height of the balance beam, but their performance was slightly limited. We conclude that for the first contact with the height of a balance beam, VR is a suitable tool to reduce the danger of injury occurring from falling off the beam and facilitate entry to balance beam gymnastics.

Virtual Reality Applications Market Analysis—On the Example of Steam Digital Platform

This paper presents research on the topic of virtual reality (VR) applications. It conducts a quantitative analysis of virtual reality applications available in the international market using the example of a digital platform, which was the Steam platform. The study presents and analyzes data on the number of applications in the selected categories, such as genres, types of headsets, and language. The research also includes the analysis of the top-rated VR applications, their reviews, and their features, recognized based on the tags describing them. Additionally, the article provides and systematizes new knowledge about the VR applications environment. Based on the results, it was concluded that the most numerous group of VR applications was action applications, and they account for more than half of all VR apps (51.22%). Following this, there were casual games (40.78%) and then simulation VR apps (37.35%). Referring to the results of the top-rated VR applications (‘overwhelmingly positive’ status on Steam), there were only two apps with a result of 98% (the highest rated) positive feedback: Half-Life: Alyx, the action and adventure app, which is a shooter described as zombie horror, and Walkabout Mini Golf VR, a casual and minimalist sport application. When it comes to the analysis of the tags of the top-rated VR applications, the most repeated tags, despite the ‘VR’ tag, included ‘first-person’ and ‘singleplayer’ (occurred in the descriptions of 68% of the applications).

Immersive Virtual Reality during Robot-Assisted Gait Training: Validation of a New Device in Stroke Rehabilitation

Background and objective: Duration of rehabilitation and active participation are crucial for gait rehabilitation in the early stage after stroke onset. Virtual reality (VR) is an innovative tool providing engaging and playful environments that could promote intrinsic motivation and higher active participation for non-ambulatory stroke patients when combined with robot-assisted gait training (RAGT). We have developed a new, fully immersive VR application for RAGT, which can be used with a head-mounted display and wearable sensors providing real-time gait motion in the virtual environment. The aim of this study was to validate the use of this new device and assess the onset of cybersickness in healthy participants before testing the device in stroke patients. Materials and Methods: Thirty-seven healthy participants were included and performed two sessions of RAGT using a fully immersive VR device. They physically walked with the Gait Trainer for 20 min in a virtual forest environment. The occurrence of cybersickness, sense of presence, and usability of the device were assessed with three questionnaires: the Simulator Sickness Questionnaire (SSQ), the Presence Questionnaire (PQ), and the System Usability Scale (SUS). Results: All of the participants completed both sessions. Most of the participants (78.4%) had no significant adverse effects (SSQ < 5). The sense of presence in the virtual environment was particularly high (106.42 ± 9.46). Participants reported good usability of the device (86.08 ± 7.54). Conclusions: This study demonstrated the usability of our fully immersive VR device for gait rehabilitation and did not lead to cybersickness. Future studies should evaluate the same parameters and the effectiveness of this device with non-ambulatory stroke patients.

Evaluation of a Low-Cost Virtual Reality Surround-Screen Projection System

Two of the most popular mediums for virtual reality are head-mounted displays and surround-screen projection systems, such as CAVE Automatic Virtual Environments. In recent years, HMDs suffered a significant reduction in cost and have become widespread consumer products. In contrast, CAVEs are still expensive and remain accessible to a limited number of researchers. This study aims to evaluate both objective and subjective characteristics of a CAVE-like monoscopic low-cost virtual reality surround-screen projection system compared to advanced setups and HMDs. For objective results, we measured the head position estimation accuracy and precision of a low-cost active infrared (IR) based tracking system, used in the proposed low-cost CAVE, relatively to an infrared marker-based tracking system, used in a laboratory-grade CAVE system. For subjective characteristics, we investigated the sense of presence and cybersickness elicited in users during a visual search task outside personal space, beyond arms reach, where the importance of stereo vision is diminished. Thirty participants rated their sense of presence and cybersickness after performing the VR search task with our CAVE-like system and a modern HMD. The tracking showed an accuracy error of 1.66 cm and .4 mm of precision jitter. The system was reported to elicit presence but at a lower level than the HMD, while causing significant lower cybersickness. Our results were compared to a previous study performed with a laboratory-grade CAVE and support that a VR system implemented with low-cost devices could be a viable alternative to laboratory-grade CAVEs for visual search tasks outside the user's personal space.

Optimal integration of kinematic and ball-flight information when perceiving the speed of a moving ball

In order to intercept a moving target such as a baseball with high spatio-temporal accuracy, the perception of the target's movement speed is important for estimating when and where the target will arrive. However, it is unclear what sources of information are used by a batter to estimate ball speed and how those sources of information are integrated to facilitate successful interception. In this study, we examined the degree to which kinematic and ball-flight information are integrated when estimating ball speed in baseball batting. Thirteen university level baseball batters performed a ball-speed evaluation task in a virtual environment where they were required to determine which of two comparison baseball pitches (i.e., a reference and comparison stimuli) they perceived to be faster. The reference and comparison stimuli had the same physical ball speed, but with different pitching movement speeds in the comparison stimuli. The task was performed under slow (125 km/h) and fast (145 km/h) ball-speed conditions. Results revealed that the perceived ball-speed was influenced by the movement speed of the pitcher's motion, with the influence of the pitcher's motion more pronounced in the fast ball-speed condition when ball-flight information was presumably less reliable. Moreover, exploratory analyses suggested that the more skilled batters were increasingly likely to integrate the two sources of information according to their relative reliability when making judgements of ball speed. The results provide important insights into how skilled performers may make judgements of speed and time to contact, and further enhance our understanding of how the ability to make those judgements might improve when developing expertise in hitting.

Virtual reality gaming in rehabilitation after stroke - user experiences and perceptions

PURPOSE

The present study explored participants' experiences with and perceptions of using fully immersive head-mounted virtual reality (VR) gaming as rehabilitation after stroke.

METHODS

Four men and three women (median age 64 years) with chronic stroke and varying motor impairment (mild to severe) were interviewed after 10 weeks of VR training on the commercial HTC Vive system, focusing on the upper extremities. Inductive qualitative thematic analysis was performed.

RESULTS

The analysis revealed three main themes: playing the game, benefits and effects, and personalizing the game. Playing the game encompasses both the feeling of being immersed in the game and descriptions of the gaming being motivating and fun. Benefits and effects describe the participants' expectations of potential benefits, the importance of getting feed-back, and the impact in daily life. Personalizing the game includes finding the right game and level, and the participants' need for support to achieve full use of the training.

CONCLUSIONS

Participants with chronic stroke described the fully immersive VR gaming intervention as a fun and motivating way to improve their functioning in everyday life. Qualitative studies are needed to explore how people with stroke perceive VR gaming when it is implemented in real clinical environments.Clinical implicationsVR gaming was perceived as a positive and motivating rehabilitation after stroke.Getting feedback and perceiving benefits are essential parts of VR rehabilitation.Commercial fully immersive VR-games might be an option for stroke rehabilitation when the game can be personalized and support is available.

Effects of using immersive virtual reality on time and steps during a locomotor task in young adults

Immersive virtual reality makes possible to perceive and interact in a standardized, reproductible and digital environment, with a wide range of simulated situations possibilities. This study aimed to measure the potential effect of virtual reality on time and number of steps when performing a locomotor task, in a young adult’s population. Sixty young adults (32W, 28M, mean age 21.55 ± 1.32), who had their first immersive virtual reality experience, performed a locomotor task based on "Timed Up and Go" (TUG) task in real, in virtual reality in a stopped train and in virtual reality in a moving train. Time and number of steps variables representing primary locomotion indicators were measured and compared between each condition. Results showed significant increases in time and number of steps in the two virtual reality conditions compared to real but not between the two virtual reality conditions. There was an effect of virtual reality in young adults when performing the locomotor task. It means that technological and digital characteristics of the immersive virtual reality experience led to modify motor strategies employed. Adding a plausible visual optic flow did not appear to affect motor control further when the information is negligible and not essential for performing the task.

A New Sensitive Test Using Virtual Reality and Foam to Probe Postural Control in Vestibular Patients: The Unilateral Schwannoma Model

Vestibular schwannomas (VS) are benign tumors of the vestibular nerve that may trigger hearing loss, tinnitus, rotatory vertigo, and dizziness in patients. Vestibular and auditory tests can determine the precise degree of impairment of the auditory nerve, and superior and inferior vestibular nerves. However, balance is often poorly quantified in patients with untreated vestibular schwannoma, for whom validated standardized assessments of balance are often lacking. Balance can be quantified with the EquiTest. However, this device was developed a long time ago and is expensive, specific, and not sensitive enough to detect early deficits because it assesses balance principally in the sagittal plane on a firm platform. In this study, we assessed postural performances in a well-defined group of VS patients. We used the Dizziness Handicap Inventory (DHI) and a customized device consisting of a smartphone, a mask delivering a fixed or moving visual scene, and foam rubber. Patients were tested in four successive sessions of 25 s each: eyes open (EO), eyes closed (EC), fixed visual scene (VR0), and visual moving scenes (VR1) delivered by the HTC VIVE mask. Postural oscillations were quantified with sensors from an android smartphone (Galaxy S9) fixed to the back. The results obtained were compared to those obtained with the EquiTest. Vestibulo-ocular deficits were also quantified with the caloric test and vHIT. The function of the utricle and saccule were assessed with ocular and cervical vestibular-evoked myogenic potentials (o-VEMPs and c-VEMPs), respectively. We found that falls and abnormal postural oscillations were frequently detected in the VS patients with the VR/Foam device. We detected no correlation between falls or abnormal postural movements and horizontal canal deficit or age. In conclusion, this new method provides a simpler, quicker, and cheaper method for quantifying balance. It will be very helpful for (1) determining balance deficits in VS patients; (2) optimizing the optimal therapy indications (active follow-up, surgery, or gamma therapy) and follow-up of VS patients before and after treatment; (3) developing new rehabilitation methods based on balance training in extreme conditions with disturbed visual and proprioceptive inputs.

Data-Driven Classification of Human Movements in Virtual Reality-Based Serious Games: Preclinical Rehabilitation Study in Citizen Science

BACKGROUND

Sustained engagement is essential for the success of telerehabilitation programs. However, patients' lack of motivation and adherence could undermine these goals. To overcome this challenge, physical exercises have often been gamified. Building on the advantages of serious games, we propose a citizen science-based approach in which patients perform scientific tasks by using interactive interfaces and help advance scientific causes of their choice. This approach capitalizes on human intellect and benevolence while promoting learning. To further enhance engagement, we propose performing citizen science activities in immersive media, such as virtual reality (VR).

OBJECTIVE

This study aims to present a novel methodology to facilitate the remote identification and classification of human movements for the automatic assessment of motor performance in telerehabilitation. The data-driven approach is presented in the context of a citizen science software dedicated to bimanual training in VR. Specifically, users interact with the interface and make contributions to an environmental citizen science project while moving both arms in concert.

METHODS

In all, 9 healthy individuals interacted with the citizen science software by using a commercial VR gaming device. The software included a calibration phase to evaluate the users' range of motion along the 3 anatomical planes of motion and to adapt the sensitivity of the software's response to their movements. During calibration, the time series of the users' movements were recorded by the sensors embedded in the device. We performed principal component analysis to identify salient features of movements and then applied a bagged trees ensemble classifier to classify the movements.

RESULTS

The classification achieved high performance, reaching 99.9% accuracy. Among the movements, elbow flexion was the most accurately classified movement (99.2%), and horizontal shoulder abduction to the right side of the body was the most misclassified movement (98.8%).

CONCLUSIONS

Coordinated bimanual movements in VR can be classified with high accuracy. Our findings lay the foundation for the development of motion analysis algorithms in VR-mediated telerehabilitation.

Integration of Virtual Reality into Transcranial Magnetic Stimulation Improves Cognitive Function in Patients with Parkinson's Disease with Cognitive Impairment: A Proof-of-Concept Study

BACKGROUND

Emerging evidence has indicated the positive effects of repetitive transcranial magnetic stimulation (rTMS) on patients with Parkinson's disease (PD) for the treatment of mild cognitive impairment (MCI).

OBJECTIVE

Investigating whether combining virtual reality (VR) training with rTMS can further enhance cognitive improvement induced by rTMS treatment.

METHODS

We randomly assigned 40 patients with PD and MCI into three groups, namely the rTMS-VR group (n = 13), rTMS group (n = 11), and sham rTMS group (n = 16). rTMS was administered as 10 consecutive sessions of intermittent theta burst stimulation (iTBS) over the left dorsolateral prefrontal cortex. In the rTMS-VR group, VR training was administered immediately after each rTMS session. Cognitive function was measured using the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) and Montreal Cognitive Assessment (MoCA) at baseline, immediately after intervention, and at 3-month follow-up.

RESULTS

Compared with the rTMS group, the rTMS-VR group exhibited significantly more improvements in total and delayed memory scores of the RBANS and the visuospatial/executive function score of the MoCA after intervention (p = 0.000∼0.046) and the delayed memory score of the RBANS at 3-month follow-up (p = 0.028).

CONCLUSION

The integrated rTMS-VR protocol achieved a superior outcome in global cognitive function, more effectively enhancing working memory and visuospatial executive function than did the rTMS protocol alone. The combination of VR and rTMS can be an effective regimen for improving the cognitive function of patients with PD.

Proof of Concept for the Use of Immersive Virtual Reality in Upper Limb Rehabilitation of Multiple Sclerosis Patients

Multiple sclerosis (MS) is a debilitating disease which gradually reduces motor function and mobility. Virtual reality (VR) has been successfully utilised in support of existing therapeutic approaches for many different conditions, and new innovative and experimental features could be the future of VR rehabilitation. The Quest is a new headset by Oculus, with its built-in tracking, relatively low cost, portability and lack of reliance on expensive processing heavy PCs to power it, and could be an ideal system to facilitate at-home or clinic-based upper limb rehabilitation. A hand-tracking-based rehabilitation game aimed at people with MS was developed for Oculus Quest using Unity. Two distinct games were made to replicate different types of hand exercises, piano playing for isolated finger flexion and maze tracking for coordination and arm flexion. This pilot study assesses the value of such approach along with evaluating intrinsic and extrinsic methods of providing feedback, namely, positive scoring, negative scoring and audio response. One physiotherapist and two individuals with MS were surveyed. Participant response was positive although small sample size impacts the user testing validity of the results. Future research is recommended to build off the data gathered as a pilot study and increase sample size to collect richer feedback.

Attention Detection in Virtual Environments Using EEG Signals: A Scoping Review

The competitive demand for attention is present in our daily lives, and the identification of neural processes in the EEG signals associated with the demand for specific attention can be useful to the individual’s interactions in virtual environments. Since EEG-based devices can be portable, non-invasive, and present high temporal resolution technology for recording neural signal, the interpretations of virtual systems user’s attention, fatigue and cognitive load based on parameters extracted from the EEG signal are relevant for several purposes, such as games, rehabilitation, and therapies. However, despite the large amount of studies on this subject, different methodological forms are highlighted and suggested in this work, relating virtual environments, demand of attention, workload and fatigue applications. In our summarization, we discuss controversies, current research gaps and future directions together with the background and final sections.

An ecologically valid examination of event-based and time-based prospective memory using immersive virtual reality: The influence of attention, memory, and executive function processes on real-world prospective memory

Studies on prospective memory (PM) predominantly assess either event- or time-based PM by implementing non-ecological laboratory-based tasks. The results deriving from these paradigms have provided findings that are discrepant with ecologically valid research paradigms that converge on the complexity and cognitive demands of everyday tasks. The Virtual Reality Everyday Assessment Lab (VR-EAL), an immersive virtual reality (VR) neuropsychological battery with enhanced ecological validity, was implemented to assess everyday event- and time-based PM, as well as the influence of other cognitive functions on everyday PM functioning. The results demonstrated the role of delayed recognition, planning, and visuospatial attention on everyday PM. Delayed recognition and planning ability were found to be central in event- and time-based PM respectively. In order of importance, delayed recognition, visuospatial attention speed, and planning ability were found to be involved in event-based PM functioning. Comparably, planning, visuospatial attention accuracy, delayed recognition, and multitasking/task-shifting ability were found to be involved in time-based PM functioning. These findings further suggest the importance of ecological validity in the study of PM, which may be achieved using immersive VR paradigms.

Accuracy and precision of visual and auditory stimulus presentation in virtual reality in Python 2 and 3 environments for human behavior research

Virtual reality (VR) is a new methodology for behavioral studies. In such studies, the millisecond accuracy and precision of stimulus presentation are critical for data replicability. Recently, Python, which is a widely used programming language for scientific research, has contributed to reliable accuracy and precision in experimental control. However, little is known about whether modern VR environments have millisecond accuracy and precision for stimulus presentation, since most standard methods in laboratory studies are not optimized for VR environments. The purpose of this study was to systematically evaluate the accuracy and precision of visual and auditory stimuli generated in modern VR head-mounted displays (HMDs) from HTC and Oculus using Python 2 and 3. We used the newest Python tools for VR and Black Box Toolkit to measure the actual time lag and jitter. The results showed that there was an 18-ms time lag for visual stimulus in both HMDs. For the auditory stimulus, the time lag varied between 40 and 60 ms, depending on the HMD. The jitters of those time lags were 1 ms for visual stimulus and 4 ms for auditory stimulus, which are sufficiently low for general experiments. These time lags were robustly equal, even when auditory and visual stimuli were presented simultaneously. Interestingly, all results were perfectly consistent in both Python 2 and 3 environments. Thus, the present study will help establish a more reliable stimulus control for psychological and neuroscientific research controlled by Python environments.

Effects of prolonged use of virtual reality smartphone-based head-mounted display on visual parameters: a randomised controlled trial

We investigated the effects of using a virtual reality smartphone-based head-mounted display (VR SHMD) device for 2 h on visual parameters. Fifty-eight healthy volunteers were recruited. The participants played games using VR SHMD or smartphones for 2 h on different days. Visual parameters including refraction, accommodation, convergence, stereopsis, and ocular alignment and measured choroidal thickness before and after the use of VR SHMD or smartphones were investigated. Subjective symptoms were assessed using questionnaires. We analyzed the differences in visual parameters before and after the use of VR SHMD or smartphones and correlations between baseline visual parameters and those after the use of the devices. Significant changes were observed in near-point convergence and accommodation, exophoric deviation, stereopsis, and accommodative lag after the use of VR SHMD but not after that of smartphones. The subjective discomfort associated with dry eye and neurologic symptoms were more severe in the VR group than in the smartphone group. There were no significant changes in refraction and choroidal thickness after the use of either of the two devices. The poorer the participants' accommodation and convergence ability the greater the resistance to changes in these visual parameters, and participants with a large exophoria were more prone to worsening of exophoria than those with a small exophoria.

Autonomous Fingerprinting and Large Experimental Data Set for Visible Light Positioning

This paper presents an autonomous method of collecting data for Visible Light Positioning (VLP) and a comprehensive investigation of VLP using a large set of experimental data. Received Signal Strength (RSS) data are efficiently collected using a novel method that utilizes consumer grade Virtual Reality (VR) tracking for accurate ground truth recording. An investigation into the accuracy of the ground truth system showed median and 90th percentile errors of 4.24 and 7.35 mm, respectively. Co-locating a VR tracker with a photodiode-equipped VLP receiver on a mobile robotic platform allows fingerprinting on a scale and accuracy that has not been possible with traditional manual collection methods. RSS data at 7344 locations within a 6.3 × 6.9 m test space fitted with 11 VLP luminaires is collected and has been made available for researchers. The quality and the volume of the data allow for a robust study of Machine Learning (ML)- and channel model-based positioning utilizing visible light. Among the ML-based techniques, ridge regression is found to be the most accurate, outperforming Weighted k Nearest Neighbor, Multilayer Perceptron, and random forest, among others. Model-based positioning is more accurate than ML techniques when a small data set is available for calibration and training. However, if a large data set is available for training, ML-based positioning outperforms its model-based counterparts in terms of localization accuracy.

Personalized 3D exergames for in-home rehabilitation after stroke: a pilot study

PURPOSE

To describe a novel 3-dimensional (3D) exergames system and the results of a clinical feasibility study of stroke survivors needing in-home rehabilitation.

MATERIALS AND METHODS

The customisable Personalized In-home eXErgames for Rehabilitation (PIXER) system captures the user's image, generates a live model, and incorporates it into a virtual exergame. PIXER provides a recording system for home exercise programs (HEPs) by adapting virtual objects, customizes the exergame and creates a digital diary. Ten persons with stroke, performed HEPs with PIXER for 1 month, and without PIXER for 2 additional months. In-game performance data, measures of physical functioning (PF) including Stroke Impact Scale (SIS), Timed Up & Go (TUG) and Goal Attainment (GA) Scale obtained at baseline, 1- and 3 months were evaluated.

RESULTS

Seventy percent of participants completed the 1-month timepoint, 50% completed all timepoints. In-game data: Number of repetitions completed; Anomalies reported; and Percentage of bubbles hit showed positive trends. Compared to baseline, all SIS physical functioning (PF) scores were higher at 1 month, TUG scores showed no overall improvement and GA scale scores were 77% at 3 months.

CONCLUSION

It is feasible for community-dwelling patients to perform HEP after stroke using PIXER, a novel, exergames system, and potentially improve their function.IMPLICATIONS FOR REHABILITATIONHome Exercises performed using a novel, 3-dimensional, customizable Personalized In-home eXErgames for Rehabilitation (PIXER) system is feasible for community-dwelling patients after stroke.In-game performance data obtained in this clinical pilot study showed positive trends of improvement in several study participants.PIXER has potential to improve functional outcomes for community-dwelling adults with stroke.

Feedback from HTC Vive Sensors Results in Transient Performance Enhancements on a Juggling Task in Virtual Reality

Virtual reality headsets, such as the HTC Vive, can be used to model objects, forces, and interactions between objects with high perceived realism and accuracy. Moreover, they can accurately track movements of the head and the hands. This combination makes it possible to provide subjects with precise quantitative feedback on their performance while they are learning a motor task. Juggling is a challenging motor task that requires precise coordination of both hands. Professional jugglers throw objects so that the arc peaks just above head height, and they time their throws so that the second ball is thrown when the first ball reaches its peak. Here, we examined whether it is possible to learn to juggle in virtual reality and whether the height and the timing of the throws can be improved by providing immediate feedback derived from the motion sensors. Almost all participants became better at juggling in the ~30 min session: the height and timing of their throws improved and they dropped fewer balls. Feedback on height, but not timing, improved performance, albeit only temporarily.

An ecologically valid examination of event-based and time-based prospective memory using immersive virtual reality: the effects of delay and task type on everyday prospective memory

Recent research has focused on assessing either event- or time-based prospective memory (PM) using laboratory tasks. Yet, the findings pertaining to PM performance on laboratory tasks are often inconsistent with the findings on corresponding naturalistic experiments. Ecologically valid neuropsychological tasks resemble the complexity and cognitive demands of everyday tasks, offer an adequate level of experimental control, and allow a generalisation of the findings to everyday performance. The Virtual Reality Everyday Assessment Lab (VR-EAL), an immersive virtual reality neuropsychological battery with enhanced ecological validity, was implemented to comprehensively assess everyday PM (i.e., focal and non-focal event-based, and time-based). The effects of the length of delay between encoding and initiating the PM intention and the type of PM task on everyday PM performance were examined. The results revealed that everyday PM performance was affected by the length of delay rather than the type of PM task. The effect of the length of delay differentially affected performance on the focal, non-focal, and time-based tasks and was proportional to the PM cue focality (i.e., semantic relationship with the intended action). This study also highlighted methodological considerations such as the differentiation between functioning and ability, distinction of cue attributes, and the necessity of ecological validity.

Immersive virtual reality health games: a narrative review of game design

BACKGROUND

High quality head-mounted display based virtual reality (HMD-VR) has become widely available, spurring greater development of HMD-VR health games. As a behavior change approach, these applications use HMD-VR and game-based formats to support long-term engagement with therapeutic interventions. While the bulk of research to date has primarily focused on the therapeutic efficacy of particular HMD-VR health games, how developers and researchers incorporate best-practices in game design to achieve engaging experiences remains underexplored. This paper presents the findings of a narrative review exploring the trends and future directions of game design for HMD-VR health games.

METHODS

We searched the literature on the intersection between HMD-VR, games, and health in databases including MEDLINE, Embase, CINAHL, PsycINFO, and Compendex. We identified articles describing HMD-VR games designed specifically as health applications from 2015 onwards in English. HMD-VR health games were charted and tabulated according to technology, health context, outcomes, and user engagement in game design.

FINDINGS

We identified 29 HMD-VR health games from 2015 to 2020, with the majority addressing health contexts related to physical exercise, motor rehabilitation, and pain. These games typically involved obstacle-based challenges and extrinsic reward systems to engage clients in interventions related to physical functioning and pain. Less common were games emphasizing narrative experiences and non-physical exercise interventions. However, discourse regarding game design was diverse and often lacked sufficient detail. Game experience was evaluated using primarily ad-hoc questionnaires. User engagement in the development of HMD-VR health games primarily manifested as user studies.

CONCLUSION

HMD-VR health games are promising tools for engaging clients in highly immersive experiences designed to address diverse health contexts. However, more in-depth and structured attention to how HMD-VR health games are designed as game experiences is needed. Future development of HMD-VR health games may also benefit from greater involvement of end-users in participatory approaches.

Validation of the Virtual Reality Everyday Assessment Lab (VR-EAL): An Immersive Virtual Reality Neuropsychological Battery with Enhanced Ecological Validity

OBJECTIVE

The assessment of cognitive functions such as prospective memory, episodic memory, attention, and executive functions benefits from an ecologically valid approach to better understand how performance outcomes generalize to everyday life. Immersive virtual reality (VR) is considered capable of simulating real-life situations to enhance ecological validity. The present study attempted to validate the Virtual Reality Everyday Assessment Lab (VR-EAL), an immersive VR neuropsychological battery, against an extensive paper-and-pencil neuropsychological battery.

METHODS

Forty-one participants (21 females) were recruited: 18 gamers and 23 non-gamers who attended both an immersive VR and a paper-and-pencil testing session. Bayesian Pearson's correlation analyses were conducted to assess construct and convergent validity of the VR-EAL. Bayesian t-tests were performed to compare VR and paper-and-pencil testing in terms of administration time, similarity to real-life tasks (i.e., ecological validity), and pleasantness.

RESULTS

VR-EAL scores were significantly correlated with their equivalent scores on the paper-and-pencil tests. The participants' reports indicated that the VR-EAL tasks were significantly more ecologically valid and pleasant than the paper-and-pencil neuropsychological battery. The VR-EAL battery also had a shorter administration time.

CONCLUSION

The VR-EAL appears as an effective neuropsychological tool for the assessment of everyday cognitive functions, which has enhanced ecological validity, a highly pleasant testing experience, and does not induce cybersickness.

Embodied virtual reality for the study of real-world motor learning

Motor-learning literature focuses on simple laboratory-tasks due to their controlled manner and the ease to apply manipulations to induce learning and adaptation. Recently, we introduced a billiards paradigm and demonstrated the feasibility of real-world-neuroscience using wearables for naturalistic full-body motion-tracking and mobile-brain-imaging. Here we developed an embodied virtual-reality (VR) environment to our real-world billiards paradigm, which allows to control the visual feedback for this complex real-world task, while maintaining sense of embodiment. The setup was validated by comparing real-world ball trajectories with the trajectories of the virtual balls, calculated by the physics engine. We then ran our short-term motor learning protocol in the embodied VR. Subjects played billiard shots when they held the physical cue and hit a physical ball on the table while seeing it all in VR. We found comparable short-term motor learning trends in the embodied VR to those we previously reported in the physical real-world task. Embodied VR can be used for learning real-world tasks in a highly controlled environment which enables applying visual manipulations, common in laboratory-tasks and rehabilitation, to a real-world full-body task. Embodied VR enables to manipulate feedback and apply perturbations to isolate and assess interactions between specific motor-learning components, thus enabling addressing the current questions of motor-learning in real-world tasks. Such a setup can potentially be used for rehabilitation, where VR is gaining popularity but the transfer to the real-world is currently limited, presumably, due to the lack of embodiment.

Commercial head-mounted display virtual reality for upper extremity rehabilitation in chronic stroke: a single-case design study

Background

Rehabilitation is crucial for maximizing recovery after stroke. Rehabilitation activities that are fun and rewarding by themselves can be more effective than those who are not. Gamification with virtual reality (VR) exploits this principle. This single-case design study probes the potential for using commercial off-the-shelf, room-scale head-mounted virtual reality for upper extremity rehabilitation in individuals with chronic stroke, the insights of which can inform further research.

Methods

A heterogeneous volunteer sample of seven participants living with stroke were recruited through advertisement. A single-case design was employed with a 5-week baseline (A), followed by a 10-week intervention (B) and a 6-month follow-up. Upper extremity motor function was assessed with validated kinematic analysis of drinking task. Activity capacity was assessed with Action Research Arm Test, Box and Block Test and ABILHAND questionnaire. Assessments were done weekly and at follow-up. Playing games on a VR-system with head-mounted display (HTC Vive) was used as rehabilitation intervention. Approximately 300 games were screened and 6 tested. Visual analysis and Tau-U statistics were used to interpret the results.

Results

Visual analysis of trend, level shift and overlap as well as Tau-U statistics indicated improvement of Action Research Arm Test in six participants. Four of these had at least a moderate Tau-U score (0.50–0.92), in at least half of the assessed outcomes. These four participants trained a total of 361 to 935 min. Two out of four participants who were able to perform the drinking task, had the highest training dose (> 900 min) and showed also improvements in kinematics. The predominant game played was Beat Saber. No serious adverse effects related to the study were observed, one participant interrupted the intervention phase due to a fall at home.

Conclusions

This first study of combining commercial games, a commercial head-mounted VR, and commercial haptic hand controls, showed promising results for upper extremity rehabilitation in individuals with chronic stroke. By being affordable yet having high production values, as well as being an easily accessible off-the-shelf product, this variant of VR technology might facilitate widespread adaption. Insights garnered in this study can facilitate the execution of future studies.

Trial registration The study was registered at researchweb.org (project number 262331, registered 2019-01-30, https://www.researchweb.org/is/vgr/project/262331) prior to participant enrolment.

Emotion Recognition in Immersive Virtual Reality: From Statistics to Affective Computing

Emotions play a critical role in our daily lives, so the understanding and recognition of emotional responses is crucial for human research. Affective computing research has mostly used non-immersive two-dimensional (2D) images or videos to elicit emotional states. However, immersive virtual reality, which allows researchers to simulate environments in controlled laboratory conditions with high levels of sense of presence and interactivity, is becoming more popular in emotion research. Moreover, its synergy with implicit measurements and machine-learning techniques has the potential to impact transversely in many research areas, opening new opportunities for the scientific community. This paper presents a systematic review of the emotion recognition research undertaken with physiological and behavioural measures using head-mounted displays as elicitation devices. The results highlight the evolution of the field, give a clear perspective using aggregated analysis, reveal the current open issues and provide guidelines for future research.

Development and Calibration of an Eye-Tracking Fixation Identification Algorithm for Immersive Virtual Reality

Fixation identification is an essential task in the extraction of relevant information from gaze patterns; various algorithms are used in the identification process. However, the thresholds used in the algorithms greatly affect their sensitivity. Moreover, the application of these algorithm to eye-tracking technologies integrated into head-mounted displays, where the subject's head position is unrestricted, is still an open issue. Therefore, the adaptation of eye-tracking algorithms and their thresholds to immersive virtual reality frameworks needs to be validated. This study presents the development of a dispersion-threshold identification algorithm applied to data obtained from an eye-tracking system integrated into a head-mounted display. Rules-based criteria are proposed to calibrate the thresholds of the algorithm through different features, such as number of fixations and the percentage of points which belong to a fixation. The results show that distance-dispersion thresholds between 1-1.6° and time windows between 0.25-0.4 s are the acceptable range parameters, with 1° and 0.25 s being the optimum. The work presents a calibrated algorithm to be applied in future experiments with eye-tracking integrated into head-mounted displays and guidelines for calibrating fixation identification algorithms.

Predictive eye movements when hitting a bouncing ball

Predictive eye movements targeted toward the direction of ball bounce are a feature of gaze behavior when intercepting a target soon after it has bounced. However, there is conjecture over the exact location toward which these predictive eye movements are directed, and whether gaze during this period is moving or instead "lies in wait" for the ball to arrive. Therefore, the aim of this study was to further examine the location toward which predictive eye movements are made when hitting a bouncing ball. We tracked the eye and head movements of 23 novice participants who attempted to hit approaching tennis balls in a virtual environment. The balls differed in time from bounce to contact (300, 550, and 800 ms). Results revealed that participants made predictive saccades shortly before the ball bounced in two-thirds of all trials. These saccades were directed several degrees above the position at which the ball bounced, rather than toward the position at which it bounced or toward a position the ball would occupy shortly after the bounce. After the saccade, a separation of roles for the eyes and head ensured that gaze continued to change so that it was as close as possible to the ball soon after bounce. Smooth head movements were responsible for the immediate and ongoing changes in gaze to align it with the ball in the lateral direction, while eye movements realigned gaze with the ball in the vertical direction from approximately 100 ms after the ball changed its direction of motion after bounce. We conclude that predictive saccades direct gaze above the location at which the ball will bounce, presumably in order to facilitate ball tracking after the bounce.

Guidelines for immersive virtual reality in psychological research

Virtual reality (VR) holds immense promise as a research tool to deliver results that are generalizable to the real world. However, the methodology used in different VR studies varies substantially. While many of these approaches claim to use 'immersive VR', the different hardware and software choices lead to issues regarding reliability and validity of psychological VR research. Questions arise about quantifying presence, the optimal level of graphical realism, the problem of being in dual realities and reproducibility of VR research. We discuss how VR research paradigms could be evaluated and offer a list of practical recommendations to have common guidelines for psychological VR research.

Feasibility and Tolerability of a Culture-Based Virtual Reality (VR) Training Program in Patients with Mild Cognitive Impairment: A Randomized Controlled Pilot Study

The present study examined whether a culture-based virtual reality (VR) training program is feasible and tolerable for patients with amnestic mild cognitive impairment (aMCI), and whether it could improve cognitive function in these patients. Twenty-one outpatients with aMCI were randomized to either the VR-based training group or the control group in a 1:1 ratio. The VR-based training group participated in training for 30 min/day, two days/week, for three months (24 times). The VR-based program was designed based on Korean traditional culture and used attention, processing speed, executive function and memory conditions to stimulate cognitive function. The adherence to the culture-based VR training program was 91.55% ± 6.41% in the VR group. The only adverse events observed in the VR group were dizziness (4.2%) and fatigue (8.3%). Analysis revealed that the VR-based training group exhibited no significant differences following the three-month VR program in Korean Mini-Mental State Examination (K-MMSE) scores, working memory functions such as performance on the digit span test, or in Stroop test performance and word fluency. We conclude that although the 12-week culture-based VR training program did not improve cognitive function, our findings revealed that the culture-based VR training program was feasible and tolerable for participants with aMCI.

Embodiment and Presence in Virtual Reality After Stroke. A Comparative Study With Healthy Subjects

The ability of virtual reality (VR) to recreate controlled, immersive, and interactive environments that provide intensive and customized exercises has motivated its therapeutic use after stroke. Interaction and bodily presence in VR-based interventions is usually mediated through virtual selves, which synchronously represent body movements or responses to events on external input devices. Embodied self-representations in the virtual world not only provide an anchor for visuomotor tasks, but their morphologies can have behavioral implications. While research has focused on the underlying subjective mechanisms of exposure to VR on healthy individuals, the transference of these findings to individuals with stroke is not evident and remains unexplored, which could affect the experience and, ultimately, the clinical effectiveness of neurorehabilitation interventions. This study determined and compared the sense of embodiment and presence elicited by a virtual environment under different perspectives and levels of immersion in healthy subjects and individuals with stroke. Forty-six healthy subjects and 32 individuals with stroke embodied a gender-matched neutral avatar in a virtual environment that was displayed in a first-person perspective with a head-mounted display and in a third-person perspective with a screen, and the participants were asked to interact in a virtual task for 10 min under each condition in counterbalanced order, and to complete two questionnaires about the sense of embodiment and presence experienced during the interaction. The sense of body-ownership, self-location, and presence were more vividly experienced in a first-person than in a third-person perspective by both healthy subjects (p < 0.001, ηp2 = 0.212; p = 0.005, ηp2 = 0.101; p = 0.001, ηp2 = 0.401, respectively) and individuals with stroke (p = 0.019, ηp2 = 0.070; p = 0.001, ηp2 = 0.135; p = 0.014, ηp2 = 0.077, respectively). In contrast, no agency perspective-related differences were found in any group. All measures were consistently higher for healthy controls than for individuals with stroke, but differences between groups only reached statistical significance in presence under the first-person condition (p < 0.010, ηp2 = 0.084). In spite of these differences, the participants experienced a vivid sense of embodiment and presence in almost all conditions. These results provide first evidence that, although less intensively, embodiment and presence are similarly experienced by individuals who have suffered a stroke and by healthy individuals, which could support the vividness of their experience and, consequently, the effectiveness of VR-based interventions.

Technological Competence Is a Pre-condition for Effective Implementation of Virtual Reality Head Mounted Displays in Human Neuroscience: A Technological Review and Meta-Analysis

Immersive virtual reality (VR) emerges as a promising research and clinical tool. However, several studies suggest that VR induced adverse symptoms and effects (VRISE) may undermine the health and safety standards, and the reliability of the scientific results. In the current literature review, the technical reasons for the adverse symptomatology are investigated to provide suggestions and technological knowledge for the implementation of VR head-mounted display (HMD) systems in cognitive neuroscience. The technological systematic literature indicated features pertinent to display, sound, motion tracking, navigation, ergonomic interactions, user experience, and computer hardware that should be considered by the researchers. Subsequently, a meta-analysis of 44 neuroscientific or neuropsychological studies involving VR HMD systems was performed. The meta-analysis of the VR studies demonstrated that new generation HMDs induced significantly less VRISE and marginally fewer dropouts. Importantly, the commercial versions of the new generation HMDs with ergonomic interactions had zero incidents of adverse symptomatology and dropouts. HMDs equivalent to or greater than the commercial versions of contemporary HMDs accompanied with ergonomic interactions are suitable for implementation in cognitive neuroscience. In conclusion, researchers' technological competency, along with meticulous methods and reports pertinent to software, hardware, and VRISE, are paramount to ensure the health and safety standards and the reliability of neuroscientific results.

A Novel Real-Time Match-Moving Method with HoloLens

With the advancement of media and computing technologies, video compositing techniques have improved to a great extent. These techniques have been used not only in the entertainment industry but also in advertisement and new media. Match-moving is a cinematic technology in virtual-real image synthesis that allows the insertion of computer graphics (virtual objects) into real world scenes. To make a realistic virtual-real image synthesis, it is important to obtain internal parameters (such as focal length) and external parameters (position and rotation) from an Red-Green-Blue(RGB) camera. Conventional methods recover these parameters by extracting feature points from recorded video frames to guide the virtual camera. These methods fail when there is occlusion or motion blur in the recorded scene. In this paper, we propose a novel method (system) for pre-visualization and virtual-real image synthesis that overcomes the limitations of conventional methods. This system uses the spatial understanding principle of Microsoft HoloLens to perform the match-moving of virtual-real video scenes. Experimental results demonstrate that our system is much more accurate and efficient than existing systems for video compositing.

Sound source localization with varying amount of visual information in virtual reality

To achieve accurate spatial auditory perception, subjects typically require personal head-related transfer functions (HRTFs) and the freedom for head movements. Loudspeaker-based virtual sound environments allow for realism without individualized measurements. To study audio-visual perception in realistic environments, the combination of spatially tracked head mounted displays (HMDs), also known as virtual reality glasses, and virtual sound environments may be valuable. However, HMDs were recently shown to affect the subjects’ HRTFs and thus might influence sound localization performance. Furthermore, due to limitations of the reproduction of visual information on the HMD, audio-visual perception might be influenced. Here, a sound localization experiment was conducted both with and without an HMD and with a varying amount of visual information provided to the subjects. Furthermore, interaural time and level difference errors (ITDs and ILDs) as well as spectral perturbations induced by the HMD were analyzed and compared to the perceptual localization data. The results showed a reduction of the localization accuracy when the subjects were wearing an HMD and when they were blindfolded. The HMD-induced error in azimuth localization was found to be larger in the left than in the right hemisphere. When visual information of the limited set of source locations was provided, the localization error induced by the HMD was found to be negligible. Presenting visual information of hand-location and room dimensions showed better sound localization performance compared to the condition with no visual information. The addition of possible source locations further improved the localization accuracy. Also adding pointing feedback in form of a virtual laser pointer improved the accuracy of elevation perception but not of azimuth perception.

Virtual Reality and Simulation for Progressive Treatments in Urology

In urology technologies and surgical practices are constantly evolving and virtual reality (VR) simulation has become a significant supplement to existing urology methods in the training curricula of urologists. However, new developments in urology also require training and simulation for a wider application. In order to achieve this VR and simulation could play a central role. The purpose of this article is a review of the principal applications for VR and simulation in the field of urology education and to demonstrate the potential for the propagation of new progressive treatments. Two different cases are presented as examples: exposure therapy for paruresis and virtual cystoscopy for diagnosis and surgery of bladder cancer. The article uses research and publications listed in openly accessible directories and is organized into 3 sections: The first section covers features of VR and simulation technologies. The second one presents confirmed applications of current technologies in urology education and showcases example future applications in the domain of bladder treatment and surgery. The final section discusses the potential of the technology to improve health care quality.

Virtual Functional Endoscopic Sinus Surgery Simulation with 3D-Printed Models for Mixed-Reality Nasal Endoscopy

The surgeon's knowledge of a patient's individual anatomy is critical in skull base surgery. Trainees and experienced surgeons can benefit from surgical simulation; however, current models are expensive and impractical for widespread use. In this study, we report a next-generation mixed-reality surgical simulator. We segmented critical anatomic structures for 3-dimensional (3D) models to develop a modular teaching tool. We then developed a navigation tracking system utilizing a 3D-printed endoscope as a trackable virtual-reality (VR) controller and validated the accuracy on VR and 3D-printed skull models within 1 cm. We combined VR and augmented-reality visual cues with our 3D physical model to simulate sinus endoscopy and highlight segmented structures in real time. This report provides evidence that a mixed-reality simulator combining VR and 3D-printed models is feasible and may prove useful as an educational tool that is low cost and customizable.