The Statistics of Eye Movements and Binocular Disparities during VR Gaming: Implications for Headset Design

Eye posture and screen alignment with simulated see-through head-mounted displays

When rendering the visual scene for near-eye head-mounted displays, accurate knowledge of the geometry of the displays, scene objects, and eyes is required for the correct generation of the binocular images. Despite possible design and calibration efforts, these quantities are subject to positional and measurement errors, resulting in some misalignment of the images projected to each eye. Previous research investigated the effects in virtual reality (VR) setups that triggered such symptoms as eye strain and nausea. This work aimed at investigating the effects of binocular vertical misalignment (BVM) in see-through augmented reality (AR). In such devices, two conflicting environments coexist. One environment corresponds to the real world, which lies in the background and forms geometrically aligned images on the retinas. The other environment corresponds to the augmented content, which stands out as foreground and might be subject to misalignment. We simulated a see-through AR environment using a standard three-dimensional (3D) stereoscopic display to have full control and high accuracy of the real and augmented contents. Participants were involved in a visual search task that forced them to alternatively interact with the real and the augmented contents while being exposed to different amounts of BVM. The measured eye posture indicated that the compensation for vertical misalignment is equally shared by the sensory (binocular fusion) and the motor (vertical vergence) components of binocular vision. The sensitivity of each participant varied, both in terms of perceived discomfort and misalignment tolerance, suggesting that a per-user calibration might be useful for a comfortable visual experience.

Comparative Analysis of Physiological Vergence Angle Calculations from Objective Measurements of Gaze Position

Eccentric photorefractometry is widely used to measure eye refraction, accommodation, gaze position, and pupil size. While the individual calibration of refraction and accommodation data has been extensively studied, gaze measurements have received less attention. PowerRef 3 does not incorporate individual calibration for gaze measurements, resulting in a divergent offset between the measured and expected gaze positions. To address this, we proposed two methods to calculate the physiological vergence angle based on the visual vergence data obtained from PowerRef 3. Twenty-three participants aged 25 ± 4 years viewed Maltese cross stimuli at distances of 25, 30, 50, 70, and 600 cm. The expected vergence angles were calculated considering the individual interpupillary distance at far. Our results demonstrate that the PowerRef 3 gaze data deviated from the expected vergence angles by 9.64 ± 2.73° at 25 cm and 9.25 ± 3.52° at 6 m. The kappa angle calibration method reduced the discrepancy to 3.93 ± 1.19° at 25 cm and 3.70 ± 0.36° at 600 cm, whereas the linear regression method further improved the accuracy to 3.30 ± 0.86° at 25 cm and 0.26 ± 0.01° at 600 cm. Both methods improved the gaze results, with the linear regression calibration method showing greater overall accuracy.

Evaluation of monocular and binocular contrast perception on virtual reality head-mounted displays

Purpose

Visualization of medical images on a virtual reality (VR) head-mounted display (HMD) requires binocular fusion of a stereoscopic pair of graphical views. However, current image quality assessment on VR HMDs for medical applications has been primarily limited to time-consuming monocular optical bench measurement on a single eyepiece.

Approach

As an alternative to optical bench measurement to quantify the image quality on VR HMDs, we developed a WebXR test platform to perform contrast perceptual experiments that can be used for binocular image quality assessment. We obtained monocular and binocular contrast sensitivity responses (CSRs) from participants on a Meta Quest 2 VR HMD using varied interpupillary distance (IPD) configurations.

Results

The perceptual result shows that contrast perception on VR HMDs is primarily affected by optical aberration of the VR HMD. As a result, monocular CSR degrades at a high spatial frequency greater than 4 cycles per degree when gazing at the periphery of the display field of view, especially for mismatched IPD settings consistent with optical bench measurements. On the contrary, binocular contrast perception is dominated by the monocular view with superior image quality measured by the contrast.

Conclusions

We developed a test platform to investigate monocular and binocular contrast perception by performing perceptual experiments. The test method can be used to evaluate monocular and/or binocular image quality on VR HMDs for potential medical applications without extensive optical bench measurements.

Integrating eye rotation and contrast sensitivity into image quality evaluation of virtual reality head-mounted displays

Visual perception on virtual reality head-mounted displays (VR HMDs) involves human vision in the imaging pipeline. Image quality evaluation of VR HMDs may need to be expanded from optical bench testing by incorporating human visual perception. In this study, we implement a 5-degree-of-freedom (5DoF) experimental setup that simulates the human eye geometry and rotation mechanism. Optical modulation transfer function (MTF) measurements are performed using various camera rotation configurations namely pupil rotation, eye rotation, and eye rotation with angle kappa of the human visual system. The measured MTFs of the VR HMD are inserted into a human eye contrast sensitivity model to predict the perceptual contrast sensitivity function (CSF) on a VR HMD. At the same time, we develop a WebXR test platform to perform human observer experiments. Monocular CSFs of human subjects with different interpupillary distance (IPD) are extracted and compared with those calculated from optical MTF measurements. The result shows that image quality, measured as MTF and CSF, degrades at the periphery of display field of view, especially for subjects with an IPD different than that of the HMD. We observed that both the shift of visual point on the HMD eyepiece and the angle between the optical axes of the eye and eyepiece degrade image quality due to optical aberration. The computed CSFs from optical measurement correlates with those of the human observer experiment, with the optimal correlation achieved using the eye rotation with angle kappa setup. The finding demonstrates that more precise image quality assessment can be achieved by integrating eye rotation and human eye contrast sensitivity into optical bench testing.

High-fidelity color characterization in virtual reality across head mounted displays, game engines, and materials

We present a comprehensive colorimetric analysis of three head mounted displays (HMDs) - HTC Vive Pro Eye, Pimax 8K X DMAS, and Varjo Aero - focusing on their color calibration and uniformity across different game engines (Unity and Unreal) and for different materials/shaders. We developed a robust methodology combining hardware and software tools, including spectroradiometry and imaging colorimetry, to characterize and calibrate these HMDs for accurate color reproduction. The study showcases substantial advancements in colorimetric accuracy, with a reduction in the average deltaE00 of 90% or more across all tested HMDs and conditions. This level of color reproduction quality is below human discrimination thresholds, ensuring that any color inaccuracies remain imperceptible to the human eye. We also identified key areas for improvement, particularly in display uniformity, which could impact peripheral color reproduction. By making our tools and code publicly available, this study aims to facilitate future research and development in virtual reality (VR) technology, emphasizing the importance of color fidelity in virtual environments. The new insight enabled by our work is the extension and application of a traditional calibration method to currently available HMDs.

How robust are wearable eye trackers to slow and fast head and body movements?

How well can modern wearable eye trackers cope with head and body movement? To investigate this question, we asked four participants to stand still, walk, skip, and jump while fixating a static physical target in space. We did this for six different eye trackers. All the eye trackers were capable of recording gaze during the most dynamic episodes (skipping and jumping). The accuracy became worse as movement got wilder. During skipping and jumping, the biggest error was 5.8∘. However, most errors were smaller than 3∘. We discuss the implications of decreased accuracy in the context of different research scenarios.

Learning bio-inspired head-centric representations of 3D shapes in an active fixation setting

When exploring the surrounding environment with the eyes, humans and primates need to interpret three-dimensional (3D) shapes in a fast and invariant way, exploiting a highly variant and gaze-dependent visual information. Since they have front-facing eyes, binocular disparity is a prominent cue for depth perception. Specifically, it serves as computational substrate for two ground mechanisms of binocular active vision: stereopsis and binocular coordination. To this aim, disparity information, which is expressed in a retinotopic reference frame, is combined along the visual cortical pathways with gaze information and transformed in a head-centric reference frame. Despite the importance of this mechanism, the underlying neural substrates still remain widely unknown. In this work, we investigate the capabilities of the human visual system to interpret the 3D scene exploiting disparity and gaze information. In a psychophysical experiment, human subjects were asked to judge the depth orientation of a planar surface either while fixating a target point or while freely exploring the surface. Moreover, we used the same stimuli to train a recurrent neural network to exploit the responses of a modelled population of cortical (V1) cells to interpret the 3D scene layout. The results for both human performance and from the model network show that integrating disparity information across gaze directions is crucial for a reliable and invariant interpretation of the 3D geometry of the scene.