Application of virtual reality head mounted display for investigation of movement: a novel effect of orientation of attention

Image2Brain: a cross-modality model for blind stereoscopic image quality ranking

Objective.Human beings perceive stereoscopic image quality through the cerebral visual cortex, which is a complex brain activity. As a solution, the quality of stereoscopic images can be evaluated more accurately by attempting to replicate the human perception from electroencephalogram (EEG) signals on image quality in a machine, which is different from previous stereoscopic image quality assessment methods focused only on the extraction of image features.Approach.Our proposed method is based on a novel image-to-brain (I2B) cross-modality model including a spatial-temporal EEG encoder (STEE) and an I2B deep convolutional generative adversarial network (I2B-DCGAN). Specifically, the EEG representations are first learned by STEE as real samples of I2B-DCGAN, which is designed to extract both quality and semantic features from the stereoscopic images by a semantic-guided image encoder, and utilize a generator to conditionally create the corresponding EEG features for images. Finally, the generated EEG features are classified to predict the image perceptual quality level.Main results.Extensive experimental results on the collected brain-visual multimodal stereoscopic image quality ranking database, demonstrate that the proposed I2B cross-modality model can better emulate the visual perception mechanism of the human brain and outperform the other methods by achieving an average accuracy of 95.95%.Significance.The proposed method can convert the learned stereoscopic image features into brain representations without EEG signals during testing. Further experiments verify that the proposed method has good generalization ability on new datasets and the potential for practical applications.

The effect of feedback in virtual attention training on orienting attention in individuals with sluggish cognitive tempo

OBJECTIVE

This study was conducted to assess the effectiveness of feedback in a virtual attention training program to improve the attentional characteristics of sluggish cognitive tempo (SCT).

METHOD

The SCT group (N = 60) and control group (N = 30) were identified, and the attention network test-revised (ANT-R) was performed to measure attention characteristics. Based on this result, a virtual reality (VR) feedback attention training program was developed to improve the efficiency of engagement and disengagement of attention in SCT. Sixty participants with SCT were recruited and grouped into two conditions: VR feedback (n = 30) and no-feedback (n = 30) conditions.

RESULTS

The results show that the VR attention training program with feedback significantly improves the attention-orienting network.

CONCLUSION

This suggests that it is necessary to provide immediate feedback for effective attention training for SCT and continuous intervention may be possible when feedback is provided together.

Accuracy and precision of stimulus timing and reaction times with Unreal Engine and SteamVR

The increasing interest in Virtual Reality (VR) as a tool for neuroscientific research contrasts with the current lack of established toolboxes and standards. In several recent studies, game engines like Unity or Unreal Engine were used. It remains to be tested whether these software packages provide sufficiently precise and accurate stimulus timing and time measurements that allow inferring ongoing mental and neural processes. We here investigated the precision and accuracy of the timing mechanisms of Unreal Engine 4 and SteamVR in combination with the HTC Vive VR system. In a first experiment, objective external measures revealed that stimulus durations were highly accurate. In contrast, in a second experiment, the assessment of the precision of built-in timing procedures revealed highly variable reaction time measurements and inaccurate determination of stimulus onsets. Hence, we developed a new software-based method that allows precise and accurate reaction time measurements with Unreal Engine and SteamVR. Instead of using the standard timing procedures implemented within Unreal Engine, time acquisition was outsourced to a background application. Timing benchmarks revealed that the newly developed method allows reaction time measurements with a precision and accuracy in the millisecond range. Overall, the present results indicate that the HTC Vive together with Unreal Engine and SteamVR can achieve high levels of precision and accuracy both concerning stimulus duration and critical time measurements. The latter can be achieved using a newly developed routine that allows not only accurate reaction time measures but also provides precise timing parameters that can be used in combination with time-sensitive functional measures such as electroencephalography (EEG) or transcranial magnetic stimulation (TMS).

Technological Competence Is a Pre-condition for Effective Implementation of Virtual Reality Head Mounted Displays in Human Neuroscience: A Technological Review and Meta-Analysis

Immersive virtual reality (VR) emerges as a promising research and clinical tool. However, several studies suggest that VR induced adverse symptoms and effects (VRISE) may undermine the health and safety standards, and the reliability of the scientific results. In the current literature review, the technical reasons for the adverse symptomatology are investigated to provide suggestions and technological knowledge for the implementation of VR head-mounted display (HMD) systems in cognitive neuroscience. The technological systematic literature indicated features pertinent to display, sound, motion tracking, navigation, ergonomic interactions, user experience, and computer hardware that should be considered by the researchers. Subsequently, a meta-analysis of 44 neuroscientific or neuropsychological studies involving VR HMD systems was performed. The meta-analysis of the VR studies demonstrated that new generation HMDs induced significantly less VRISE and marginally fewer dropouts. Importantly, the commercial versions of the new generation HMDs with ergonomic interactions had zero incidents of adverse symptomatology and dropouts. HMDs equivalent to or greater than the commercial versions of contemporary HMDs accompanied with ergonomic interactions are suitable for implementation in cognitive neuroscience. In conclusion, researchers' technological competency, along with meticulous methods and reports pertinent to software, hardware, and VRISE, are paramount to ensure the health and safety standards and the reliability of neuroscientific results.

Development of novel adjustable focus head mount display for concurrent image-guided treatment applications

PURPOSE

A conventional see-through head mount display contains many optical lenses, which can be problematic in image-guided treatment applications due to its size, weight, structure, and focus limitation. Therefore, we have designed a new type of see-through head mount display with a reduced number of optical lenses and an adequate optical resolution that can be utilized for image-guided treatment applications.

MATERIALS AND METHODS

A new type of adjustable focus head mount display with expanded virtual images and an external treatment space that can be provided to the eyes of a user by enlarging the images of a small display is designed and investigated in this study. This type of head mount display can be used in image-guided treatment applications because of the dual paths of imaging and treatment from the optical systems. Therefore, this system with an adjustable focus function can aid doctors in obtaining images for the treatment of the eyes of patients because every patient has a unique pupil size.

RESULTS

The results of the adjustable focus see-through head mount display showed distortion values of +0.36% in the +1 diopter location and -0.55% in the -4 diopter location, and there are less significant modulation transfer function differences within the ±5 diopter locations.

CONCLUSIONS

Low optical distortions within ±0.5 diopters can help doctors image the eye conditions of patients through fewer image processing techniques. Therefore, the designed adjustable focus head mount display can provide low optical aberrations and high optical modulation transfer function resolutions for image-guided treatment applications.

Sampling rate influences saccade detection in mobile eye tracking of a reading task

The purpose of this study was to compare saccade detection characteristics in two mobile eye trackers with different sampling rates in a natural task. Gaze data of 11 participants were recorded in one 60 Hz and one 120 Hz mobile eye tracker and compared directly to the saccades detected by a 1000 HZ stationary tracker while a reading task was performed. Saccades and fixations were detected using a velocity based algorithm and their properties analyzed. Results showed that there was no significant difference in the number of detected fixations but mean fixation durations differed between the 60 Hz mobile and the stationary eye tracker. The 120 Hz mobile eye tracker showed a significant increase in the detection rate of saccades and an improved estimation of the mean saccade duration, compared to the 60 Hz eye tracker. To conclude, for the detection and analysis of fast eye movements, such as saccades, it is better to use a 120 Hz mobile eye tracker.