Heading assessment by "tunnel vision" patients and control subjects standing or walking in a virtual reality environment

Where was this thing again? Evaluating methods to indicate remembered object positions in virtual reality

A current focus in sensorimotor research is the study of human perception and action in increasingly naturalistic tasks and visual environments. This is further enabled by the recent commercial success of virtual reality (VR) technology, which allows for highly realistic but well-controlled three-dimensional (3D) scenes. VR enables a multitude of different ways to interact with virtual objects, but only rarely are such interaction techniques evaluated and compared before being selected for a sensorimotor experiment. Here, we compare different response techniques for a memory-guided action task, in which participants indicated the position of a previously seen 3D object in a VR scene: pointing, using a virtual laser pointer of short or unlimited length, and placing, either the target object itself or a generic reference cube. Response techniques differed in availability of 3D object cues and requirement to physically move to the remembered object position by walking. Object placement was the most accurate but slowest due to repeated repositioning. When placing objects, participants tended to match the original object's orientation. In contrast, the laser pointer was fastest but least accurate, with the short pointer showing a good speed-accuracy compromise. Our findings can help researchers in selecting appropriate methods when studying naturalistic visuomotor behavior in virtual environments.

Egocentric Boundaries on Distinguishing Colliding and Non-Colliding Pedestrians while Walking in a Virtual Environment

Avoiding person-to-person collisions is critical for visual field loss patients. Any intervention claiming to improve the safety of such patients should empirically demonstrate its efficacy. To design a VR mobility testing platform presenting multiple pedestrians, a distinction between colliding and non-colliding pedestrians must be clearly defined. We measured nine normally sighted subjects' collision envelopes (CE; an egocentric boundary distinguishing collision and non-collision) and found it changes based on the approaching pedestrian's bearing angle and speed. For person-to-person collision events for the VR mobility testing platform, non-colliding pedestrians should not evade the CE.

Optimization and Validation of a Virtual Reality Orientation and Mobility Test for Inherited Retinal Degenerations

Purpose

To optimize a virtual reality (VR) orientation and mobility (O&M) test of functional vision in patients with inherited retinal degenerations (IRDs).

Methods

We developed an O&M test using commercially available VR hardware and custom-generated software. Normally sighted subjects (n = 20, ages = 14-67 years) and patients with IRDs (n = 29, ages = 15-63 years) participated. Individuals followed a dim red arrow path to a "course exit," while trying to identify nine obstacles adjacent to, or directly in their path. Dark-adapted subjects completed 35 randomly selected VR courses at increasing luminances, twice per luminance step, binocularly, and uni-ocularly. Performance was graded automatically by the software. Patients with IRD completed a modified Visual Function Questionnaire (VFQ).

Results

Normally sighted subjects identified approximately 50% of the obstacles at the dimmest course luminance. Except for two patients with IRD with poor vision, all patients were able to complete the test, although they required brighter (by >2 log units) luminances to identify 50% of the obstacles. In a single-luminance screening test in which normal subjects detected at least eight of nine objects, most patients with IRD underperformed; their performance related to disease severity, as measured by visual acuity, kinetic visual field extent, and VFQ scores. Test-retest differences in object detection were similar to the differences between the two eyes (±2 SD = ±2 objects).

Conclusions

This VR-O&M test was able to distinguish subjects with IRDs from normal subjects reliably and reproducibly.

Translational Relevance

This easily implemented, flexible, and objectively scored VR-O&M test promises to become a useful tool to assess the impact that IRDs and their treatments have on functional vision.

A Virtual Reality Orientation and Mobility Test for Inherited Retinal Degenerations: Testing a Proof-of-Concept After Gene Therapy

Purpose

To test the ability of a virtual reality (VR) orientation and mobility (O&M) protocol to serve a measure of functional vision for patients with inherited retinal degenerations (IRDs).

Methods

A VR-O&M protocol designed using a commercially available VR hardware was tested in normally sighted control subjects (n=7; ages 10–35yo; Average 22.5yo) and patients with RPE65-associated Leber Congenital Amaurosis (n=3; ages 7–18yo; Average 12.7yo), in two of them before and after gene therapy. Patients underwent perimetry and full-field sensitivity testing. VR-O&M parameters correlated with the visual dysfunction.

Results

Visual acuities in RPE65 patients were on average worse than 20/200, dark-adapted sensitivity losses >5 log units, and fields constricted between 20° and 40°. Before treatment, patients required ~1000-fold brighter environment to navigate, had at least x4 more collisions, and were slower both to orient and navigate compared to control subjects. Improvements in cone- (by 1–2 L.u.) and rod-mediated (by >4 L.u.) sensitivities post-treatment led to fewer collisions (at least by half) at ~100-fold dimmer luminances, and to x4 times faster navigation times.

Conclusion

This study provides proof-of-concept data in support for the use of VR-O&M systems to quantify the impact that the visual dysfunction and improvement of vision following treatments has on functional vision in IRDs. The VR-O&M was useful in potentially challenging scenarios such as in pediatric patients with severe IRDs.

Translational Relevance

A VR-O&M test will provide much needed flexibility, both in its deployment as well as in the possibility to test various attributes of vision that may be impacted by gene therapy in the setting of translational studies.

Precis

This study provides proof-of-concept data in support for the use of a virtual reality orientation and mobility test to quantify the impact of the disease and of treatments thereof on functional vision in inherited retinal degenerations.

Neurolight: A Deep Learning Neural Interface for Cortical Visual Prostheses

Visual neuroprosthesis, that provide electrical stimulation along several sites of the human visual system, constitute a potential tool for vision restoration for the blind. Scientific and technological progress in the fields of neural engineering and artificial vision comes with new theories and tools that, along with the dawn of modern artificial intelligence, constitute a promising framework for the further development of neurotechnology. In the framework of the development of a Cortical Visual Neuroprosthesis for the blind (CORTIVIS), we are now facing the challenge of developing not only computationally powerful tools and flexible approaches that will allow us to provide some degree of functional vision to individuals who are profoundly blind. In this work, we propose a general neuroprosthesis framework composed of several task-oriented and visual encoding modules. We address the development and implementation of computational models of the firing rates of retinal ganglion cells and design a tool — Neurolight — that allows these models to be interfaced with intracortical microelectrodes in order to create electrical stimulation patterns that can evoke useful perceptions. In addition, the developed framework allows the deployment of a diverse array of state-of-the-art deep-learning techniques for task-oriented and general image pre-processing, such as semantic segmentation and object detection in our system’s pipeline. To the best of our knowledge, this constitutes the first deep-learning-based system designed to directly interface with the visual brain through an intracortical microelectrode array. We implement the complete pipeline, from obtaining a video stream to developing and deploying task-oriented deep-learning models and predictive models of retinal ganglion cells’ encoding of visual inputs under the control of a neurostimulation device able to send electrical train pulses to a microelectrode array implanted at the visual cortex.

Measuring Pedestrian Collision Detection With Peripheral Field Loss and the Impact of Peripheral Prisms

Purpose

Peripheral field loss (PFL) due to retinitis pigmentosa, choroideremia, or glaucoma often results in a highly constricted residual central field, which makes it difficult for patients to avoid collision with approaching pedestrians. We developed a virtual environment to evaluate the ability of patients to detect pedestrians and judge potential collisions. We validated the system with both PFL patients and normally sighted subjects with simulated PFL. We also tested whether properly placed high-power prisms may improve pedestrian detection.

Methods

A virtual park-like open space was rendered using a driving simulator (configured for walking speeds), and pedestrians in testing scenarios appeared within and outside the residual central field. Nine normally sighted subjects and eight PFL patients performed the pedestrian detection and collision judgment tasks. The performance of the subjects with simulated PFL was further evaluated with field of view expanding prisms.

Results

The virtual system for testing pedestrian detection and collision judgment was validated. The performance of PFL patients and normally sighted subjects with simulated PFL were similar. The prisms for simulated PFL improved detection rates, reduced detection response times, and supported reasonable collision judgments in the prism-expanded field; detections and collision judgments in the residual central field were not influenced negatively by the prisms.

Conclusions

The scenarios in a virtual environment are suitable for evaluating PFL and the impact of field of view expanding devices.

Translational Relevance

This study validated an objective means to evaluate field expansion devices in reproducible near-real-life settings.

ScreenMasker: An Open-source Gaze-contingent Screen Masking Environment

The moving-window paradigm, based on gazecontingent technic, traditionally used in a studies of the visual perceptual span. There is a strong demand for new environments that could be employed by non-technical researchers. We have developed an easy-to-use tool with a graphical user interface (GUI) allowing both execution and control of visual gaze-contingency studies. This work describes ScreenMasker, an environment that allows create gaze-contingent textured displays used together with stimuli presentation software. ScreenMasker has an architecture that meets the requirements of low-latency real-time eye-movement experiments. It also provides a variety of settings and functions. Effective rendering times and performance are ensured by means of GPU processing under CUDA technology. Performance tests show ScreenMasker's latency to be 67-74 ms on a typical office computer, and high-end 144-Hz screen latencies of about 25-28 ms. ScreenMasker is an open-source system distributed under the GNU Lesser General Public License and is available at https://github.com/PaulOrlov/ScreenMasker .

Outcome measure for the treatment of cone photoreceptor diseases: orientation to a scene with cone-only contrast

BACKGROUND

Inherited retinal degenerations (IRDs) preferentially affecting cone photoreceptor function are being considered for treatment trials aiming to improve day vision. The purpose of the current work was to develop cone-specific visual orientation outcomes that can differentiate day vision improvement in the presence of retained night vision.

METHODS

A lighted wall (1.4 m wide, 2 m high) resembling a beaded curtain was formed with 900 individually addressable red, blue and green LED triplets placed in 15 vertical strips hanging 0.1 m apart. Under computer control, different combination of colors and intensities were used to produce the appearance of a door on the wall. Scotopically-matched trials were designed to be perceptible to the cone-, but not rod-, photoreceptor based visual systems. Unmatched control trials were interleaved at each luminance level to determine the existence of any vision available for orientation. Testing started with dark-adapted eyes and a scene luminance attenuated 8 log units from the maximum attainable, and continued with progressively increasing levels of luminance. Testing was performed with a three-alternative forced choice method in healthy subjects and patients with Leber congenital amaurosis (LCA) caused by mutations in GUCY2D, the gene that encodes retinal guanylate cyclase-1.

RESULTS

Normal subjects could perform the orientation task using cone vision at 5 log attenuation and brighter luminance levels. Most GUCY2D-LCA patients failed to perform the orientation task with scotopically-matched test trials at any luminance level even though they were able to perform correctly with unmatched control trials. These results were consistent with a lack of cone system vision and use of the rod system under ambient conditions normally associated with cone system activity. Two GUCY2D-LCA patients demonstrated remnant cone vision but at a luminance level 2 log brighter than normal.

CONCLUSIONS

The newly developed device can probe the existence or emergence of cone-based vision in patients for an orientation task involving the identification of a door on the wall under free-viewing conditions. This key advance represents progress toward developing an appropriate outcome measure for a clinical trial to treat currently incurable eye diseases severely affecting cone vision despite retained rod vision.

Extended wearing trial of Trifield lens device for 'tunnel vision'

Severe visual field constriction (tunnel vision) impairs the ability to navigate and walk safely. We evaluated Trifield glasses as a mobility rehabilitation device for tunnel vision in an extended wearing trial. Twelve patients with tunnel vision (5-22 degrees wide) due to retinitis pigmentosa or choroideremia participated in the 5-visit wearing trial. To expand the horizontal visual field, one spectacle lens was fitted with two apex-to-apex prisms that vertically bisected the pupil on primary gaze. This provides visual field expansion at the expense of visual confusion (two objects with the same visual direction). Patients were asked to wear these spectacles as much as possible for the duration of the wearing trial (median 8, range 6-60 weeks). Clinical success (continued wear, indicating perceived overall benefit), visual field expansion, perceived direction and perceived visual ability were measured. Of 12 patients, nine chose to continue wearing the Trifield glasses at the end of the wearing trial. Of those nine patients, at long-term follow-up (35-78 weeks), three reported still wearing the Trifield glasses. Visual field expansion (median 18, range 9-38 degrees) was demonstrated for all patients. No patient demonstrated adaptation to the change in visual direction produced by the Trifield glasses (prisms). For reported difficulty with obstacles, some differences between successful and non-successful wearers were found. Trifield glasses provided reported benefits in obstacle avoidance to 7 of the 12 patients completing the wearing trial. Crowded environments were particularly difficult for most wearers. Possible reasons for long-term discontinuation and lack of adaptation to perceived direction are discussed.

A Feedback-Controlled Interface for Treadmill Locomotion in Virtual Environments

Virtual environments (VEs) allow safe, repeatable, and controlled evaluations of obstacle avoidance and navigation performance of people with visual impairments using visual aids. Proper simulation of mobility in a VE requires an interface, which allows subjects to set their walking pace. Using conventional treadmills, the subject can change their walking speed by pushing the tread with their feet, while leveraging handrails or ropes (self-propelled mode). We developed a feedback-controlled locomotion interface that allows the VE workstation to control the speed of the treadmill, based on the position of the user. The position and speed information is also used to implement automated safety measures, so that the treadmill can be halted in case of erratic behavior. We compared the feedback-controlled mode to the self-propelled mode by using speed-matching tasks (follow a moving object or match the speed of an independently moving scene) to measure the efficacy of each mode in maintaining constant subject position, subject control of the treadmill, and subject pulse rates. Additionally, we measured the perception of speed in the VE on each mode. The feedback-controlled mode required less physical exertion than self-propelled. The average position of subjects on the feedback-controlled treadmill was always within a centimeter of the desired position. There was a smaller standard deviation in subject position when using the self-propelled mode than when using the feedback-controlled mode, but the difference averaged less than six centimeters across all subjects walking at a constant speed. Although all subjects underestimated the speed of an independently moving scene at higher speeds, their estimates were more accurate when using the feedback-controlled treadmill than the self-propelled.