Orientation and mobility assessment in retinal prosthetic clinical trials

Measuring visually guided motor performance in ultra low vision using virtual reality

Introduction

Ultra low vision (ULV) refers to profound visual impairment where an individual cannot read even the top line of letters on an ETDRS chart from a distance of 0.5 m. There are limited tools available to assess visual ability in ULV. The aim of this study was to develop and calibrate a new performance test, Wilmer VRH, to assess hand-eye coordination in individuals with ULV.

Methods

A set of 55 activities was developed for presentation in a virtual reality (VR) headset. Activities were grouped into 2-step and 5-step items. Participants performed a range of tasks involving reaching and grasping, stacking, sorting, pointing, throwing, and cutting. Data were collected from 20 healthy volunteers under normal vision (NV) and simulated ULV (sULV) conditions, and from 33 participants with ULV. Data were analyzed using the method of successive dichotomizations (MSD), a polytomous Rasch model, to estimate item (difficulty) and person (ability) measures. MSD was applied separately to 2-step and 5-step performance data, then merged to a single equal interval scale.

Results

The mean ± SD of completion rates were 98.6 ± 1.8%, 78.2 ± 12.5% and 61.1 ± 34.2% for NV, sULV and ULV, respectively. Item measures ranged from -1.09 to 5.7 logits and - 4.3 to 4.08 logits and person measures ranged from -0.03 to 4.2 logits and -3.5 to 5.2 logits in sULV and ULV groups, respectively. Ninety percent of item infits were within the desired range of [0.5,1.5], and 97% of person infits were within that range. Together with item and person reliabilities of 0.94 and 0.91 respectively, this demonstrates unidimensionality of Wilmer VRH. A Person Item map showed that the items were well-targeted to the sample of individuals with ULV in the study.

Discussion

We present the development of a calibrated set of activities in VR that can be used to assess hand-eye coordination in individuals with ULV. This helps bridge a gap in the field by providing a validated outcome measure that can be used in vision restoration trials that recruit people with ULV, and to assess rehabilitation outcomes in people with ULV.

Self-Reported Visual Ability Versus Task Performance in Individuals With Ultra-Low Vision

Purpose

Visual functioning questionnaires are commonly used as patient-reported outcome measures to estimate visual ability. Performance measures, on the other hand, provide a direct measure of visual ability. For individuals with ultra-low vision (ULV; visual acuity (VA) <20/1600), the ultra-low vision visual functioning questionnaire (ULV-VFQ) and the Wilmer VRI-a virtual reality-based performance test-estimate self-reported and actual visual ability, respectively, for activities of daily living. But how well do self-reports from ULV-VFQ predict actual task performance in the Wilmer VRI?

Methods

We administered a subset of 10 matching items from the ULV-VFQ and Wilmer VRI to 27 individuals with ULV. We estimated item measures (task difficulty) and person measures (visual ability) using Rasch analysis for ULV-VFQ and using latent variable signal detection theory for the Wilmer VRI. We then used regression analysis to compare person and item measure estimates from self-reports and task performance.

Results

Item and person measures were modestly correlated between the two instruments, with r2 = 0.47 (P = 0.02) and r2 = 0.36 (P = 0.001), demonstrating that self-reports are an imperfect predictor of task difficulty and performance.

Conclusions

While self-reports impose a lower demand for equipment and personnel, actual task performance should be measured to assess visual ability in ULV.

Translational Relevance

Visual performance measures should be the preferred outcome measure in clinical trials recruiting individuals with ULV. Virtual reality can be used to standardize tasks.

Measuring visual information gathering in individuals with ultra low vision using virtual reality

People with ULV (visual acuity ≤ 20/1600 or 1.9 logMAR) lack form vision but have rudimentary levels of vision that can be used for a range of activities in daily life. However, current clinical tests are designed to assess form vision and do not provide information about the range of visually guided activities that can be performed in daily life using ULV. This is important to know given the growing number of clinical trials that recruit individuals with ULV (e.g., gene therapy, stem cell therapy) or restore vision to the ULV range in the blind (visual prosthesis). In this study, we develop a set of 19 activities (items) in virtual reality involving spatial localization/detection, motion detection, and direction of motion that can be used to assess visual performance in people with ULV. We estimated measures of item difficulty and person ability on a relative d prime (d') axis using a signal detection theory based analysis for latent variables. The items represented a range of difficulty levels (- 1.09 to 0.39 in relative d') in a heterogeneous group of individuals with ULV (- 0.74 to 2.2 in relative d') showing the instrument's utility as an outcome measure in clinical trials.

Towards aSmart Bionic Eye: AI-powered artificial vision for the treatment of incurable blindness

Objective.How can we return a functional form of sight to people who are living with incurable blindness? Despite recent advances in the development of visual neuroprostheses, the quality of current prosthetic vision is still rudimentary and does not differ much across different device technologies.Approach.Rather than aiming to represent the visual scene as naturally as possible, aSmart Bionic Eyecould provide visual augmentations through the means of artificial intelligence-based scene understanding, tailored to specific real-world tasks that are known to affect the quality of life of people who are blind, such as face recognition, outdoor navigation, and self-care.Main results.Complementary to existing research aiming to restore natural vision, we propose a patient-centered approach to incorporate deep learning-based visual augmentations into the next generation of devices.Significance.The ability of a visual prosthesis to support everyday tasks might make the difference between abandoned technology and a widely adopted next-generation neuroprosthetic device.

Functional evaluation in inherited retinal disease

Functional assessments are a fundamental part of the clinical evaluation of patients with inherited retinal diseases (IRDs). Their importance and impact have become increasingly notable, given the significant breadth and number of clinical trials and studies investigating multiple avenues of intervention across a wide range of IRDs, including gene, pharmacological and cellular therapies. Moreover, the fact that many clinical trials are reporting improvements in vision, rather than the previously anticipated structural stability/slowing of degeneration, makes functional evaluation of primary relevance. In this review, we will describe a range of methods employed to characterise retinal function and functional vision, beginning with tests variably included in the clinic, such as visual acuity, electrophysiological assessment and colour discrimination, and then discussing assessments often reserved for clinical trials/research studies such as photoaversion testing, full-field static perimetry and microperimetry, and vision-guided mobility testing; addressing perimetry in greatest detail, given it is commonly a primary outcome metric. We will focus on how these tests can help diagnose and monitor particular genotypes, also noting their limitations/challenges and exploring analytical methodologies for better exploiting functional measurements, as well as how they facilitate patient inclusion and stratification in clinical trials and serve as outcome measures.

Validation of a Vision-Guided Mobility Assessment for RPE65-Associated Retinal Dystrophy

Purpose

To validate a vision-guided mobility assessment for individuals affected by RPE65-associated retinal dystrophy (RPE65-RD).

Methods

In this comparative cross-sectional study, 29 subjects, comprising 19 subjects with RPE65-RD and 10 normally-sighted subjects undertook three assessments of mobility: following a straight line, navigating a simple maze, and stepping over a sidewalk "kerb." Performance was quantified as the time taken to complete each assessment, number of errors made, walking speed, and percent preferred walking speed, for each assessment. Subjects also undertook assessments of visual acuity, contrast sensitivity, full-field static perimetry, and age-appropriate quality of life questionnaires. To identify the most relevant metric to quantify vision-guided mobility, we investigated repeatability, as well as convergent, discriminant, and criterion validity. We also measured the effect of illumination on mobility.

Results

Walking speed through the maze assessment best discriminated between RPE65-RD and normally-sighted subjects, with both convergent and discriminant validity. Walking speed also approached statistical significance when assessed for criterion validity (P = 0.052). Subjects with RPE65-RD had quantifiably poorer mobility at lower illumination levels. A relatively small mean difference (-0.09 m/s) was identified in comparison to a relatively large repeatability coefficient (1.10 m/s).

Conclusions

We describe a novel, quantifiable, repeatable, and valid assessment of mobility designed specifically for subjects with RPE65-RD. The assessment is sensitive to the visual impairment of individuals with RPE65-RD in low illumination, identifies the known phenotypic heterogeneity and will furthermore provide an important outcome measure for RPE65-RD.

Translational Relevance

This assessment of vision-guided mobility, validated in a dedicated cohort of subjects with RPE65-RD, is a relevant and quantifiable outcome measure for RPE65-RD.

Harmonization of Outcomes and Vision Endpoints in Vision Restoration Trials: Recommendations from the International HOVER Taskforce

Translational research in vision prosthetics, gene therapy, optogenetics, stem cell and other forms of transplantation, and sensory substitution is creating new therapeutic options for patients with neural forms of blindness. The technical challenges faced by each of these disciplines differ considerably, but they all face the same challenge of how to assess vision in patients with ultra-low vision (ULV), who will be the earliest subjects to receive new therapies.

Historically, there were few tests to assess vision in ULV patients. In the 1990s, the field of visual prosthetics expanded rapidly, and this activity led to a heightened need to develop better tests to quantify end points for clinical studies. Each group tended to develop novel tests, which made it difficult to compare outcomes across groups. The common lack of validation of the tests and the variable use of controls added to the challenge of interpreting the outcomes of these clinical studies.

In 2014, at the bi-annual International “Eye and the Chip” meeting of experts in the field of visual prosthetics, a group of interested leaders agreed to work cooperatively to develop the International Harmonization of Outcomes and Vision Endpoints in Vision Restoration Trials (HOVER) Taskforce. Under this banner, more than 80 specialists across seven topic areas joined an effort to formulate guidelines for performing and reporting psychophysical tests in humans who participate in clinical trials for visual restoration. This document provides the complete version of the consensus opinions from the HOVER taskforce, which, together with its rules of governance, will be posted on the website of the Henry Ford Department of Ophthalmology (www.artificialvision.org).

Research groups or companies that choose to follow these guidelines are encouraged to include a specific statement to that effect in their communications to the public. The Executive Committee of the HOVER Taskforce will maintain a list of all human psychophysical research in the relevant fields of research on the same website to provide an overview of methods and outcomes of all clinical work being performed in an attempt to restore vision to the blind. This website will also specify which scientific publications contain the statement of certification. The website will be updated every 2 years and continue to exist as a living document of worldwide efforts to restore vision to the blind.

The HOVER consensus document has been written by over 80 of the world's experts in vision restoration and low vision and provides recommendations on the measurement and reporting of patient outcomes in vision restoration trials.

Data Acquisition, Processing, and Reduction for Home-Use Trial of a Wearable Video Camera-Based Mobility Aid

Purpose

Evaluating mobility aids in naturalistic conditions across many days is challenging owing to the sheer amount of data and hard-to-control environments. For a wearable video camera-based collision warning device, we present the methodology for acquisition, reduction, review, and coding of video data for quantitative analyses of mobility outcomes in blind and visually impaired participants.

Methods

Scene videos along with collision detection information were obtained from a chest-mounted collision warning device during daily use of the device. The recorded data were analyzed after use. Collision risk events flagged by the device were manually reviewed and coded using a detailed annotation protocol by two independent masked reviewers. Data reduction was achieved by predicting agreements between reviewers based on a machine learning algorithm. Thus, only those events for which disagreements were predicted would be reviewed by the second reviewer. Finally, the ultimate disagreements were resolved via consensus, and mobility-related outcome measures such as percentage of body contacts were obtained.

Results

There were 38 hours of device use from 10 participants that were reviewed by both reviewers, with an agreement level of 0.66 for body contacts. The machine learning algorithm trained on 2714 events correctly predicted 90.5% of disagreements. For another 1943 events, the trained model successfully predicted 82% of disagreements, resulting in 81% data reduction.

Conclusions

The feasibility of mobility aid evaluation based on a large volume of naturalistic data is demonstrated. Machine learning–based disagreement prediction can lead to data reduction.

Translational Relevance

These methods provide a template for determining the real-world benefit of a mobility aid.

Word recognition: re-thinking prosthetic vision evaluation

OBJECTIVE

Evaluations of vision prostheses and sensory substitution devices have frequently relied on repeated training and then testing with the same small set of items. These multiple forced-choice tasks produced above chance performance in blind users, but it is unclear if the observed performance represents restoration of vision that transfers to novel, untrained items.

APPROACH

Here, we tested the generalizability of the forced-choice paradigm on discrimination of low-resolution word images. Extensive visual training was conducted with the same 10 words used in previous BrainPort tongue stimulation studies. The performance on these 10 words and an additional 50 words was measured before and after the training sessions.

MAIN RESULTS

The results revealed minimal performance improvement with the untrained words, demonstrating instead pattern discrimination limited mostly to the trained words.

SIGNIFICANCE

These findings highlight the need to reconsider current evaluation practices, in particular, the use of forced-choice paradigms with a few highly trained items. While appropriate for measuring the performance thresholds in acuity or contrast sensitivity of a functioning visual system, performance on such tasks cannot be taken to indicate restored spatial pattern vision.

Optimising technology to measure functional vision, mobility and service outcomes for people with low vision or blindness: protocol for a prospective cohort study in Australia and Malaysia

INTRODUCTION

Orientation and mobility (O&M) specialists assess the functional vision and O&M skills of people with mobility problems, usually relating to low vision or blindness. There are numerous O&M assessment checklists but no measures that reduce qualitative assessment data to a single comparable score suitable for assessing any O&M client, of any age or ability, in any location. Functional measures are needed internationally to align O&M assessment practices, guide referrals, profile O&M clients, plan appropriate services and evaluate outcomes from O&M programmes (eg, long cane training), assistive technology (eg, hazard sensors) and medical interventions (eg, retinal implants). This study aims to validate two new measures of functional performance vision-related outcomes in orientation and mobility (VROOM) and orientation and mobility outcomes (OMO) in the context of ordinary O&M assessments in Australia, with cultural comparisons in Malaysia, also developing phone apps and online training to streamline professional assessment practices.

METHODS AND ANALYSIS

This multiphase observational study will employ embedded mixed methods with a qualitative/quantitative priority: corating functional vision and O&M during social inquiry. Australian O&M agencies (n=15) provide the sampling frame. O&M specialists will use quota sampling to generate cross-sectional assessment data (n=400) before investigating selected cohorts in outcome studies. Cultural relevance of the VROOM and OMO tools will be investigated in Malaysia, where the tools will inform the design of assistive devices and evaluate prototypes. Exploratory and confirmatory factor analysis, Rasch modelling, cluster analysis and analysis of variance will be undertaken along with descriptive analysis of measurement data. Qualitative findings will be used to interpret VROOM and OMO scores, filter statistically significant results, warrant their generalisability and identify additional relevant constructs that could also be measured.

ETHICS AND DISSEMINATION

Ethical approval has been granted by the Human Research Ethics Committee at Swinburne University (SHR Project 2016/316). Dissemination of results will be via agency reports, journal articles and conference presentations.

Novel mobility test to assess functional vision in patients with inherited retinal dystrophies

IMPORTANCE

This novel endpoint tracks functional vision changes in patients with inherited retinal dystrophies (IRDs) over time.

BACKGROUND

The aims of the study were to determine whether a multi-luminance mobility test (MLMT) can detect functional vision changes over time in subjects with IRDs and to assess natural history and potential effects of investigational agents.

DESIGN

This is a prospective, observational study.

PARTICIPANTS

Sixty-two subjects were enrolled. Sixty (29 normal sighted and 31 visually impaired) were eligible; 54 (28 visually impaired and 26 normal-sighted) completed all testing visits.

METHODS

Subjects navigated MLMT courses three times over 1 year. At each visit, subjects completed testing using individual eyes, and both eyes, at up to nine standardized, increasing luminance levels (range 1 to 400 lux). Accuracy and speed were evaluated and compared with visual acuity (VA), visual field (VF) and a visual function questionnaire.

MAIN OUTCOME MEASURES

Accuracy and speed of normal and visually impaired subjects on MLMT, and reliability and content validity of MLMT were the main outcome measures.

RESULTS

MLMT distinguished normal-sighted from visually impaired subjects. All control subjects passed all MLMT attempts at all tested light levels. Visually impaired subjects' performance varied widely; some declined over 1 year. Performance declined markedly below certain VA and VF thresholds. Concordance on performance on two baseline visits was high: correlations for accuracy were 94% and 98% for lowest common and highest common lux levels.

CONCLUSIONS AND RELEVANCE

MLMT differentiated visually impaired from control populations and, in visually impaired subjects, identified a range of performances; and tracked performance declines over time, consistent with these progressive conditions.

Developing a Very Low Vision Orientation and Mobility Test Battery (O&M-VLV)

PURPOSE

This study aimed to determine the feasibility of an assessment of vision-related orientation and mobility (O&M) tasks in persons with severe vision loss. These tasks may be used for future low vision rehabilitation clinical assessments or as outcome measures in vision restoration trials.

METHODS

Forty legally blind persons (mean visual acuity logMAR 2.3, or hand movements) with advanced retinitis pigmentosa participated in the Orientation & Mobility-Very Low Vision (O&M-VLV) subtests from the Low Vision Assessment of Daily Activities (LoVADA) protocol. Four categories of tasks were evaluated: route travel in three indoor hospital environments, a room orientation task (the "cafe"), a visual exploration task (the "gallery"), and a modified version of the Timed Up and Go (TUG) test, which assesses re-orientation and route travel. Spatial cognition was assessed using the Stuart Tactile Maps test. Visual acuity and visual fields were measured.

RESULTS

A generalized linear regression model showed that a number of measures in the O&M-VLV tasks were related to residual visual function. The percentage of preferred walking speed without an aid on three travel routes was associated with visual field (p < 0.01 for all routes) whereas the number of contacts with obstacles during route travel was associated with acuity (p = 0.001). TUG-LV task time was associated with acuity (p = 0.003), as was the cafe time and distance traveled (p = 0.006 and p < 0.001, respectively). The gallery score was the only measure that was significantly associated with both residual acuity and fields (p < 0.001 and p = 0.001, respectively).

CONCLUSIONS

The O&M-VLV was designed to capture key elements of O&M performance in persons with severe vision loss, which is a population not often studied previously. Performance on these tasks was associated with both binocular visual acuity and visual field. This new protocol includes assessments of orientation, which may be of benefit in vision restoration clinical trials.

Development of the Ultra-Low Vision Visual Functioning Questionnaire (ULV-VFQ)

Purpose

To develop and psychometrically evaluate a visual functioning questionnaire (VFQ) in an ultra-low vision (ULV) population.

Methods

Questionnaire items, based on visual activities self-reported by a ULV population, were categorized by functional visual domain (e.g., mobility) and visual aspect (e.g., contrast) to ensure a representative distribution. In Round 1, an initial set of 149 items was generated and administered to 90 participants with ULV (visual acuity [VA] ≤ 20/500; mean [SD] age 61 [15] years), including six patients with a retinal implant. Psychometric properties were evaluated through Rasch analysis and a revised set (150 items) was administered to 80 participants in Round 2.

Results

In Round 1, the person measure distribution (range, 8.6 logits) was centered at −1.50 logits relative to the item measures. In Round 2, the person measure distribution (range, 9.5 logits) was centered at −0.86 relative to the item mean. The reliability index in both rounds was 0.97 for Items and 0.99 for Persons. Infit analysis showed four underfit items in Round 1, five underfit items in Round 2 with a z-score greater than 4 cutoff. Principal component analysis on the residuals found 69.9% explained variance; the largest component in the unexplained variance was less than 3%.

Conclusions

The ULV-VFQ, developed with content generated from a ULV population, showed excellent psychometric properties as well as superior measurement validity in a ULV population.

Translational Relevance

The ULV-VFQ, part of the Prosthetic Low Vision Rehabilitation (PLoVR) development program, is a new VFQ developed for assessment of functional vision in ULV populations.

Restoration of Vision following Long-Term Blindness: Considerations for Providing Rehabilitation

The purpose of this article is to introduce the reader to a new area of research and practice known as vision restoration. The various approaches to restoration will be introduced and the implications of restored vision on rehabilitation strategies will be presented.

FLORA™: Phase I development of a functional vision assessment for prosthetic vision users

BACKGROUND

Research groups and funding agencies need a functional assessment suitable for an ultra-low vision population to evaluate the impact of new vision-restoration treatments. The purpose of this study was to develop a pilot assessment to capture the functional visual ability and well-being of subjects whose vision has been partially restored with the Argus II Retinal Prosthesis System.

METHODS

The Functional Low-Vision Observer Rated Assessment (FLORA) pilot assessment involved a self-report section, a list of functional visual tasks for observation of performance and a case narrative summary. Results were analysed to determine whether the interview questions and functional visual tasks were appropriate for this ultra-low vision population and whether the ratings suffered from floor or ceiling effects. Thirty subjects with severe to profound retinitis pigmentosa (bare light perception or worse in both eyes) were enrolled in a clinical trial and implanted with the Argus II System. From this population, 26 subjects were assessed with the FLORA. Seven different evaluators administered the assessment.

RESULTS

All 14 interview questions were asked. All 35 tasks for functional vision were selected for evaluation at least once, with an average of 20 subjects being evaluated for each test item. All four rating options—impossible (33 per cent), difficult (23 per cent), moderate (24 per cent) and easy (19 per cent)—were used by the evaluators. Evaluators also judged the amount of vision they observed the subjects using to complete the various tasks, with 'vision only' occurring 75 per cent on average with the System ON, and 29 per cent with the System OFF.

CONCLUSION

The first version of the FLORA was found to contain useful elements for evaluation and to avoid floor and ceiling effects. The next phase of development will be to refine the assessment and to establish reliability and validity to increase its value as an assessment tool for functional vision and well-being.

A standardized obstacle course for assessment of visual function in ultra low vision and artificial vision

We describe an indoor, portable, standardized course that can be used to evaluate obstacle avoidance in persons who have ultralow vision. Six sighted controls and 36 completely blind but otherwise healthy adult male (n=29) and female (n=13) subjects (age range 19-85 years), were enrolled in one of three studies involving testing of the BrainPort sensory substitution device. Subjects were asked to navigate the course prior to, and after, BrainPort training. They completed a total of 837 course runs in two different locations. Means and standard deviations were calculated across control types, courses, lights, and visits. We used a linear mixed effects model to compare different categories in the PPWS (percent preferred walking speed) and error percent data to show that the course iterations were properly designed. The course is relatively inexpensive, simple to administer, and has been shown to be a feasible way to test mobility function. Data analysis demonstrates that for the outcome of percent error as well as for percentage preferred walking speed, that each of the three courses is different, and that within each level, each of the three iterations are equal. This allows for randomization of the courses during administration.