Feature search in persons with severe visual impairment

Field Evaluation of a Mobile App for Assisting Blind and Visually Impaired Travelers to Find Bus Stops

Purpose

GPS location-based navigation apps are insufficient to aid blind and visually impaired (BVI) travelers for micro-navigation tasks, such as finding the exact location of bus stops. The resulting large gaps could lead to BVI travelers missing their bus. We evaluated the ability of a signage detection mobile app, All_Aboard, to guide BVI travelers precisely to the bus stops compared to Google Maps alone.

Methods

The All_Aboard app detected bus stop signs in real-time via smartphone camera using a deep neural network model, and provided distance coded audio feedback to help localize the detected sign. BVI individuals used the All_Aboard and Google Maps app to localize 10 bus stops each in downtown and suburban Boston, Massachusetts. For each bus stop, the subjects used both apps to navigate as close as possible to the physical bus stop sign, starting from 30 to 50 meters away. The outcome measures were success rate and gap distance between the app-indicated location and the actual physical location of the bus stop.

Results

The study involved 24 legally blind participants (mean age [SD] = 51 [14] years; 11 [46%] women). The success rate of the All_Aboard app (91%) was significantly higher than the Google Maps (52%, P < 0.001). The gap distance when using the All_Aboard app was significantly lower (mean = 1.8, 95% confidence interval [CI] = 1.2-2.3 meters) compared to the Google Maps alone (mean = 7, 95% CI = 6.5-7.5 meters, P < 0.001).

Conclusions

All_Aboard micro-navigation app guided BVI travelers to bus stops more accurately and reliably than a location-based macro-navigation app alone.

Translational Relevance

The All_Aboard app together with a macro-navigation app can potentially help BVI individuals independently access public transportation.

Visual search and childhood vision impairment: A GAMLSS-oriented multiverse analysis approach

The aim of this report was to analyze reaction times and accuracy in children with a vision impairment performing a feature-based visual search task using a multiverse statistical approach. The search task consisted of set sizes 4, 16, and 24, consisting of distractors (circle) and a target (ellipse) that were presented randomly to school-aged individuals with or without a vision impairment. Interactions and main effects of key variables relating to reaction times and accuracy were analyzed via a novel statistical method blending GAMLSS (generalized additive models for location, scale, and shape) and distributional regression trees. Reaction times for the target-present and target-absent conditions were significantly slower in the vision impairment group with increasing set sizes (p < .001). Female participants were significantly slower than were males for set sizes 16 and 24 in the target-absent condition (p < .001), with male participants being significantly slower than females in the target-present condition (p < .001). Accuracy was only significantly worse (p = .03) for participants less than 14 years of age for the target-absent condition with set sizes 16 and 24. There was a positive association between binocular visual acuity and search time (p < .001). The application of GAMLSS with distributional regression trees to the analysis of visual search data may provide further insights into underlying factors affecting search performance in case-control studies where psychological or physical differences may influence visual search outcomes.

Saccades during visual search in macular degeneration

Macular degeneration (MD) compromises both high-acuity vision and eye movements when the foveal regions of both eyes are affected. Individuals with MD adapt to central field loss by adopting a preferred retinal locus (PRL) for fixation. Here, we investigate how individuals with bilateral MD use eye movements to search for targets in a visual scene under realistic binocular viewing conditions. Five individuals with binocular scotomata, 3 individuals with monocular scotomata and 6 age-matched controls participated in our study. We first extensively mapped the binocular scotoma with an eyetracker, while fixation was carefully monitored (Vullings & Verghese, 2020). Participants then completed a visual search task where 0, 1, or 2 Gaussian blobs were distributed randomly across a natural scene. Participants were given 10 s to actively search the display and report the number of blobs. An analysis of saccade characteristics showed that individuals with binocular scotomata made more saccades in the direction of their scotoma than controls for the same directions. Saccades in the direction of the scotoma were typically of small amplitude, and did not fully uncover the region previously hidden by the scotoma. Rather than make more saccades to explore this hidden region, participants frequently made saccades back toward newly uncovered regions. Backward saccades likely serve a similar purpose to regressive saccades exhibited during reading in MD, by inspecting previously covered regions near the direction of gaze. Our analysis suggests that the higher prevalence of backward saccades in individuals with binocular scotomata might be related to the PRL being adjacent to the scotoma.

The effects of induced optical blur on visual search performance and training

Visual search is a task often used in the rehabilitation of patients with cortical and non-cortical visual pathologies such as visual field loss. Reduced visual acuity is often comorbid with these disorders, and it remains poorly defined how low visual acuity may affect a patient’s ability to recover visual function through visual search training. The two experiments reported here investigated whether induced blurring of vision (from 6/15 to 6/60) in a neurotypical population differentially affected various types of feature search tasks, whether there is a minimal acceptable level of visual acuity required for normal search performance, and whether these factors affected the degree to which participants could improve with training. From the results, it can be seen that reducing visual acuity did reduce search speed, but only for tasks where the target was defined by shape or size (not colour), and only when acuity was worse than 6/15. Furthermore, searching behaviour was seen to improve with training in all three feature search tasks, irrespective of the degree of blurring that was induced. The improvement also generalised to a non-trained search task, indicating that an enhanced search strategy had been developed. These findings have important implications for the use of visual search as a rehabilitation aid for partial visual loss, indicating that individuals with even severe comorbid blurring should still be able to benefit from such training.

Novel computer-based assessments of everyday visual function in people with age-related macular degeneration

PURPOSE

To test the hypothesis that the performance in novel computer-based tasks of everyday visual function worsens with disease severity in people with non-neovascular age-related macular degeneration.

METHODS

Participants with and without non-neovascular age-related macular degeneration (≥60 years, minimum logMAR binocular visual acuity 0.7) performed a series of standard visual function tests and two novel computer-based tasks. In a visual search task, participants had to locate an image of a single real-world object within an array of 49 distractor images. Next, in a series of simulated dynamic driving scenes, participants were asked to identify one or two approaching real-world road signs and then select these road signs from four options. Outcome measures were median response times and total correct responses.

RESULTS

Forty-nine participants had no macular disease (n = 11), early/intermediate age-related macular degeneration (n = 16) or geographic atrophy (n = 22). Groups were age-similar with median (interquartile range) logMAR visual acuity of 0.00 (-0.08,0.12), 0.13 (-0.08,0.70) and 0.32 (0.12,0.70) respectively. Median (interquartile range) visual search response times were 1.9 (1.0,2.4), 1.8 (1.1,3.7) and 2.4 (1.2,6.0) seconds respectively. Median (interquartile range) road sign response times (single road signs) were 1.2 (0.4,1.7), 1.5 (0.9,2.8) and 1.8 (1.0,5.5) seconds respectively. Median (interquartile range) road sign response times (double road signs) were 1.7 (0.7,2.4), 2.3 (1.2,3.1) and 2.5 (1.7,6) seconds respectively. Participants with geographic atrophy recorded slower response times in all tasks and over 50% performed outside the normative limit for task performance. There were no significant differences between groups in total correct responses across all tasks.

CONCLUSIONS

In a novel computer-based assessment, people with increasing severity of age-related macular degeneration take longer to perform visual search of everyday objects and take longer to identify road signs than those with no age-related macular degeneration. These novel assessments could be useful as patient-relevant, secondary outcomes for clinical trials.

Object search in neovascular age-related macular degeneration: the crowding effect

BACKGROUND

Visual search, an activity that relies on central vision, is frequent in daily life. This study investigates the effect of spacing between items in an object search task in participants with central vision loss.

METHODS

Patients with neovascular age-related macular degeneration (AMD), age-matched controls, and young controls were included. The stimuli were displays of four, six and nine objects randomly presented in a 'crowded' (spacing 1.5°) or 'uncrowded' (spacing 6°) condition. For each of 96 trials, participants were asked to search for a predefined target that remained on the screen until the response was recorded. Accuracy, search time, and eye movements (number of fixations and scan path ratio) were recorded.

RESULTS

Compared to older controls, accuracy decreased by 31 per cent and search time increased by 61 per cent in AMD participants. Ageing also affected performance with a lower accuracy by 13.5 per cent and longer search times by 46 per cent in older compared to younger controls. Increasing the spacing between elements increased accuracy by 21 per cent in AMD participants but it had no effect in older and younger controls. Performance was not related to visual acuity or to duration of neovascular AMD, but search time was correlated to the lesion size in the 'crowded' condition.

CONCLUSIONS

Object search is ubiquitous in daily life activities. When visual acuity is irrevocably reduced, increasing the spacing between elements can reliably improve object search performance in patients.

A Mobile Application for Keyword Search in Real-World Scenes

Keyword search in a cluttered environment is difficult in general, and even more challenging for people with low vision. While magnification can help in reading for low vision people, it does not facilitate efficient visual search due to the constriction of the field of view. The motivating observation for this study is that, in a large number of visual search tasks, people know what are they looking for (i.e., they know the keywords), they just do not know where to find them in the scene. We have developed a mobile application that allows the users to input keywords (by voice or by typing), uses an optical character recognition (OCR) engine to search for the provided keyword in the scene captured by the smartphone camera, and zooms in on the instances of the keyword detected in the captured images, to facilitate efficient information acquisition. In this paper we describe the development and evaluation of various aspects of the application, including comparing the various mainstream OCR engines that power the app, and an evaluation study comparing the app to the conventional optical magnifier vision aid. Normally sighted adults, while wearing blur glasses to lower their visual acuity, performed keyword searches for a series of items ranging from easy to difficult with the app and with a handheld magnifier. While there was no difference in the search times between the two methods for the easier tasks, the app was significantly faster than the magnifier for the difficult tasks.

Tactile Cues for Improving Target Localization in Subjects with Tunnel Vision

The loss of peripheral vision is experienced by millions of people with glaucoma or retinitis pigmentosa, and has a major impact in everyday life, specifically to locate visual targets in the environment. In this study, we designed a wearable interface to render the location of specific targets with private and non-intrusive tactile cues. Three experimental studies were completed to design and evaluate the tactile code and the device. In the first study, four different tactile codes (single stimuli or trains of pulses rendered either in a Cartesian or a Polar coordinate system) were evaluated with a head pointing task. In the following studies, the most efficient code, trains of pulses with Cartesian coordinates, was used on a bracelet located on the wrist, and evaluated during a visual search task in a complex virtual environment. The second study included ten subjects with a simulated restrictive field of view (10°). The last study consisted of proof of a concept with one visually impaired subject with restricted peripheral vision due to glaucoma. The results show that the device significantly improved the visual search efficiency with a factor of three. Including object recognition algorithm to smart glass, the device could help to detect targets of interest either on demand or suggested by the device itself (e.g., potential obstacles), facilitating visual search, and more generally spatial awareness of the environment.

Visual Search Training and Obstacle Avoidance in Adults with Visual Impairments

This study assessed the effect of visual search training on the avoidance of obstacles by adults with visual impairments. A significant reduction in contacts with obstacles under mesopic conditions was found in individuals who received search training. This finding suggests that search training had a positive effect on mobility performance.

Does Central Vision Loss Impair Visual Search Performance of Adults More than Children?

SIGNIFICANCE

In general, young adults with normal vision show the best visual search performance when compared with children and older adults. Through our study, we show that this trend is not observed in individuals with vision impairment. An interaction effect of vision impairment with visual development and aging is observed.

PURPOSE

Performance in many visual tasks typically shows improvement with age until young adulthood and then declines with aging. Using a visual search task, this study investigated whether a similar age effect on performance is present in people with central vision loss.

METHODS

A total of 98 participants, 37 with normal sight (NS) and 61 with visual impairment (VI) searched for targets in 150 real-world digital images. Search performance was quantified by an integrated measure combining speed and accuracy. Participant ages ranged from 5 to 74 years, visual acuity from -0.14 (20/14.5) to 1.16 logMAR (20/290), and log contrast sensitivity (CS) from 0.48 to 2.0. Data analysis was performed with participants divided into three age groups: children (aged <14 years, n = 25), young adults (aged 14 to 45 years, n = 47), and older adults (aged >45 years, n = 26).

RESULTS

Regression (r = 0.7) revealed CS (P < .001) and age (P = .003) were significant predictors of search performance. Performance of VI participants was normalized to the age-matched average performance of the NS group. In the VI group, it was found that children's normalized performance (52%) was better than both young (39%, P = .05) and older (40%, P = .048) adults.

CONCLUSIONS

Unlike NS participants, young adults in the VI group may not have search ability superior to children with VI, despite having the same level of visual functions (quantified by visual acuity and CS). This could be because of vision impairment limiting the developmental acquisition of the age dividend for peak performance. Older adults in the VI group had the worst performance, indicating an interaction of aging.

Visual Search Performance in Patients with Vision Impairment: A Systematic Review

PURPOSE

Patients with visual impairment are constantly facing challenges to achieve an independent and productive life, which depends upon both a good visual discrimination and search capacities. Given that visual search is a critical skill for several daily tasks and could be used as an index of the overall visual function, we investigated the relationship between vision impairment and visual search performance.

METHODS

A comprehensive search was undertaken using electronic PubMed, EMBASE, LILACS, and Cochrane databases from January 1980 to December 2016, applying the following terms: "visual search", "visual search performance", "visual impairment", "visual exploration", "visual field", "hemianopia", "search time", "vision lost", "visual loss", and "low vision". Two hundred seventy six studies from 12,059 electronic database files were selected, and 40 of them were included in this review.

RESULTS

Studies included participants of all ages, both sexes, and the sample sizes ranged from 5 to 199 participants. Visual impairment was associated with worse visual search performance in several ophthalmologic conditions, which were either artificially induced, or related to specific eye and neurological diseases.

CONCLUSIONS

This systematic review details all the described circumstances interfering with visual search tasks, highlights the need for developing technical standards, and outlines patterns for diagnosis and therapy using visual search capabilities.

Searching for unity: Real-world versus item-based visual search in age-related eye disease

When studying visual search, item-based approaches using synthetic targets and distractors limit the real-world applicability of results. Everyday visual search can be impaired in patients with common eye diseases like glaucoma and age-related macular degeneration. We highlight some results in the literature that suggest assessment of real-word search tasks in these patients could be clinically useful.

A new measure for the assessment of visual awareness in individuals with tunnel vision

BACKGROUND

Individuals with a restricted peripheral visual field or tunnel vision (TV) have problems moving about and avoiding obstacles. Some individuals adapt better than others and some use assistive optical aids, so measurement of the visual field is not sufficient to describe their performance. In the present study, we developed a new clinical test called the 'Assessment of Visual Awareness (AVA)', which can be used to measure detection of peripheral targets.

METHODS

The participants were 20 patients with TV due to retinitis pigmentosa (PTV) and 50 normally sighted participants with simulated tunnel vision (STV) using goggles. In the AVA test, detection times were measured, when subjects searched for 24 individually presented, one degree targets, randomly positioned in a 60 degrees noise background. Head and eye movements were allowed and the presentation time was unlimited. The test validity was investigated by correlating the detection times with the 'percentage of preferred walking speed' (PPWS) and the 'number of collisions' on an indoor mobility course.

RESULTS

In PTV and STV, the detection times had significant negative correlation with the field of view. The detection times had significant positive relations with target location. In the STV, the detection time was significantly negatively correlated with the PPWS and significantly positively correlated with the collisions score on the indoor mobility course. In the PTV, the relationship was not statistically significant. No significant difference in performance of STV was found when repeating the test one to two weeks later.

CONCLUSION

The proposed AVA test was sensitive to the field of view and target location. The test is unique in design, quick, simple to deliver and both repeatable and valid. It could be a valuable tool to test different rehabilitation strategies in patients with TV.

Macular degeneration affects eye movement behavior during visual search

Patients with a scotoma in their central vision (e.g., due to macular degeneration, MD) commonly adopt a strategy to direct the eyes such that the image falls onto a peripheral location on the retina. This location is referred to as the preferred retinal locus (PRL). Although previous research has investigated the characteristics of this PRL, it is unclear whether eye movement metrics are modulated by peripheral viewing with a PRL as measured during a visual search paradigm. To this end, we tested four MD patients in a visual search paradigm and contrasted their performance with a healthy control group and a healthy control group performing the same experiment with a simulated scotoma. The experiment contained two conditions. In the first condition the target was an unfilled circle hidden among c-shaped distractors (serial condition) and in the second condition the target was a filled circle (pop-out condition). Saccadic search latencies for the MD group were significantly longer in both conditions compared to both control groups. Results of a subsequent experiment indicated that this difference between the MD and the control groups could not be explained by a difference in target selection sensitivity. Furthermore, search behavior of MD patients was associated with saccades with smaller amplitudes toward the scotoma, an increased intersaccadic interval and an increased number of eye movements necessary to locate the target. Some of these characteristics, such as the increased intersaccadic interval, were also observed in the simulation group, which indicate that these characteristics are related to the peripheral viewing itself. We suggest that the combination of the central scotoma and peripheral viewing can explain the altered search behavior and no behavioral evidence was found for a possible reorganization of the visual system associated with the use of a PRL. Thus the switch from a fovea-based to a PRL-based reference frame impairs search efficiency.

Visual search with image modification in age-related macular degeneration

PURPOSE

AMD results in loss of central vision and a dependence on low-resolution peripheral vision. While many image enhancement techniques have been proposed, there is a lack of quantitative comparison of the effectiveness of enhancement. We developed a natural visual search task that uses patients' eye movements as a quantitative and functional measure of the efficacy of image modification.

METHODS

Eye movements of 17 patients (mean age = 77 years) with AMD were recorded while they searched for target objects in natural images. Eight different image modification methods were implemented and included manipulations of local image or edge contrast, color, and crowding. In a subsequent task, patients ranked their preference of the image modifications.

RESULTS

Within individual participants, there was no significant difference in search duration or accuracy across eight different image manipulations. When data were collapsed across all image modifications, a multivariate model identified six significant predictors for normalized search duration including scotoma size and acuity, as well as interactions among scotoma size, age, acuity, and contrast (P < 0.05). Additionally, an analysis of image statistics showed no correlation with search performance across all image modifications. Rank ordering of enhancement methods based on participants' preference revealed a trend that participants preferred the least modified images (P < 0.05).

CONCLUSIONS

There was no quantitative effect of image modification on search performance. A better understanding of low- and high-level components of visual search in natural scenes is necessary to improve future attempts at image enhancement for low vision patients. Different search tasks may require alternative image modifications to improve patient functioning and performance.

Visual search performance of patients with vision impairment: effect of JPEG image enhancement

PURPOSE

To measure natural image search performance in patients with central vision impairment. To evaluate the performance effect for a JPEG based image enhancement technique using the visual search task.

METHODS

One hundred and fifty JPEG images were presented on a touch screen monitor in either an enhanced or original version to 19 patients (visual acuity 0.4-1.2 logMAR, 6/15 to 6/90, 20/50 to 20/300) and seven normally sighted controls (visual acuity -0.12 to 0.1 logMAR, 6/4.5 to 6/7.5, 20/15 to 20/25). Each image fell into one of three categories: faces, indoors, and collections. The enhancement was realized by moderately boosting a mid-range spatial frequency band in the discrete cosine transform (DCT) coefficients of the image luminance component. Participants pointed to an object in a picture that matched a given target displayed at the upper-left corner of the monitor. Search performance was quantified by the percentage of correct responses, the median search time of correct responses, and an 'integrated performance' measure - the area under the curve of cumulative correct response rate over search time.

RESULTS

Patients were able to perform the search tasks but their performance was substantially worse than the controls. Search performances for the three image categories were significantly different (p <= 0.001) for all the participants, with searching for faces being the most difficult. When search time and correct response were analyzed separately, the effect of enhancement led to increase in one measure but decrease in another for many patients. Using the integrated performance, it was found that search performance declined with decrease in acuity (p = 0.005). An improvement with enhancement was found mainly for the patients whose acuity ranged from 0.4 to 0.8 logMAR (6/15 to 6/38, 20/50 to 20/125). Enhancement conferred a small but significant improvement in integrated performance for indoor and collection images (p = 0.025) in the patients.

CONCLUSION

Search performance for natural images can be measured in patients with impaired vision to evaluate the effect of image enhancement. Patients with moderate vision loss might benefit from the moderate level of enhancement used here.

Visual search training in subjects with severe to profound low vision

Effects of practicing on feature search tasks (a 2 degrees square target amid 1 degrees square distracters) for 5 days were compared between 45 visually impaired (VI) subjects with severe to profound low vision and 23 age-matched normal controls (NV). Search accuracy and speed improved in both groups. VI subjects had larger training gains than NV subjects, but their proportional gains were similar to that of NV subjects. There were no significant differences in training effect at different set sizes in both groups. Search performance on a 40 degrees field improved more than that on a 10 degrees or 20 degrees field in VI subjects, but not in NV subjects. No significant change was found between day 5 and 1-month follow-up. The fact that feature search training is equally efficient in VI and NV subjects encourages development of general purpose perceptual training protocols for low vision rehabilitation.

Relationships between feature search and mobility performance in persons with severe visual impairment

PURPOSE

Because visual search requires both the ability to discriminate visual features and the ability to process information in a large field of view, the association between feature search and mobility of visually impaired (VI) subjects was studied.

METHODS

Forty-four subjects with severe visual impairment participated in the study. Feature search performance (2 x 2 deg square target amid 1 x 1 deg square distracters) was measured for 8- to 16- and 32-item set-sizes on 10 x 10, 20 x 20, and 40 x 40 deg fields. Mobility was evaluated on indoor high-density obstacle courses under photopic and mesopic illumination.

RESULTS

In feature search, VI subjects were slower and made more errors than normal subjects, but they searched in a parallel fashion. On the mobility task, VI subjects walked slower and made more obstacle contacts than age-matched normal controls. In VI subjects, performances on feature search and mobility tasks were significantly associated, with 37.5% to 66.9 of variations in the mobility measurements being accounted for by visual search speed.

CONCLUSIONS

Feature search reaction time can be a good predictor of VI patients' mobility.