Touch-screen technology for the dynamic display of -2D spatial information without vision: promise and progress

Many developers wish to capitalize on touch-screen technology for developing aids for the blind, particularly by incorporating vibrotactile stimulation to convey patterns on their surfaces, which otherwise are featureless. Our belief is that they will need to take into account basic research on haptic perception in designing these graphics interfaces. We point out constraints and limitations in haptic processing that affect the use of these devices. We also suggest ways to use sound to augment basic information from touch, and we include evaluation data from users of a touch-screen device with vibrotactile and auditory feedback that we have been developing, called a vibro-audio interface.

The effect of extended sensory range via the EyeCane sensory substitution device on the characteristics of visionless virtual navigation

Mobility training programs for helping the blind navigate through unknown places with a White-Cane significantly improve their mobility. However, what is the effect of new assistive technologies, offering more information to the blind user, on the underlying premises of these programs such as navigation patterns? We developed the virtual-EyeCane, a minimalistic sensory substitution device translating single-point-distance into auditory cues identical to the EyeCane's in the real world. We compared performance in virtual environments when using the virtual-EyeCane, a virtual-White-Cane, no device and visual navigation. We show that the characteristics of virtual-EyeCane navigation differ from navigation with a virtual-White-Cane or no device, and that virtual-EyeCane users complete more levels successfully, taking shorter paths and with less collisions than these groups, and we demonstrate the relative similarity of virtual-EyeCane and visual navigation patterns. This suggests that additional distance information indeed changes navigation patterns from virtual-White-Cane use, and brings them closer to visual navigation.

Is there a future for sensory substitution outside academic laboratories?

Sensory substitution devices (SSDs) have been developed with the ultimate purpose of supporting sensory deprived individuals in their daily activities. However, more than forty years after their first appearance in the scientific literature, SSDs still remain more common in research laboratories than in the daily life of people with sensory deprivation. Here, we seek to identify the reasons behind the limited diffusion of SSDs among the blind community by discussing the ergonomic, neurocognitive and psychosocial issues potentially associated with the use of these systems. We stress that these issues should be considered together when developing future devices or improving existing ones. We provide some examples of how to achieve this by adopting a multidisciplinary and participatory approach. These efforts would contribute not solely to address fundamental theoretical research questions, but also to better understand the everyday needs of blind people and eventually promote the use of SSDs outside laboratories.

Visual objects in the auditory system in sensory substitution: how much information do we need?

Sensory substitution devices such as The vOICe convert visual imagery into auditory soundscapes and can provide a basic 'visual' percept to those with visual impairment. However, it is not known whether technical or perceptual limits dominate the practical efficacy of such systems. By manipulating the resolution of sonified images and asking naïve sighted participants to identify visual objects through a six-alternative forced-choice procedure (6AFC) we demonstrate a 'ceiling effect' at 8 x 8 pixels, in both visual and tactile conditions, that is well below the theoretical limits of the technology. We discuss our results in the context of auditory neural limits on the representation of 'auditory' objects in a cortical hierarchy and how perceptual training may be used to circumvent these limitations.

Intra-session test-retest reliability of magnitude and structure of center of pressure from the Nintendo Wii Balance Board™ for a visually impaired and normally sighted population

Individuals with visual impairment (VI) have irreparable damage to one of the input streams contributing to postural stability. Here, we evaluated the intra-session test-retest reliability of the Wii Balance Board (WBB) for measuring Center of Pressure (COP) magnitude and structure, i.e. approximate entropy (ApEn) in fourteen legally blind participants and 21 participants with corrected-to-normal vision. Participants completed a validated balance protocol which included four sensory conditions: double-leg standing on a firm surface with eyes open (EO-firm); a firm surface with eyes closed (EC-firm); a foam surface with EO (EO-foam); and a foam surface with EC (EC-foam). Participants performed the full balance protocol twice during the session, separated by a period of 15min, to determine the intraclass correlation coefficient (ICC). Absolute reliability was determined by the standard error of measurement (SEM). The minimal difference (MD) was estimated to determine clinical significance for future studies. COP measures were derived from data sent by the WBB to a laptop via Bluetooth. COP scores increased with the difficulty of sensory condition indicating WBB sensitivity (all p<0.01). ICCs in the VI group ranged from 0.73 to 0.95, indicating high to very high correlations, and the normal group showed moderate to very high ICCs (0.62-0.94). The SEM was comparable between groups regardless of between-subject variability. The reliability of the WBB makes it practical to screen for balance impairment among VI persons.

Seeking a reading machine for the blind and discovering the speech code

A machine that can read printed material to the blind became a priority at the end of World War II with the appointment of a U.S. Government committee to instigate research on sensory aids to improve the lot of blinded veterans. The committee chose Haskins Laboratories to lead a multisite research program. Initially, Haskins researchers overestimated the capacities of users to learn an acoustic code based on the letters of a text, resulting in unsuitable designs. Progress was slow because the researchers clung to a mistaken view that speech is a sound alphabet and because of persisting gaps in man-machine technology. The tortuous route to a practical reading machine transformed the scientific understanding of speech perception and reading at Haskins Labs and elsewhere, leading to novel lines of basic research and new technologies. Research at Haskins Laboratories made valuable contributions in clarifying the physical basis of speech. Researchers recognized that coarticulatory overlap eliminated the possibility of alphabet-like discrete acoustic segments in speech. This work advanced the study of speech perception and contributed to our understanding of the relation of speech perception to production. Basic findings on speech enabled the development of speech synthesis, part science and part technology, essential for development of a reading machine, which has found many applications. Findings on the nature of speech further stimulated a new understanding of word recognition in reading across languages and scripts and contributed to our understanding of reading development and reading disabilities.

PERCEPT-II: smartphone based indoor navigation system for the blind

In this paper we introduce PERCEPT-II, a low cost and user friendly indoor navigation system for blind and visually impaired users. Using an Android Smartphone that runs PERCEPT-II application with accessibility features, the blind user obtains navigation instructions to the chosen destination when touching specific landmarks tagged with Near Field Communication tags. The system was deployed and tested in a large building at the University of Massachusetts at Amherst.

Proposal for SVG2DOT: - An Interoperable Tactile Graphics Creation System Using SVG outputs from Inkscape

For this study, I developed a software that converts SVG outputs from Inkscape to tactile graphics. Inkscape is easy to learn because it can be manipulated like any other typical drawing application and has a layer function which is enables robust support in the creation of tactile graphics In a simple evaluation, it is confirmed that the method of drawing a master in Inkscape and converting it with SVG2DOT was nearly as efficient as using a conventional tactile graphics drawing application.

Real-time Pedestrian Crossing Recognition for Assistive Outdoor Navigation

Navigation in urban environments can be difficult for people who are blind or visually impaired. In this project, we present a system and algorithms for recognizing pedestrian crossings in outdoor environments. Our goal is to provide navigation cues for crossing the street and reaching an island or sidewalk safely. Using a state-of-the-art Multisense S7S sensor, we collected 3D pointcloud data for real-time detection of pedestrian crossing and generation of directional guidance. We demonstrate improvements to a baseline, monocular-camera-based system by integrating 3D spatial prior information extracted from the pointcloud. Our system's parameters can be set to the actual dimensions of real-world settings, which enables robustness of occlusion and perspective transformation. The system works especially well in non-occlusion situations, and is reasonably accurate under different kind of conditions. As well, our large dataset of pedestrian crossings, organized by different types and situations of pedestrian crossings in order to reflect real-word environments, is publicly available in a commonly used format (ROS bagfiles) for further research.

BrailleEasy: One-handed Braille Keyboard for Smartphones

The evolution of mobile technology is moving at a very fast pace. Smartphones are currently considered a primary communication platform where people exchange voice calls, text messages and emails. The human-smartphone interaction, however, is generally optimized for sighted people through the use of visual cues on the touchscreen, e.g., typing text by tapping on a visual keyboard. Unfortunately, this interaction scheme renders smartphone technology largely inaccessible to visually impaired people as it results in slow typing and higher error rates. Apple and some third party applications provide solutions specific to blind people which enables them to use Braille on smartphones. These applications usually require both hands for typing. However, Brailling with both hands while holding the phone is not very comfortable. Furthermore, two-handed Brailling is not possible on smartwatches, which will be used more pervasively in the future. Therefore, we develop a platform for one-handed Brailing consisting of a custom keyboard called BrailleEasy to input Arabic or English Braille codes within any application, and a BrailleTutor application for practicing. Our platform currently supports Braille grade 1, and will be extended to support contractions, spelling correction, and more languages. Preliminary analysis of user studies for blind participants showed that after less than two hours of practice, participants were able to type significantly faster with the BrailleEasy keyboard than with the standard QWERTY keyboard.

Acquisition of Visual Perception in Blind Adults Using the BrainPort Artificial Vision Device

OBJECTIVE

We sought to determine whether intensive low vision rehabilitation would confer any functional improvement in a sample of blind adults using the BrainPort artificial vision device.

METHOD

Eighteen adults ages 28-69 yr (n=10 men and n=8 women) who had light perception only or worse vision bilaterally spent up to 6 hr per day for 1 wk undergoing structured rehabilitation interventions. The functional outcomes of object identification and word recognition were tested at baseline and after rehabilitation training.

RESULTS

At baseline, participants were unable to complete the two functional assessments. After participation in the 1-wk training protocol, participants were able to use the BrainPort device to complete the two tasks with moderate success.

CONCLUSION

Without training, participants were not able to perform above chance level using the BrainPort device. As artificial vision technologies become available, occupational therapy practitioners can play a key role in clients' success or failure in using these devices.

Observing the use of tactile schedules

This article explores the use of tactile schedules in a heterogeneous sample of children with congenital blindness and varying degrees of additional disabilities. Basic conditions for the use of tactile schedules are proposed and discussed. Child behaviour indicative of some particular functions that can be attained with the use of tactile schedules, such as sense of agency and increased predictability and communication, is described and discussed. The role of the teacher is emphasized throughout the article.

Perception of tactile graphics: embossings versus cutouts

Graphical information, such as illustrations, graphs, and diagrams, are an essential complement to text for conveying knowledge about the world. Although graphics can be communicated well via the visual modality, conveying this information via touch has proven to be challenging. The lack of easily comprehensible tactile graphics poses a problem for the blind. In this paper, we advance a hypothesis for the limited effectiveness of tactile graphics. The hypothesis contends that conventional graphics that rely upon embossings on two-dimensional surfaces do not allow the deployment of tactile exploratory procedures that are crucial for assessing global shape. Besides potentially accounting for some of the shortcomings of current approaches, this hypothesis also serves a prescriptive purpose by suggesting a different strategy for conveying graphical information via touch, one based on cutouts. We describe experiments demonstrating the greater effectiveness of this approach for conveying shape and identity information. These results hold the potential for creating more comprehensible tactile drawings for the visually impaired while also providing insights into shape estimation processes in the tactile modality.

Results of late surgical intervention in children with early-onset bilateral cataracts

BACKGROUND

Cataracts are a major cause of childhood blindness globally. Although surgically treatable, it is unclear whether children would benefit from such interventions beyond the first few years of life, which are believed to constitute 'critical' periods for visual development.

AIMS

To study visual acuity outcomes after late treatment of early-onset cataracts and also to determine whether there are longitudinal changes in postoperative acuity.

METHODS

We identified 53 children with dense cataracts with an onset within the first half-year after birth through a survey of over 20,000 rural children in India. All had accompanying nystagmus and were older than 8 years of age at the time of treatment. They underwent bilateral cataract surgery and intraocular lens implantation. We then assessed their best-corrected visual acuity 6 weeks and 6 months after surgery.

RESULTS

48 children from the pool of 53 showed improvement in their visual acuity after surgery. Our longitudinal assessments demonstrated further improvements in visual acuity for the majority of these children proceeding from the 6-week to 6-month assessment. Interestingly, older children in our subject pool did not differ significantly from the younger ones in the extent of improvement they exhibit.

CONCLUSIONS AND RELEVANCE

Our results demonstrate that not only can significant vision be acquired until late in childhood, but that neural processes underlying even basic aspects of vision like resolution acuity remain malleable until at least adolescence. These data argue for the provision of cataract treatment to all children, irrespective of their age.

Performance of photovoltaic arrays in-vivo and characteristics of prosthetic vision in animals with retinal degeneration

Loss of photoreceptors during retinal degeneration leads to blindness, but information can be reintroduced into the visual system using electrical stimulation of the remaining retinal neurons. Subretinal photovoltaic arrays convert pulsed illumination into pulsed electric current to stimulate the inner retinal neurons. Since required irradiance exceeds the natural luminance levels, an invisible near-infrared (915 nm) light is used to avoid photophobic effects. We characterized the thresholds and dynamic range of cortical responses to prosthetic stimulation with arrays of various pixel sizes and with different number of photodiodes. Stimulation thresholds for devices with 140 μm pixels were approximately half those of 70 μm pixels, and with both pixel sizes, thresholds were lower with 2 diodes than with 3 diodes per pixel. In all cases these thresholds were more than two orders of magnitude below the ocular safety limit. At high stimulation frequencies (>20 Hz), the cortical response exhibited flicker fusion. Over one order of magnitude of dynamic range could be achieved by varying either pulse duration or irradiance. However, contrast sensitivity was very limited. Cortical responses could be detected even with only a few illuminated pixels. Finally, we demonstrate that recording of the corneal electric potential in response to patterned illumination of the subretinal arrays allows monitoring the current produced by each pixel, and thereby assessing the changes in the implant performance over time.

Special home adaptation grants for members of the Armed Forces and veterans with certain vision impairment. Final rule

The Department of Veterans Affairs (VA) is issuing a final rule to amend its adjudication regulations regarding special home adaptation grants for members of the Armed Forces and veterans with certain vision impairment. This regulatory amendment is necessary to conform the regulations to changes mandated in the Honoring America's Veterans and Caring for Camp Lejeune Families Act of 2012.

Navigation within buildings: novel movement detection algorithms supporting people with visual impairments

This study aimed at finding simple algorithms to identify three different movements registered by accelerometer and to detect differences in the acceleration signals of people with and without visual impairments. The Tactile Acoustical Navigation and Information Assistant (TANIA) is construed to provide persons suffering from visual impairments support for an independent navigation indoors and outdoors. Attaining this goal, TANIA uses vertical acceleration signal extrema to assess its user's walking distance. This study investigated first the sit-to-stand movement, stumbling and walking up- and down stairs of 25 subjects with visual impairments using TANIA sensor system. The objective was to improve the user's movement detection using sensors to get valid and reliable data. In a second step of the study it was investigated if there is a difference between the above-mentioned movements in people with or without visual impairments (n=10). The acceleration signals of the subjects were compared. Three simple algorithms were found, which are able to separate the movement signals based on accelerometers of the respective daily movements. The second step analysis revealed a detectable difference in the second phase of stumbling (p=.034), where the subjects had to get back into walking forward. No differences in the other acceleration signals were found.

Microswitch-aided programs to support physical exercise or adequate ambulation in persons with multiple disabilities

Three microswitch-aided programs were assessed in three single-case studies to enhance physical exercise or ambulation in participants with multiple disabilities. Study I was aimed at helping a woman who tended to have the head bending forward and the arms down to exercise a combination of appropriate head and arms movements. Study II was aimed at promoting ambulation continuity with a man who tended to have ambulation breaks. Study III was aimed at promoting ambulation with appropriate foot position in a girl who usually showed toe walking. The experimental designs of the studies consisted of a multiple probe across responses (Study I), an ABAB sequence (Study II), and an ABABB(1) sequence (Study III). The last phase of each study was followed by a post-intervention check. The microswitches monitored the target responses selected for the participants and triggered a computer system to provide preferred stimuli contingent on those responses during the intervention phases of the studies. Data showed that the programs were effective with each of the participants who learned to exercise head and arms movements, increased ambulation continuity, and acquired high levels of appropriate foot position during ambulation, respectively. The positive performance levels were retained during the post-intervention checks. The discussion focused on (a) the potential of technology-aided programs for persons with multiple disabilities and (b) the need of replication studies to extend the evidence available in the area.

Stereo camera based virtual cane system with identifiable distance tactile feedback for the blind

In this paper, we propose a new haptic-assisted virtual cane system operated by a simple finger pointing gesture. The system is developed by two stages: development of visual information delivery assistant (VIDA) with a stereo camera and adding a tactile feedback interface with dual actuators for guidance and distance feedbacks. In the first stage, user's pointing finger is automatically detected using color and disparity data from stereo images and then a 3D pointing direction of the finger is estimated with its geometric and textural features. Finally, any object within the estimated pointing trajectory in 3D space is detected and the distance is then estimated in real time. For the second stage, identifiable tactile signals are designed through a series of identification experiments, and an identifiable tactile feedback interface is developed and integrated into the VIDA system. Our approach differs in that navigation guidance is provided by a simple finger pointing gesture and tactile distance feedbacks are perfectly identifiable to the blind.

People with multiple disabilities learn to engage in occupation and work activities with the support of technology-aided programs

These two studies were aimed at assessing technology-aided programs to help persons with multiple disabilities engage in basic occupation or work activities. Specifically, Study I focused on teaching two participants (an adolescent and an adult) with low vision or total blindness, severe/profound intellectual disabilities, and minimal object interaction to engage in constructive object-manipulation responses. The technology monitored their responses and followed them with brief stimulation periods automatically. Study II focused on teaching three adults with deafness, severe visual impairment, and profound intellectual disabilities to perform a complex activity, that is, to assemble a five-component water pipe. The technology regulated (a) light cues to guide the participants through the workstations containing single pipe components and the carton for completed pipes and (b) stimulation events. The results of both studies were positive. The participants of Study I showed consistent and independent engagement in object-manipulation responses. The participants of Study II showed consistent and independent pipe assembling performance. General implications of the two programs and the related technology packages for intervention with persons with multiple disabilities are discussed.

Design, development, and clinical evaluation of the electronic mobility cane for vision rehabilitation

This paper proposes a new electronic mobility cane (EMC) for providing obstacle detection and way-finding assistance to the visually impaired people. The main feature of this cane is that it constructs the logical map of the surrounding environment to deduce the priority information. It provides a simplified representation of the surrounding environment without causing any information overload. It conveys this priority information to the subject by using intuitive vibration, audio or voice feedback. The other novel features of the EMC are staircase detection and nonformal distance scaling scheme. It also provides information about the floor status. It consists of a low power embedded system with ultrasonic sensors and safety indicators. The EMC was subjected to series of clinical evaluations in order to verify its design and to assess its ability to assist the subjects in their daily-life mobility. Clinical evaluations were performed with 16 totally blind and four low vision subjects. All subjects walked controlled and the real-world test environments with the EMC and the traditional white cane. The evaluation results and significant scores of subjective measurements have shown the usefulness of the EMC in vision rehabilitation services.

Enhancing activity by means of tactile symbols: a study of a heterogeneous group of pupils with congenital blindness, intellectual disability and autism spectrum disorder

This study investigates responses to verbal versus tactile requests in children with congenital blindness, intellectual disability and autism spectrum disorder (ASD). Observation was conducted on two occasions. At T1, requests were given verbally, and at T2, tactile requests were given. All pupils perceived tactile symbols to be explicit requests to perform the act referred to by the symbols. The children seldom or never followed verbal requests. All children followed more tactile than verbal requests. Individual differences in verbal skills, motivations and the complexity of the activity are discussed. The availability of tactile symbols for individuals with congenital blindness, intellectual disability and ASD seems to increase their level of activity and their participation in school.

Vision rehabilitation in the case of blindness

This article examines the various vision rehabilitation procedures that are available for early and late blindness. Depending on the pathology involved, several vision rehabilitation procedures exist, or are in development. Visual aids are available for low vision individuals, as are sensory aids for blind persons. Most noninvasive sensory substitution prostheses as well as implanted visual prostheses in development are reviewed. Issues dealing with vision rehabilitation are also discussed, such as problems of biocompatibility, electrical safety, psychosocial aspects, and ethics. Basic studies devoted to vision rehabilitation such as simulation in mathematical models and simulation of artificial vision are also presented. Finally, the importance of accurate rehabilitation assessment is addressed, and tentative market figures are given.

Artificial human vision

Can vision be restored to the blind? As early as 1929 it was discovered that stimulating the visual cortex of an individual led to the perception of spots of light, known as phosphenes [1] . The aim of artificial human vision systems is to attempt to utilize the perception of phosphenes to provide a useful substitute for normal vision. Currently, four locations for electrical stimulation are being investigated; behind the retina (subretinal), in front of the retina (epiretinal), the optic nerve and the visual cortex (using intra- and surface electrodes). This review discusses artificial human vision technology and requirements, and reviews the current development projects.

[PASSIVE TACTILE STIMULATION AS A WAY OF SPATIAL RECOGNITION IN SUBJECTS WITH SEVERE VISUAL IMPAIRMENT OR BLINDNESS]

Regular tactile passive stimulation increases the lasting activation of a greater number of synaptic connections (neuroplasticity) in posterior cortical areas in subjects with blidness with duration and permanence of the neuroplasticity process. Assuming that the lack of stimulation logically involves a deficit due to the cerebral hypofunctionality; in the case of blind subjects such deficit should be more pronounced in the occipital lobe responsible of human vision. We found, after long period of táctile stimulation, that the occipital lobe was activated while having at the same time a visual "qualia" subjective sensation using transcranial magnetic stimulation occipital lobe inhibition decreases visual response.