Improving spatial working memory in blind and sighted youngsters using programmable tactile displays

A Technology Aid to Help People with Blindness and Moderate Intellectual Disability Retrieve Common Objects from Storage Units: A Proof-of-Concept Study

BACKGROUND

People with blindness and intellectual disability can have problems locating, identifying, and retrieving objects needed for daily activities (e.g., clothes and food items) from familiar storage contexts, such as cupboards and cabinets.

OBJECTIVE

This preliminary study assessed a technological system designed to help three people with those problems improve their performance.

METHODS

The technological system, which involved the use of tags with radio frequency identification codes, a smartphone, and a tag reader, aimed to guide the participants in searching and retrieving objects from three different storage units. In practice, the system provided different feedbacks depending on whether the participants were searching (a) in a wrong storage unit, (b) in a wrong shelf/drawer of the right storage unit, or (c) in the right shelf/drawer of the right storage unit.

RESULTS

All participants were successful in retrieving objects correctly with the technological system. The results also showed that (a) the participants preferred using the system over a control strategy, (b) were able to switch on and off the system independently, and (b) staff rated the system positively.

CONCLUSIONS

These preliminary findings suggest that the system might be a useful support tool for people with blindness and intellectual disability.

Blind people can actively manipulate virtual objects with a novel tactile device

Frequently in rehabilitation, visually impaired persons are passive agents of exercises with fixed environmental constraints. In fact, a printed tactile map, i.e. a particular picture with a specific spatial arrangement, can usually not be edited. Interaction with map content, instead, facilitates the learning of spatial skills because it exploits mental imagery, manipulation and strategic planning simultaneously. However, it has rarely been applied to maps, mainly because of technological limitations. This study aims to understand if visually impaired people can autonomously build objects that are completely virtual. Specifically, we investigated if a group of twelve blind persons, with a wide age range, could exploit mental imagery to interact with virtual content and actively manipulate it by means of a haptic device. The device is mouse-shaped and designed to jointly perceive, with one finger only, local tactile height and inclination cues of arbitrary scalar fields. Spatial information can be mentally constructed by integrating local tactile cues, given by the device, with global proprioceptive cues, given by hand and arm motion. The experiment consisted of a bi-manual task, in which one hand explored some basic virtual objects and the other hand acted on a keyboard to change the position of one object in real-time. The goal was to merge basic objects into more complex objects, like a puzzle. The experiment spanned different resolutions of the tactile information. We measured task accuracy, efficiency, usability and execution time. The average accuracy in solving the puzzle was 90.5%. Importantly, accuracy was linearly predicted by efficiency, measured as the number of moves needed to solve the task. Subjective parameters linked to usability and spatial resolutions did not predict accuracy; gender modulated the execution time, with men being faster than women. Overall, we show that building purely virtual tactile objects is possible in absence of vision and that the process is measurable and achievable in partial autonomy. Introducing virtual tactile graphics in rehabilitation protocols could facilitate the stimulation of mental imagery, a basic element for the ability to orient in space. The behavioural variable introduced in the current study can be calculated after each trial and therefore could be used to automatically measure and tailor protocols to specific user needs. In perspective, our experimental setup can inspire remote rehabilitation scenarios for visually impaired people.

Learning and navigating digitally rendered haptic spatial layouts

Learning spatial layouts and navigating through them rely not simply on sight but rather on multisensory processes, including touch. Digital haptics based on ultrasounds are effective for creating and manipulating mental images of individual objects in sighted and visually impaired participants. Here, we tested if this extends to scenes and navigation within them. Using only tactile stimuli conveyed via ultrasonic feedback on a digital touchscreen (i.e., a digital interactive map), 25 sighted, blindfolded participants first learned the basic layout of an apartment based on digital haptics only and then one of two trajectories through it. While still blindfolded, participants successfully reconstructed the haptically learned 2D spaces and navigated these spaces. Digital haptics were thus an effective means to learn and translate, on the one hand, 2D images into 3D reconstructions of layouts and, on the other hand, navigate actions within real spaces. Digital haptics based on ultrasounds represent an alternative learning tool for complex scenes as well as for successful navigation in previously unfamiliar layouts, which can likely be further applied in the rehabilitation of spatial functions and mitigation of visual impairments.

Early blindness modulates haptic object recognition

Haptic object recognition is usually an efficient process although slower and less accurate than its visual counterpart. The early loss of vision imposes a greater reliance on haptic perception for recognition compared to the sighted. Therefore, we may expect that congenitally blind persons could recognize objects through touch more quickly and accurately than late blind or sighted people. However, the literature provided mixed results. Furthermore, most of the studies on haptic object recognition focused on performance, devoting little attention to the exploration procedures that conducted to that performance. In this study, we used iCube, an instrumented cube recording its orientation in space as well as the location of the points of contact on its faces. Three groups of congenitally blind, late blind and age and gender-matched blindfolded sighted participants were asked to explore the cube faces where little pins were positioned in varying number. Participants were required to explore the cube twice, reporting whether the cube was the same or it differed in pins disposition. Results showed that recognition accuracy was not modulated by the level of visual ability. However, congenitally blind touched more cells simultaneously while exploring the faces and changed more the pattern of touched cells from one recording sample to the next than late blind and sighted. Furthermore, the number of simultaneously touched cells negatively correlated with exploration duration. These findings indicate that early blindness shapes haptic exploration of objects that can be held in hands.

Mental Rotation Skill Shapes Haptic Exploration Strategies

Haptic exploration strategies have been traditionally studied focusing on hand movements and neglecting how objects are moved in space. However, in daily life situations touch and movement cannot be disentangled. Furthermore, the relation between object manipulation as well as performance in haptic tasks and spatial skill is still little understood. In this study, we used iCube, a sensorized cube recording its orientation in space as well as the location of the points of contact on its faces. Participants had to explore the cube faces where little pins were positioned in varying number and count the number of pins on the faces with either even or odd number of pins. At the end of this task, they also completed a standard visual mental rotation test (MRT). Results showed that higher MRT scores were associated with better performance in the task with iCube both in term of accuracy and exploration speed and exploration strategies associated with better performance were identified. High performers tended to rotate the cube so that the explored face had the same spatial orientation (i.e., they preferentially explored the upward face and rotated iCube to explore the next face in the same orientation). They also explored less often twice the same face and were faster and more systematic in moving from one face to the next. These findings indicate that iCube could be used to infer subjects' spatial skill in a more natural and unobtrusive fashion than with standard MRTs.

Standardized and Experimental Tools to Assess Spatial Cognition in Visually Impaired Children: A Mini-Review

The acquisition of spatial cognition is essential for both everyday functioning (e.g., navigation) and more specific goals (e.g., mathematics), therefore being able to assess and monitor spatial cognition from the first years of life would be essential to predict developmental outcomes and timely intervene whenever spatial development is compromised. Several shreds of evidence have indicated that spatial development can be compromised in the case of development with atypical sensory experience such as blindness. Despite the massive importance of spatial abilities for the development of psychomotor competencies across childhood, only a few standardized and experimental methods have been developed to assess them in visually impaired children. In this review, we will give a short overview of current formal (standardized) and informal (experimental) methods to assess spatial cognition in visually impaired children, demonstrating that very few validated tools have been proposed to date. The main contribution of this current work is to highlight the need of ad hoc studies to create and validate clinical measures to assess spatial cognition in visually impaired individuals and address potential future developments in this area of research.

Identification of Textured Tactile Pictures in Visually Impaired and Blindfolded Sighted Children

A high level of variability in the capacity of visually impaired children to accurately identify tactile images is reported in the literature, with on average rather low percentages of correct naming responses. However, most of these studies used raised-line drawings as stimuli to be explored and named. The present experiment investigated whether blind children of 3 to 8 years of age would demonstrate a satisfactory ability to name the elements making up tactile images when tested in an experimental setting similar to their natural reading conditions. Textured tactile images taken from genuine illustrated tactile books for young children were used, and the participants received information about the title of the book or listened the text that accompanied each picture before exploration, as it would occur in a natural reading setting. The results showed that their naming scores were indeed higher than previously reported at equivalent ages and did not differ from those of age-matched sighted children. These scores were positively impacted by haptic practice in blind children and correlated with the use of some specific exploratory procedures. The blind children benefited from information provided before exploration, as did their sighted counterparts. However, only in the former did the condition in which full information was provided influence the way the children organized their exploration. The haptic identification scores increased with age regardless of visual status, with the exploration times decreasing in the blind children, while the reverse trend was observed in the sighted children. These results are discussed at the light of the image-mediation model of haptics, suggesting that during the age period considered in the present experiment, blind children would progressively learn to process haptic information directly, thus leading to a decrease of exploration times, while sighted children would learn to translate haptic information into a visual image used to retrieve semantic information, involving an increase of their exploration times.

Enhancing general spatial skills of young visually impaired people with a programmable distance discrimination training: a case control study

BACKGROUND

The estimation of relative distance is a perceptual task used extensively in everyday life. This important skill suffers from biases that may be more pronounced when estimation is based on haptics. This is especially true for the blind and visually impaired, for which haptic estimation of distances is paramount but not systematically trained. We investigated whether a programmable tactile display, used autonomously, can improve distance discrimination ability in blind and severely visually impaired youngsters between 7 and 22 years-old.

METHODS

Training consisted of four weekly sessions in which participants were asked to haptically find, on the programmable tactile display, the pairs of squares which were separated by the shortest and longest distance in tactile images with multiple squares. A battery of haptic tests with raised-line drawings was administered before and after training, and scores were compared to those of a control group that did only the haptic battery, without doing the distance discrimination training on the tactile display.

RESULTS

Both blind and severely impaired youngsters became more accurate and faster at the task during training. In haptic battery results, blind and severely impaired youngsters who used the programmable display improved in three and two tests, respectively. In contrast, in the control groups, the blind control group improved in only one test, and the severely visually impaired in no tests.

CONCLUSIONS

Distance discrimination skills can be trained equally well in both blind and severely impaired participants. More importantly, autonomous training with the programmable tactile display had generalized effects beyond the trained task. Participants improved not only in the size discrimination test but also in memory span tests. Our study shows that tactile stimulation training that requires minimal human assistance can effectively improve generic spatial skills.