The Effect of Programmable Tactile Displays on Spatial Learning Skills in Children and Adolescents of Different Visual Disability

Learning and navigating digitally rendered haptic spatial layouts

Learning spatial layouts and navigating through them rely not simply on sight but rather on multisensory processes, including touch. Digital haptics based on ultrasounds are effective for creating and manipulating mental images of individual objects in sighted and visually impaired participants. Here, we tested if this extends to scenes and navigation within them. Using only tactile stimuli conveyed via ultrasonic feedback on a digital touchscreen (i.e., a digital interactive map), 25 sighted, blindfolded participants first learned the basic layout of an apartment based on digital haptics only and then one of two trajectories through it. While still blindfolded, participants successfully reconstructed the haptically learned 2D spaces and navigated these spaces. Digital haptics were thus an effective means to learn and translate, on the one hand, 2D images into 3D reconstructions of layouts and, on the other hand, navigate actions within real spaces. Digital haptics based on ultrasounds represent an alternative learning tool for complex scenes as well as for successful navigation in previously unfamiliar layouts, which can likely be further applied in the rehabilitation of spatial functions and mitigation of visual impairments.

Assistive technology-based solutions in learning mathematics for visually-impaired people: exploring issues, challenges and opportunities

In the absence of vision, visually impaired and blind people rely upon the tactile sense and hearing to obtain information about their surrounding environment. These senses cannot fully compensate for the absence of vision, so visually impaired and blind people experience difficulty with many tasks, including learning. This is particularly true of mathematical learning. Nowadays, technology provides many effective and affordable solutions to help visually impaired and blind people acquire mathematical skills. This paper is based upon a systematic review of technology-based mathematical learning solutions for visually impaired people and discusses the findings and objectives for technological improvements. It analyses the issues, challenges and limitations of existing techniques. We note that audio feedback, tactile displays, a supportive academic environment, digital textbooks and other forms of accessible math applications improve the quality of learning mathematics in visually impaired and blind people. Based on these findings, it is suggested that smartphone-based solutions could be more convenient and affordable than desktop/laptop-based solutions as a means to enhance mathematical learning. Additionally, future research directions are discussed, which may assist researchers to propose further solutions that will improve the quality of life for visually impaired and blind people.

Do our hands see what our eyes see? Investigating spatial and haptic abilities

Spatial abilities (SAs) are cognitive resources used to mentally manipulate representations of objects to solve problems. Haptic abilities (HAs) represent tactile interactions with real-world objects transforming somatic information into mental representations. Both are proposed to be factors in anatomy education, yet relationships between SAs and HAs remain unknown. The objective of the current study was to explore SA-HA interactions. A haptic ability test (HAT) was developed based on the mental rotations test (MRT) with three-dimensional (3D) objects. The HAT was undertaken in three sensory conditions: (1) sighted, (2) sighted with haptics, and (3) haptics. Participants (n = 22; 13 females, 9 males) completed the MRT and were categorized into high spatial abilities (HSAs) (n = 12, mean± standard deviation: 13.7 ± 3.0) and low spatial abilities (LSAs) (n = 10, 5.6 ± 2.0) based on score distributions about the overall mean. Each SA group's HAT scores were compared across the three sensory conditions. Spearman's correlation coefficients between MRT and HAT scores indicated a statistically significant correlation in sighted condition (r = 0.553, p = 0.015) but were not significant in the sighted with haptics (r = 0.0.078, p = 0.212) and haptics (r = 0.043, p = 0.279) conditions. These data suggest HAs appear unrelated to SAs. With haptic exploration, LSA HAT scores were compensated; comparing HSA with LSA: sighted with haptics [median (lower and upper quartiles): 12 (12,13) vs. 12 (11,13), p = 0.254], and haptics [12 (11,13) vs. 12 (10,12), p = 0.381] conditions. Migrations to online anatomy teaching may unwittingly remove important sensory modalities from the learner. Understanding learner behaviors and performance when haptic inputs are removed from the learning environment represents valuable insight informing future anatomy curriculum and resource development.

Sixteen Years since the Convention on the Rights of Persons with Disabilities: What Have We Learned since Then?

Persons with disabilities have historically been subjected to discrimination and exclusion, placing them in dangerous situations of social vulnerability. The Convention on the Rights of Persons with Disabilities, passed on the 13th of December 2006, was an important legislative landmark for valuing the rights of this population group. This study involved a bibliometric analysis of 1024 research articles published in Scopus on the social, workplace, educational, and financial inclusion of persons with disabilities since the Convention. The results show an increase in scientific production, and there is also a great deal of multi-disciplinarity, which has led to important breakthroughs for the all-encompassing inclusion of this section of the population. The most productive journals, authors, institutions, and countries, as well as the international cooperation networks, are presented here. The review concludes by setting out the main themes and trends in the research.

Co-designed mini-games for children with visual impairment: a pilot study on their usability

Digital games aimed at improving cognitive and/or motor-sensory skills need to be carefully designed to take into account the characteristics and needs of particular categories of users. Several novel mini-games explicitly aimed at children with visual impairment (VI) were co-designed by a multidisciplinary team which involved computer engineers and a therapy team from the Robert Hollman Foundation (Padova, Italy). These games are played by children moving within a large-scale interactive environment - i.e., a floor portion placed under a motion capture system capable of tracking one or more people - with the game linking the players movements to the audio and visual output to produce meaningful interactions. We report on a pilot study of the usability of the system involving 11 children with VI. The results allowed us to improve the system and to define a set of guidelines useful for designers and developers of similar systems.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s11042-022-13665-7.

Mental Rotation Skill Shapes Haptic Exploration Strategies

Haptic exploration strategies have been traditionally studied focusing on hand movements and neglecting how objects are moved in space. However, in daily life situations touch and movement cannot be disentangled. Furthermore, the relation between object manipulation as well as performance in haptic tasks and spatial skill is still little understood. In this study, we used iCube, a sensorized cube recording its orientation in space as well as the location of the points of contact on its faces. Participants had to explore the cube faces where little pins were positioned in varying number and count the number of pins on the faces with either even or odd number of pins. At the end of this task, they also completed a standard visual mental rotation test (MRT). Results showed that higher MRT scores were associated with better performance in the task with iCube both in term of accuracy and exploration speed and exploration strategies associated with better performance were identified. High performers tended to rotate the cube so that the explored face had the same spatial orientation (i.e., they preferentially explored the upward face and rotated iCube to explore the next face in the same orientation). They also explored less often twice the same face and were faster and more systematic in moving from one face to the next. These findings indicate that iCube could be used to infer subjects' spatial skill in a more natural and unobtrusive fashion than with standard MRTs.

Research on the Method of Displaying the Contour Features of Image to the Visually Impaired on the Touch Screen

Conveying image information to the blind or visually impaired (BVI) is an important means to improve their quality of life. The touch screen devices used daily are the potential carriers for BVI to perceive image information through touch. However, touch screen devices also have the disadvantages of limited computing power and lack of rich tactile experience. In order to help BVI to access images conveniently through the touch screen, we built an image contour display system based on vibrotactile feedback. In this paper, an image smoothing algorithm based on convolutional neural network that can run quickly on the touch screen device is first used to preprocess the image to improve the effect of contour extraction. Then, based on the haptic physiological characteristics of human beings, this paper proposes a method of using the improved MH-Pen to guide the BVI to perceive image contour on the touch screen. This paper introduces the extraction and expression methods of image contours in detail, and compares and analyzes the effects of the subjects' perception of image contours in two haptic display modes through two types of user experiments. The experimental results show that the image smoothing algorithm is useful and necessary to help obtain the main contour of the image and to ensure the real-time display of the contour, and the contour expression method based on the motion direction guidance helps the subjects recognize the contour of the image more effectively.

Integrating Tactile Feedback Technologies Into Home-Based Telerehabilitation: Opportunities and Challenges in Light of COVID-19 Pandemic

The COVID-19 pandemic has highlighted the need for advancing the development and implementation of novel means for home-based telerehabilitation in order to enable remote assessment and training for individuals with disabling conditions in need of therapy. While somatosensory input is essential for motor function, to date, most telerehabilitation therapies and technologies focus on assessing and training motor impairments, while the somatosensorial aspect is largely neglected. The integration of tactile devices into home-based rehabilitation practice has the potential to enhance the recovery of sensorimotor impairments and to promote functional gains through practice in an enriched environment with augmented tactile feedback and haptic interactions. In the current review, we outline the clinical approaches for stimulating somatosensation in home-based telerehabilitation and review the existing technologies for conveying mechanical tactile feedback (i.e., vibration, stretch, pressure, and mid-air stimulations). We focus on tactile feedback technologies that can be integrated into home-based practice due to their relatively low cost, compact size, and lightweight. The advantages and opportunities, as well as the long-term challenges and gaps with regards to implementing these technologies into home-based telerehabilitation, are discussed.

Assistive technology for visual impairment and trainers at schools for the blind in Delhi

The aim of the study was to assess the availability of assistive technology (AT) for visual impairment and trainers in schools for the blind in Delhi. A cross-sectional study was conducted in 22 of the 24 schools in Delhi. The headteacher of each school was asked about availability of 52 ATs divided into writing, reading, math, sciences, sports, mobility, and daily living, using a questionnaire. Information on availability of trainers was also collected. Of the 52 ATs, the most frequently available were Braille slate with stylus and abacus (>90% of schools), followed by Taylor frame, long cane and talking watch (80% to 90% of schools). Only 11 of 52 AT devices were available in 60% or more of the schools. Tactile-based ATs were more available than vision-based ATs. In the 22 schools, 63 trainers for reading & writing were available (80% of posts), 18 for sciences (59%), 25 for math (70%), and 11 for mobility (50% of posts). Except Braille slate and stylus, there is a huge shortage of AT in these schools. The educational needs and performance of students could be helped by developing and using a list of priority Assistive Products for example, the WHO AP list.

Standardized and Experimental Tools to Assess Spatial Cognition in Visually Impaired Children: A Mini-Review

The acquisition of spatial cognition is essential for both everyday functioning (e.g., navigation) and more specific goals (e.g., mathematics), therefore being able to assess and monitor spatial cognition from the first years of life would be essential to predict developmental outcomes and timely intervene whenever spatial development is compromised. Several shreds of evidence have indicated that spatial development can be compromised in the case of development with atypical sensory experience such as blindness. Despite the massive importance of spatial abilities for the development of psychomotor competencies across childhood, only a few standardized and experimental methods have been developed to assess them in visually impaired children. In this review, we will give a short overview of current formal (standardized) and informal (experimental) methods to assess spatial cognition in visually impaired children, demonstrating that very few validated tools have been proposed to date. The main contribution of this current work is to highlight the need of ad hoc studies to create and validate clinical measures to assess spatial cognition in visually impaired individuals and address potential future developments in this area of research.

An empirical evaluation of a graphics creation technique for blind and visually impaired individuals

The representation of pictorial data by people who are blind and sight impaired has gathered momentum with research and development; however, little research has focused on the use of a screen layout to provide people who are blind and sight impaired users with the spatial orientation to create and reuse graphics. This article contributes an approach to navigating on the screen, manipulating computer graphics, and user-defined images. The technique described in this article enables features such as zooming, grouping, and drawing by calling primitive and user-defined shapes. It enables blind people to engage in and experience drawing and art production on their own. The navigation technique gives an initiative sense of autonomy with compass directions, makes it easy to learn, efficient to manipulate shape with a the simple drawing language, and takes less time to complete with system support features. An empirical evaluation was conducted to validate the suitability of the SETUP09 technique and to evaluate the accuracy, and efficiency of the navigation and drawing techniques proposed. The drawing experiment results confirmed high accuracy (88%) and efficiency among blind and visually impaired (BVI) users.

Classification of Blind Users' Image Exploratory Behaviors Using Spiking Neural Networks

Individuals who are blind adopt multiple procedures to tactually explore images. Automatically recognizing and classifying users' exploration behaviors is the first step towards the development of an intelligent system that could assist users to explore images more efficiently. In this paper, a computational framework was developed to classify different procedures used by blind users during image exploration. Translation-, rotation- and scale-invariant features were extracted from the trajectories of users' movements. These features were divided as numerical and logical features and were fed into neural networks. More specifically, we trained spiking neural networks (SNNs) to further encode the numerical features as model strings. The proposed framework employed a distance-based classification scheme to determine the final class/label of the exploratory procedures. Dempster-Shafter Theory (DST) was applied to integrate the distances obtained from all the features. Through the experiments of different dynamics of spiking neurons, the proposed framework achieved a good performance with 95.89% classification accuracy. It is extremely effective in encoding and classifying spatio-temporal data, as compared to Dynamic Time Warping and Hidden Markov Model with 61.30% and 28.70% accuracy. The proposed framework serves as the fundamental block for the development of intelligent interfaces, enhancing the image exploration experience for the blind.

Digital learning in mathematics for students with severe visual impairment: A systematic review

Mathematical education is currently undergoing significant changes that are driven by technology and digital-based learning. Students with visual impairments (VI) may face different challenges in mathematical education due to a lack of accessible materials designed to support the development of conceptual understanding in mathematics. The aim of the study was to summarize current evidence-based knowledge about e-learning in mathematics among students with severe VI. A systematic review was conducted of articles published from January 2000 to November 2017. A total of 13 publications met the inclusion criteria, of which 12 reported studies with an intervention or an experimental design and one had a cross-sectional design. The number of students with VI varied from three to 16 ( Mage = 19 years). Four publications reported either ophthalmic diagnoses or World Health Organization’s (WHO) definition of vision loss. The mathematical training was reported as lasting from one session to 18 weeks of training. Eight papers reported the use of audio-based applications as learning aids. The authors conclude that interactive e-learning with audio and tactile learning programmes may be a useful resource for students with VI to enhance their mathematical skills. However, there is lack of evidence for how digital technologies improve inclusion potentials and learning in mathematics for students with VI, and hence, there is a need for additional research and more reflection on the subject.

Enhancing general spatial skills of young visually impaired people with a programmable distance discrimination training: a case control study

BACKGROUND

The estimation of relative distance is a perceptual task used extensively in everyday life. This important skill suffers from biases that may be more pronounced when estimation is based on haptics. This is especially true for the blind and visually impaired, for which haptic estimation of distances is paramount but not systematically trained. We investigated whether a programmable tactile display, used autonomously, can improve distance discrimination ability in blind and severely visually impaired youngsters between 7 and 22 years-old.

METHODS

Training consisted of four weekly sessions in which participants were asked to haptically find, on the programmable tactile display, the pairs of squares which were separated by the shortest and longest distance in tactile images with multiple squares. A battery of haptic tests with raised-line drawings was administered before and after training, and scores were compared to those of a control group that did only the haptic battery, without doing the distance discrimination training on the tactile display.

RESULTS

Both blind and severely impaired youngsters became more accurate and faster at the task during training. In haptic battery results, blind and severely impaired youngsters who used the programmable display improved in three and two tests, respectively. In contrast, in the control groups, the blind control group improved in only one test, and the severely visually impaired in no tests.

CONCLUSIONS

Distance discrimination skills can be trained equally well in both blind and severely impaired participants. More importantly, autonomous training with the programmable tactile display had generalized effects beyond the trained task. Participants improved not only in the size discrimination test but also in memory span tests. Our study shows that tactile stimulation training that requires minimal human assistance can effectively improve generic spatial skills.

An evaluation of tactile symbols in public environment for the visually impaired

This study evaluated the identification performance of a set of tactile symbols used in public environments for the visually impaired. A questionnaire survey was carried out to investigate the public environment needs from 60 visually impaired associations. A two-stage experiment with a matching test was then conducted to explore the identification efficiency of graphic tactile public information symbols. Eighty-one students were recruited as participants from a school for the visually impaired. The survey results show that fourteen public buildings were frequently visited and ten architectural elements were mostly needed by the visually impaired. The experimental results showed the correct response of graphic symbols tested in both two-stage experiments could meet the identification criterion of 90% and even better except for the escalator/elevator with 87.0% in the second stage. Relevant confusion among the graphic symbols tested was found. Some suggestions were made in the study.

Interventions to improve functioning, participation, and quality of life in children with visual impairment: a systematic review

Visual impairment in childhood often has life-long implications. To aim for the highest levels of functioning, participation, and quality of life and to ensure children's well-being, children should be entitled to the most effective rehabilitation programs. We review evidence for the effectiveness of rehabilitation interventions for children with visual impairment to improve skills and behavior, thereby improving participation and quality of life as an ultimate goal. Of the 441 potentially relevant articles identified, 66 studies met our inclusion criteria (i.e., 28 randomized controlled trials, 18 nonrandomized controlled trials, and 20 before-after comparisons). The results suggest that sports camps, prescription and training in the use of low vision devices, and oral hygiene programs might be effective in improving functioning and elements of participation and quality of life in children with visual impairment. Other interventions showed mixed or negative results. The results should be interpreted with caution because of moderate to high risk of bias and suboptimal reporting. Heterogeneity of results and the use of over 50 different outcome measures prevented a meta-analysis. Future studies should focus on promising interventions for which effectiveness is still unclear (e.g., mobility, social skills), with adequately designed methodology.

Improving spatial working memory in blind and sighted youngsters using programmable tactile displays

OBJECTIVE

To investigate whether training with tactile matrices displayed with a programmable tactile display improves recalling performance of spatial images in blind, low-vision and sighted youngsters. To code and understand the behavioral underpinnings of learning two-dimensional tactile dispositions, in terms of spontaneous exploration strategies.

METHODS

Three groups of blind, low-vision and sighted youngsters between 6 and 18 years old performed four training sessions with a weekly schedule in which they were asked to memorize single or double spatial layouts, featured as two-dimensional matrices.

RESULTS

Results showed that all groups of participants significantly improved their recall performance compared to the first session baseline in the single-matrix task. No statistical difference in performance between groups emerged in this task. Instead, the learning effect in visually impaired participants is reduced in the double-matrix task, whereas it is still robust in blindfolded sighted controls. We also coded tactile exploration strategies in both tasks and their correlation with performance. Sighted youngsters, in particular, favored a proprioceptive exploration strategy. Finally, performance in the double-matrix task negatively correlated with using one hand and positively correlated with a proprioceptive strategy.

CONCLUSION

The results of our study indicate that blind persons do not easily process two separate spatial layouts. However, rehabilitation programs promoting bi-manual and proprioceptive approaches to tactile exploration might help improve spatial abilities. Finally, programmable tactile displays are an effective way to make spatial and graphical configurations accessible to visually impaired youngsters and they can be profitably exploited in rehabilitation.

Updated Tactile Feedback with a Pin Array Matrix Helps Blind People to Reduce Self-Location Errors

Autonomous navigation in novel environments still represents a challenge for people with visual impairment (VI). Pin array matrices (PAM) are an effective way to display spatial information to VI people in educative/rehabilitative contexts, as they provide high flexibility and versatility. Here, we tested the effectiveness of a PAM in VI participants in an orientation and mobility task. They haptically explored a map showing a scaled representation of a real room on the PAM. The map further included a symbol indicating a virtual target position. Then, participants entered the room and attempted to reach the target three times. While a control group only reviewed the same, unchanged map on the PAM between trials, an experimental group also received an updated map representing, in addition, the position they previously reached in the room. The experimental group significantly improved across trials by having both reduced self-location errors and reduced completion time, unlike the control group. We found that learning spatial layouts through updated tactile feedback on programmable displays outperforms conventional procedures on static tactile maps. This could represent a powerful tool for navigation, both in rehabilitation and everyday life contexts, improving spatial abilities and promoting independent living for VI people.

Understanding Graphics on a Scalable Latching Assistive Haptic Display Using a Shape Memory Polymer Membrane

We present a fully latching and scalable 4 × 4 haptic display with 4 mm pitch, 5 s refresh time, 400 mN holding force, and 650 μm displacement per taxel. The display serves to convey dynamic graphical information to blind and visually impaired users. Combining significant holding force with high taxel density and large amplitude motion in a very compact overall form factor was made possible by exploiting the reversible, fast, hundred-fold change in the stiffness of a thin shape memory polymer (SMP) membrane when heated above its glass transition temperature. Local heating is produced using an addressable array of 3 mm in diameter stretchable microheaters patterned on the SMP. Each taxel is selectively and independently actuated by synchronizing the local Joule heating with a single pressure supply. Switching off the heating locks each taxel into its position (up or down), enabling holding any array configuration with zero power consumption. A 3D-printed pin array is mounted over the SMP membrane, providing the user with a smooth and room temperature array of movable pins to explore by touch. Perception tests were carried out with 24 blind users resulting in 70 percent correct pattern recognition over a 12-word tactile dictionary.