1
|
Filippini C, Lancioni GE, Alberti G, Pezzuoli F, Ceccarani P. A Technology Aid to Help People with Blindness and Moderate Intellectual Disability Retrieve Common Objects from Storage Units: A Proof-of-Concept Study. SENSORS (BASEL, SWITZERLAND) 2024; 24:4453. [PMID: 39065851 PMCID: PMC11281186 DOI: 10.3390/s24144453] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/21/2024] [Revised: 07/06/2024] [Accepted: 07/09/2024] [Indexed: 07/28/2024]
Abstract
BACKGROUND People with blindness and intellectual disability can have problems locating, identifying, and retrieving objects needed for daily activities (e.g., clothes and food items) from familiar storage contexts, such as cupboards and cabinets. OBJECTIVE This preliminary study assessed a technological system designed to help three people with those problems improve their performance. METHODS The technological system, which involved the use of tags with radio frequency identification codes, a smartphone, and a tag reader, aimed to guide the participants in searching and retrieving objects from three different storage units. In practice, the system provided different feedbacks depending on whether the participants were searching (a) in a wrong storage unit, (b) in a wrong shelf/drawer of the right storage unit, or (c) in the right shelf/drawer of the right storage unit. RESULTS All participants were successful in retrieving objects correctly with the technological system. The results also showed that (a) the participants preferred using the system over a control strategy, (b) were able to switch on and off the system independently, and (b) staff rated the system positively. CONCLUSIONS These preliminary findings suggest that the system might be a useful support tool for people with blindness and intellectual disability.
Collapse
Affiliation(s)
- Chiara Filippini
- Lega F. D’Oro Research Center, 60027 Osimo, Italy; (G.E.L.); (G.A.); (P.C.)
| | - Giulio E. Lancioni
- Lega F. D’Oro Research Center, 60027 Osimo, Italy; (G.E.L.); (G.A.); (P.C.)
| | - Gloria Alberti
- Lega F. D’Oro Research Center, 60027 Osimo, Italy; (G.E.L.); (G.A.); (P.C.)
| | - Francesco Pezzuoli
- Department of Design, University of Camerino and LiMiX Srl, 63100 Ascoli Piceno, Italy;
| | - Patrizia Ceccarani
- Lega F. D’Oro Research Center, 60027 Osimo, Italy; (G.E.L.); (G.A.); (P.C.)
| |
Collapse
|
2
|
Memeo M, Sandini G, Cocchi E, Brayda L. Blind people can actively manipulate virtual objects with a novel tactile device. Sci Rep 2023; 13:22845. [PMID: 38129483 PMCID: PMC10739710 DOI: 10.1038/s41598-023-49507-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 12/08/2023] [Indexed: 12/23/2023] Open
Abstract
Frequently in rehabilitation, visually impaired persons are passive agents of exercises with fixed environmental constraints. In fact, a printed tactile map, i.e. a particular picture with a specific spatial arrangement, can usually not be edited. Interaction with map content, instead, facilitates the learning of spatial skills because it exploits mental imagery, manipulation and strategic planning simultaneously. However, it has rarely been applied to maps, mainly because of technological limitations. This study aims to understand if visually impaired people can autonomously build objects that are completely virtual. Specifically, we investigated if a group of twelve blind persons, with a wide age range, could exploit mental imagery to interact with virtual content and actively manipulate it by means of a haptic device. The device is mouse-shaped and designed to jointly perceive, with one finger only, local tactile height and inclination cues of arbitrary scalar fields. Spatial information can be mentally constructed by integrating local tactile cues, given by the device, with global proprioceptive cues, given by hand and arm motion. The experiment consisted of a bi-manual task, in which one hand explored some basic virtual objects and the other hand acted on a keyboard to change the position of one object in real-time. The goal was to merge basic objects into more complex objects, like a puzzle. The experiment spanned different resolutions of the tactile information. We measured task accuracy, efficiency, usability and execution time. The average accuracy in solving the puzzle was 90.5%. Importantly, accuracy was linearly predicted by efficiency, measured as the number of moves needed to solve the task. Subjective parameters linked to usability and spatial resolutions did not predict accuracy; gender modulated the execution time, with men being faster than women. Overall, we show that building purely virtual tactile objects is possible in absence of vision and that the process is measurable and achievable in partial autonomy. Introducing virtual tactile graphics in rehabilitation protocols could facilitate the stimulation of mental imagery, a basic element for the ability to orient in space. The behavioural variable introduced in the current study can be calculated after each trial and therefore could be used to automatically measure and tailor protocols to specific user needs. In perspective, our experimental setup can inspire remote rehabilitation scenarios for visually impaired people.
Collapse
Affiliation(s)
- Mariacarla Memeo
- Robotics, Brain and Cognitive Sciences Department Now With Center for Human Technologies, Fondazione Istituto Italiano di Tecnologia, Via Enrico Melen 83, Genoa, Italy
| | - Giulio Sandini
- Robotics, Brain and Cognitive Sciences Department, Fondazione Istituto Italiano di Tecnologia, Via Enrico Melen 83, Genoa, Italy
| | - Elena Cocchi
- Istituto David Chiossone per Ciechi e Ipovedenti Onlus, Geona, Italy
| | - Luca Brayda
- Robotics, Brain and Cognitive Sciences Department, Fondazione Istituto Italiano di Tecnologia, Via Enrico Melen 83, Genoa, Italy.
- Acoesis srl, Via Enrico Melen 83, Genoa, Italy.
- Nextage srl, Piazza della Vittoria 12, Genova, Italia.
| |
Collapse
|
3
|
Tivadar RI, Franceschiello B, Minier A, Murray MM. Learning and navigating digitally rendered haptic spatial layouts. NPJ SCIENCE OF LEARNING 2023; 8:61. [PMID: 38102127 PMCID: PMC10724186 DOI: 10.1038/s41539-023-00208-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Accepted: 11/28/2023] [Indexed: 12/17/2023]
Abstract
Learning spatial layouts and navigating through them rely not simply on sight but rather on multisensory processes, including touch. Digital haptics based on ultrasounds are effective for creating and manipulating mental images of individual objects in sighted and visually impaired participants. Here, we tested if this extends to scenes and navigation within them. Using only tactile stimuli conveyed via ultrasonic feedback on a digital touchscreen (i.e., a digital interactive map), 25 sighted, blindfolded participants first learned the basic layout of an apartment based on digital haptics only and then one of two trajectories through it. While still blindfolded, participants successfully reconstructed the haptically learned 2D spaces and navigated these spaces. Digital haptics were thus an effective means to learn and translate, on the one hand, 2D images into 3D reconstructions of layouts and, on the other hand, navigate actions within real spaces. Digital haptics based on ultrasounds represent an alternative learning tool for complex scenes as well as for successful navigation in previously unfamiliar layouts, which can likely be further applied in the rehabilitation of spatial functions and mitigation of visual impairments.
Collapse
Affiliation(s)
- Ruxandra I Tivadar
- The Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland.
- Department of Ophthalmology, Fondation Asile des Aveugles, Lausanne, Switzerland.
- Centre for Integrative and Complementary Medicine, Department of Anesthesiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland.
- Cognitive Computational Neuroscience Group, Institute for Computer Science, University of Bern, Bern, Switzerland.
- The Sense Innovation and Research Center, Lausanne and Sion, Switzerland.
| | - Benedetta Franceschiello
- The Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
- The Sense Innovation and Research Center, Lausanne and Sion, Switzerland
- Institute of Systems Engineering, School of Engineering, University of Applied Sciences Western Switzerland (HES-SO Valais), Sion, Switzerland
| | - Astrid Minier
- The Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
- Department of Ophthalmology, Fondation Asile des Aveugles, Lausanne, Switzerland
| | - Micah M Murray
- The Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland.
- Department of Ophthalmology, Fondation Asile des Aveugles, Lausanne, Switzerland.
- The Sense Innovation and Research Center, Lausanne and Sion, Switzerland.
| |
Collapse
|
4
|
Leo F, Gori M, Sciutti A. Early blindness modulates haptic object recognition. Front Hum Neurosci 2022; 16:941593. [PMID: 36158621 PMCID: PMC9498977 DOI: 10.3389/fnhum.2022.941593] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Accepted: 08/16/2022] [Indexed: 11/13/2022] Open
Abstract
Haptic object recognition is usually an efficient process although slower and less accurate than its visual counterpart. The early loss of vision imposes a greater reliance on haptic perception for recognition compared to the sighted. Therefore, we may expect that congenitally blind persons could recognize objects through touch more quickly and accurately than late blind or sighted people. However, the literature provided mixed results. Furthermore, most of the studies on haptic object recognition focused on performance, devoting little attention to the exploration procedures that conducted to that performance. In this study, we used iCube, an instrumented cube recording its orientation in space as well as the location of the points of contact on its faces. Three groups of congenitally blind, late blind and age and gender-matched blindfolded sighted participants were asked to explore the cube faces where little pins were positioned in varying number. Participants were required to explore the cube twice, reporting whether the cube was the same or it differed in pins disposition. Results showed that recognition accuracy was not modulated by the level of visual ability. However, congenitally blind touched more cells simultaneously while exploring the faces and changed more the pattern of touched cells from one recording sample to the next than late blind and sighted. Furthermore, the number of simultaneously touched cells negatively correlated with exploration duration. These findings indicate that early blindness shapes haptic exploration of objects that can be held in hands.
Collapse
Affiliation(s)
- Fabrizio Leo
- Cognitive Architecture for Collaborative Technologies Unit, Istituto Italiano di Tecnologia, Genova, Italy
- *Correspondence: Fabrizio Leo,
| | - Monica Gori
- Unit for Visually Impaired People, Istituto Italiano di Tecnologia, Genova, Italy
| | - Alessandra Sciutti
- Cognitive Architecture for Collaborative Technologies Unit, Istituto Italiano di Tecnologia, Genova, Italy
| |
Collapse
|
5
|
Leo F, Sandini G, Sciutti A. Mental Rotation Skill Shapes Haptic Exploration Strategies. IEEE TRANSACTIONS ON HAPTICS 2022; 15:339-350. [PMID: 35344495 DOI: 10.1109/toh.2022.3162321] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Haptic exploration strategies have been traditionally studied focusing on hand movements and neglecting how objects are moved in space. However, in daily life situations touch and movement cannot be disentangled. Furthermore, the relation between object manipulation as well as performance in haptic tasks and spatial skill is still little understood. In this study, we used iCube, a sensorized cube recording its orientation in space as well as the location of the points of contact on its faces. Participants had to explore the cube faces where little pins were positioned in varying number and count the number of pins on the faces with either even or odd number of pins. At the end of this task, they also completed a standard visual mental rotation test (MRT). Results showed that higher MRT scores were associated with better performance in the task with iCube both in term of accuracy and exploration speed and exploration strategies associated with better performance were identified. High performers tended to rotate the cube so that the explored face had the same spatial orientation (i.e., they preferentially explored the upward face and rotated iCube to explore the next face in the same orientation). They also explored less often twice the same face and were faster and more systematic in moving from one face to the next. These findings indicate that iCube could be used to infer subjects' spatial skill in a more natural and unobtrusive fashion than with standard MRTs.
Collapse
|
6
|
Aprile G, Cappagli G, Morelli F, Gori M, Signorini S. Standardized and Experimental Tools to Assess Spatial Cognition in Visually Impaired Children: A Mini-Review. Front Neurosci 2020; 14:562589. [PMID: 33041760 PMCID: PMC7525087 DOI: 10.3389/fnins.2020.562589] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2020] [Accepted: 08/25/2020] [Indexed: 11/13/2022] Open
Abstract
The acquisition of spatial cognition is essential for both everyday functioning (e.g., navigation) and more specific goals (e.g., mathematics), therefore being able to assess and monitor spatial cognition from the first years of life would be essential to predict developmental outcomes and timely intervene whenever spatial development is compromised. Several shreds of evidence have indicated that spatial development can be compromised in the case of development with atypical sensory experience such as blindness. Despite the massive importance of spatial abilities for the development of psychomotor competencies across childhood, only a few standardized and experimental methods have been developed to assess them in visually impaired children. In this review, we will give a short overview of current formal (standardized) and informal (experimental) methods to assess spatial cognition in visually impaired children, demonstrating that very few validated tools have been proposed to date. The main contribution of this current work is to highlight the need of ad hoc studies to create and validate clinical measures to assess spatial cognition in visually impaired individuals and address potential future developments in this area of research.
Collapse
Affiliation(s)
- Giorgia Aprile
- Centre of Child Neurophthalmology, IRCCS Mondino Foundation, Pavia, Italy
| | - Giulia Cappagli
- Centre of Child Neurophthalmology, IRCCS Mondino Foundation, Pavia, Italy
| | - Federica Morelli
- Centre of Child Neurophthalmology, IRCCS Mondino Foundation, Pavia, Italy
| | - Monica Gori
- Unit for Visually Impaired People, Istituto Italiano di Tecnologia, Genova, Italy
| | - Sabrina Signorini
- Centre of Child Neurophthalmology, IRCCS Mondino Foundation, Pavia, Italy
| |
Collapse
|
7
|
Vinter A, Orlandi O, Morgan P. Identification of Textured Tactile Pictures in Visually Impaired and Blindfolded Sighted Children. Front Psychol 2020; 11:345. [PMID: 32210879 PMCID: PMC7075489 DOI: 10.3389/fpsyg.2020.00345] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2019] [Accepted: 02/14/2020] [Indexed: 11/13/2022] Open
Abstract
A high level of variability in the capacity of visually impaired children to accurately identify tactile images is reported in the literature, with on average rather low percentages of correct naming responses. However, most of these studies used raised-line drawings as stimuli to be explored and named. The present experiment investigated whether blind children of 3 to 8 years of age would demonstrate a satisfactory ability to name the elements making up tactile images when tested in an experimental setting similar to their natural reading conditions. Textured tactile images taken from genuine illustrated tactile books for young children were used, and the participants received information about the title of the book or listened the text that accompanied each picture before exploration, as it would occur in a natural reading setting. The results showed that their naming scores were indeed higher than previously reported at equivalent ages and did not differ from those of age-matched sighted children. These scores were positively impacted by haptic practice in blind children and correlated with the use of some specific exploratory procedures. The blind children benefited from information provided before exploration, as did their sighted counterparts. However, only in the former did the condition in which full information was provided influence the way the children organized their exploration. The haptic identification scores increased with age regardless of visual status, with the exploration times decreasing in the blind children, while the reverse trend was observed in the sighted children. These results are discussed at the light of the image-mediation model of haptics, suggesting that during the age period considered in the present experiment, blind children would progressively learn to process haptic information directly, thus leading to a decrease of exploration times, while sighted children would learn to translate haptic information into a visual image used to retrieve semantic information, involving an increase of their exploration times.
Collapse
Affiliation(s)
- Annie Vinter
- LEAD, CNRS UMR 5022, University Bourgogne Franche-Comté, Dijon, France
| | - Oriana Orlandi
- LEAD, CNRS UMR 5022, University Bourgogne Franche-Comté, Dijon, France
| | - Pascal Morgan
- LEAD, CNRS UMR 5022, University Bourgogne Franche-Comté, Dijon, France
| |
Collapse
|
8
|
Leo F, Ferrari E, Baccelliere C, Zarate J, Shea H, Cocchi E, Waszkielewicz A, Brayda L. Enhancing general spatial skills of young visually impaired people with a programmable distance discrimination training: a case control study. J Neuroeng Rehabil 2019; 16:108. [PMID: 31462262 PMCID: PMC6714081 DOI: 10.1186/s12984-019-0580-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Accepted: 08/19/2019] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND The estimation of relative distance is a perceptual task used extensively in everyday life. This important skill suffers from biases that may be more pronounced when estimation is based on haptics. This is especially true for the blind and visually impaired, for which haptic estimation of distances is paramount but not systematically trained. We investigated whether a programmable tactile display, used autonomously, can improve distance discrimination ability in blind and severely visually impaired youngsters between 7 and 22 years-old. METHODS Training consisted of four weekly sessions in which participants were asked to haptically find, on the programmable tactile display, the pairs of squares which were separated by the shortest and longest distance in tactile images with multiple squares. A battery of haptic tests with raised-line drawings was administered before and after training, and scores were compared to those of a control group that did only the haptic battery, without doing the distance discrimination training on the tactile display. RESULTS Both blind and severely impaired youngsters became more accurate and faster at the task during training. In haptic battery results, blind and severely impaired youngsters who used the programmable display improved in three and two tests, respectively. In contrast, in the control groups, the blind control group improved in only one test, and the severely visually impaired in no tests. CONCLUSIONS Distance discrimination skills can be trained equally well in both blind and severely impaired participants. More importantly, autonomous training with the programmable tactile display had generalized effects beyond the trained task. Participants improved not only in the size discrimination test but also in memory span tests. Our study shows that tactile stimulation training that requires minimal human assistance can effectively improve generic spatial skills.
Collapse
Affiliation(s)
- Fabrizio Leo
- Robotics, Brain and Cognitive Sciences Department, Fondazione Istituto Italiano di Tecnologia, Genoa, Italy
| | - Elisabetta Ferrari
- Robotics, Brain and Cognitive Sciences Department, Fondazione Istituto Italiano di Tecnologia, Genoa, Italy
| | - Caterina Baccelliere
- Robotics, Brain and Cognitive Sciences Department, Fondazione Istituto Italiano di Tecnologia, Genoa, Italy
| | - Juan Zarate
- LMTS, Ecole Polytechnique Fédérale de Lausanne (EPFL), Neuchâtel, Switzerland
| | - Herbert Shea
- LMTS, Ecole Polytechnique Fédérale de Lausanne (EPFL), Neuchâtel, Switzerland
| | | | | | - Luca Brayda
- Robotics, Brain and Cognitive Sciences Department, Fondazione Istituto Italiano di Tecnologia, Genoa, Italy
| |
Collapse
|