Cognitive and brain mechanisms in sensory substitution of vision: a contribution to the study of human perception

Spatial Competence and Brain Plasticity in Congenital Blindness via Sensory Substitution Devices

In congenital blindness (CB), tactile, and auditory information can be reinterpreted by the brain to compensate for visual information through mechanisms of brain plasticity triggered by training. Visual deprivation does not cause a cognitive spatial deficit since blind people are able to acquire spatial knowledge about the environment. However, this spatial competence takes longer to achieve but is eventually reached through training-induced plasticity. Congenitally blind individuals can further improve their spatial skills with the extensive use of sensory substitution devices (SSDs), either visual-to-tactile or visual-to-auditory. Using a combination of functional and anatomical neuroimaging techniques, our recent work has demonstrated the impact of spatial training with both visual to tactile and visual to auditory SSDs on brain plasticity, cortical processing, and the achievement of certain forms of spatial competence. The comparison of performances between CB and sighted people using several different sensory substitution devices in perceptual and sensory-motor tasks uncovered the striking ability of the brain to rewire itself during perceptual learning and to interpret novel sensory information even during adulthood. We discuss here the implications of these findings for helping blind people in navigation tasks and to increase their accessibility to both real and virtual environments.

Spatial metric in blindness: behavioural and cortical processing

Visual modality dominates spatial perception and, in lack of vision, space representation might be altered. Here we review our work showing that blind individuals have a strong deficit when performing spatial bisection tasks (Gori et al., 2014). We also describe the neural correlates associated with this deficit, as blind individuals do not show the same ERP response mimicking the visual C1 reported in sighted people during spatial bisection (Campus et al., 2019). Interestingly, the deficit is not always evident in late blind individuals, and it is dependent on blindness duration. We report that the deficit disappears when one presents coherent temporal and spatial cues to blind people. This suggests that they may use time information to infer spatial maps (Gori et al., 2018). Finally, we propose a model to explain why blind individuals are impaired in this task, speculating that a lack of vision drives the construction of a multi-sensory cortical network that codes space based on temporal, rather than spatial, coordinates.

Temporal Cues Influence Space Estimations in Visually Impaired Individuals

Many works have highlighted enhanced auditory processing in blind individuals, suggesting that they compensate for lack of vision with greater sensitivity of the other senses. Few years ago, we demonstrated severely impaired auditory precision in congenitally blind individuals performing an auditory spatial metric task: their thresholds for bisecting three consecutive spatially distributed sounds were seriously compromised, ranging from three times typical thresholds to total randomness. Here, we show that the deficit disappears if blind individuals are presented with coherent temporal and spatial cues. More interestingly, when the audio information is presented in conflict for space and time, sighted individuals are unaffected by the perturbation, whereas blind individuals are strongly attracted by the temporal cue. These results highlight that temporal cues influence space estimations in blind participants, suggesting for the first time that blind individuals use temporal information to infer spatial environmental coordinates.

•

Blind individuals are not able to perform auditory spatial metric tasks

•

Their deficit disappears when coherent temporal and spatial cues are presented

•

In some cases, blind people use temporal cues to infer spatial coordinates

Hearing, feeling or seeing a beat recruits a supramodal network in the auditory dorsal stream

Hearing a beat recruits a wide neural network that involves the auditory cortex and motor planning regions. Perceiving a beat can potentially be achieved via vision or even touch, but it is currently not clear whether a common neural network underlies beat processing. Here, we used functional magnetic resonance imaging (fMRI) to test to what extent the neural network involved in beat processing is supramodal, that is, is the same in the different sensory modalities. Brain activity changes in 27 healthy volunteers were monitored while they were attending to the same rhythmic sequences (with and without a beat) in audition, vision and the vibrotactile modality. We found a common neural network for beat detection in the three modalities that involved parts of the auditory dorsal pathway. Within this network, only the putamen and the supplementary motor area (SMA) showed specificity to the beat, while the brain activity in the putamen covariated with the beat detection speed. These results highlighted the implication of the auditory dorsal stream in beat detection, confirmed the important role played by the putamen in beat detection and indicated that the neural network for beat detection is mostly supramodal. This constitutes a new example of convergence of the same functional attributes into one centralized representation in the brain.

Auditory and proprioceptive spatial impairments in blind children and adults

It is not clear what role visual information plays in the development of space perception. It has previously been shown that in absence of vision, both the ability to judge orientation in the haptic modality and bisect intervals in the auditory modality are severely compromised (Gori, Sandini, Martinoli & Burr, 2010; Gori, Sandini, Martinoli & Burr, 2014). Here we report for the first time also a strong deficit in proprioceptive reproduction and audio distance evaluation in early blind children and adults. Interestingly, the deficit is not present in a small group of adults with acquired visual disability. Our results support the idea that in absence of vision the audio and proprioceptive spatial representations may be delayed or drastically weakened due to the lack of visual calibration over the auditory and haptic modalities during the critical period of development.

Tactile feedback improves auditory spatial localization

Our recent studies suggest that congenitally blind adults have severely impaired thresholds in an auditory spatial bisection task, pointing to the importance of vision in constructing complex auditory spatial maps (Gori et al., 2014). To explore strategies that may improve the auditory spatial sense in visually impaired people, we investigated the impact of tactile feedback on spatial auditory localization in 48 blindfolded sighted subjects. We measured auditory spatial bisection thresholds before and after training, either with tactile feedback, verbal feedback, or no feedback. Audio thresholds were first measured with a spatial bisection task: subjects judged whether the second sound of a three sound sequence was spatially closer to the first or the third sound. The tactile feedback group underwent two audio-tactile feedback sessions of 100 trials, where each auditory trial was followed by the same spatial sequence played on the subject’s forearm; auditory spatial bisection thresholds were evaluated after each session. In the verbal feedback condition, the positions of the sounds were verbally reported to the subject after each feedback trial. The no feedback group did the same sequence of trials, with no feedback. Performance improved significantly only after audio-tactile feedback. The results suggest that direct tactile feedback interacts with the auditory spatial localization system, possibly by a process of cross-sensory recalibration. Control tests with the subject rotated suggested that this effect occurs only when the tactile and acoustic sequences are spatially congruent. Our results suggest that the tactile system can be used to recalibrate the auditory sense of space. These results encourage the possibility of designing rehabilitation programs to help blind persons establish a robust auditory sense of space, through training with the tactile modality.

Cortical plasticity and preserved function in early blindness

The "neural Darwinism" theory predicts that when one sensory modality is lacking, as in congenital blindness, the target structures are taken over by the afferent inputs from other senses that will promote and control their functional maturation (Edelman, 1993). This view receives support from both cross-modal plasticity experiments in animal models and functional imaging studies in man, which are presented here.

A tactile vision substitution system for the study of active sensing

This paper presents a tactile vision substitution system (TVSS) for the study of active sensing. Two algorithms, namely image processing and trajectory tracking, were developed to enhance the capability of conventional TVSS. Image processing techniques were applied to reduce the artifacts and extract important features from the active camera and effectively converted the information into tactile stimuli with much lower resolution. A fixed camera was used to record the movement of the active camera. A trajectory tracking algorithm was developed to analyze the active sensing strategy of the TVSS users to explore the environment. The image processing subsystem showed advantageous improvement in extracting object's features for superior recognition. The trajectory tracking subsystem, on the other hand, enabled accurately locating the portion of the scene pointed by the active camera and providing profound information for the study of active sensing strategy applied by TVSS users.

How soft is that pillow? The perceptual localization of the hand and the haptic assessment of contact rigidity

A new haptic illusion is described, in which the location of the mobile object affects the perception of its rigidity. There is theoretical and experimental support for the notion that limb position sense results from the brain combining ongoing sensory information with expectations arising from prior experience. How does this probabilistic state information affect one's tactile perception of the environment mechanics? In a simple estimation process, human subjects were asked to report the relative rigidity of two simulated virtual objects. One of the objects remained fixed in space and had various coefficients of stiffness. The other virtual object had constant stiffness but moved with respect to the subjects. Earlier work suggested that the perception of an object's rigidity is consistent with a process of regression between the contact force and the perceived amount of penetration inside the object's boundary. The amount of penetration perceived by the subject was affected by varying the position of the object. This, in turn, had a predictable effect on the perceived rigidity of the contact. Subjects' reports on the relative rigidity of the object are best accounted for by a probabilistic model in which the perceived boundary of the object is estimated based on its current location and on past observations. Therefore, the perception of contact rigidity is accounted for by a stochastic process of state estimation underlying proprioceptive localization of the hand.

Vision substitution and depth perception: early blind subjects experience visual perspective through their ears

AIM

Sensory substitution (SS) represents a unique opportunity to provide congenitally blind persons with visual-like experience. Although visual experience influences the way we perceive the external world, little is known about the effects of SS experience.

PURPOSE

To investigate the effects of perceptual experience (visual versus sensory substitution) on depth perception through an SS system, object localization abilities of early blind (n = 10), and blindfolded sighted control subjects (n = 20) were assessed before and after a practicing period with a visual-to-auditory SS device.

METHOD

During the pre- and post-test, subjects had to replace, by hand, an object previously localized using the device. The practicing phase consisted of three sessions during which subjects tried to localize and grasp an object using the device. Results. At the pre-test, sighted subjects spontaneously used efficiently different pictorial depth cues to estimate object distance while the blind subjects were affected by their lack of visual experience and were significantly less accurate. Post-test showed that the brief practicing phase sufficed to enable blind subjects to acquire the rules of visual depth and to use them efficiently with the device.

CONCLUSIONS

These results suggest the possibility to compensate for some effects of early and long-lasting blindness by providing visual-like experience via SS. Theoretical implications are discussed.

Functional recruitment of visual cortex for sound encoded object identification in the blind

Individuals using a visual-to-auditory sensory substitution device (SSD) called 'The vOICe' can identify objects in their environment through images encoded by sound. We have shown that identifying objects with this SSD is associated with activation of occipital visual areas. Here, we show that repetitive transcranial magnetic stimulation (rTMS) delivered to a specific area of occipital cortex (identified by functional MRI) profoundly impairs a blind user's ability to identify objects. rTMS delivered to the same site had no effect on a visual imagery task. The task and site-specific disruptive effect of rTMS in this individual suggests that the cross-modal recruitment of occipital visual areas is functional in nature and critical to the patient's ability to process and decode the image sounds using this SSD.