THERE IS SOMETHING OUT THERE: DISTAL ATTRIBUTION IN SENSORY SUBSTITUTION, TWENTY YEARS LATER

Enhancing interoceptive sensibility through exteroceptive-interoceptive sensory substitution

Exploring a novel approach to mental health technology, this study illuminates the intricate interplay between exteroception (the perception of the external world), and interoception (the perception of the internal world). Drawing on principles of sensory substitution, we investigated how interoceptive signals, particularly respiration, could be conveyed through exteroceptive modalities, namely vision and hearing. To this end, we developed a unique, immersive multisensory environment that translates respiratory signals in real-time into dynamic visual and auditory stimuli. The system was evaluated by employing a battery of various psychological assessments, with the findings indicating a significant increase in participants' interoceptive sensibility and an enhancement of the state of flow, signifying immersive and positive engagement with the experience. Furthermore, a correlation between these two variables emerged, revealing a bidirectional enhancement between the state of flow and interoceptive sensibility. Our research is the first to present a sensory substitution approach for substituting between interoceptive and exteroceptive senses, and specifically as a transformative method for mental health interventions, paving the way for future research.

Multisensory training improves the development of spatial cognition after sight restoration from congenital cataracts

Spatial cognition and mobility are typically impaired in congenitally blind individuals, as vision usually calibrates space perception by providing the most accurate distal spatial cues. We have previously shown that sight restoration from congenital bilateral cataracts guides the development of more accurate space perception, even when cataract removal occurs years after birth. However, late cataract-treated individuals do not usually reach the performance levels of the typically sighted population. Here, we developed a brief multisensory training that associated audiovisual feedback with body movements. Late cataract-treated participants quickly improved their space representation and mobility, performing as well as typically sighted controls in most tasks. Their improvement was comparable with that of a group of blind participants, who underwent training coupling their movements with auditory feedback alone. These findings suggest that spatial cognition can be enhanced by a training program that strengthens the association between bodily movements and their sensory feedback (either auditory or audiovisual).

How voluntary control over information and body movements determines "what it's like" to have perceptual, bodily, emotional and mental experiences

Two very fundamental aspects of phenomenal experiences underline the fact that they seem to have "something it's like." One aspect is the fact that experiences have a locus: they Can seem "external" (perceptual), "internal" (interoceptive, bodily or emotional) or "mental." A second fundamental aspect is the imposingness of experiences. They can seem "present" to us in different ways, sometimes seeming displayed before us with "spatio-temporal presence." Both these aspects of "what it's like" can be identified with the degree to which we can voluntarily control what we are doing when we engage in an experience. The external/internal/mental dimension is determined by how our voluntary bodily actions can influence the sensorimotor flow of information. The degree of imposingness of experiences and their "spatio-temporal presence" Is determined by how our voluntary actions are impeded or assisted by innate, attention-grabbing mechanisms. By elucidating these two most fundamental aspects of "what it's like," and taken together with prior work on inter- and intra-modal differences in experiences, this article suggests a path toward a scientific theory of the "hard problem" of phenomenal consciousness, explaining why experiences feel like something rather than feeling like nothing.

Scientific Observation Is Socio-Materially Augmented Perception: Toward a Participatory Realism

There is an overlooked similarity between three classic accounts of the conditions of object experience from three distinct disciplines. (1) Sociology: the “inversion” that accompanies discovery in the natural sciences, as local causes of effects are reattributed to an observed object. (2) Psychology: the “externalization” that accompanies mastery of a visual–tactile sensory substitution interface, as tactile sensations of the proximal interface are transformed into vision-like experience of a distal object. (3) Biology: the “projection” that brings forth an animal’s Umwelt, as impressions on its body’s sensory surfaces are reconfigured into perception of an external object. This similarity between the effects of scientific practice and interface-use on the one hand, and of sensorimotor interaction on the other, becomes intelligible once we accept that skillful engagement with instruments and interfaces constitutes a socio-material augmentation of our basic perceptual capacity. This enactive interpretation stands in contrast to anti-realism about science associated with constructivist interpretations of these three phenomena, which are motivated by viewing them as the internal mental construction of the experienced object. Instead, it favors a participatory realism: the sensorimotor basis of perceptual experience loops not only through our body, but also through the external world. This allows us to conceive of object experience in relational terms, i.e., as one or more subjects directly engaging with the world. Consequently, we can appreciate scientific observation in its full complexity: it is a socio-materially augmented process of becoming acquainted with the observed object that—like tool-use and perceiving more generally—is irreducibly self, other-, and world-involving.

The Unfolding Space Glove: A Wearable Spatio-Visual to Haptic Sensory Substitution Device for Blind People

This paper documents the design, implementation and evaluation of the Unfolding Space Glove—an open source sensory substitution device. It transmits the relative position and distance of nearby objects as vibratory stimuli to the back of the hand and thus enables blind people to haptically explore the depth of their surrounding space, assisting with navigation tasks such as object recognition and wayfinding. The prototype requires no external hardware, is highly portable, operates in all lighting conditions, and provides continuous and immediate feedback—all while being visually unobtrusive. Both blind (n = 8) and blindfolded sighted participants (n = 6) completed structured training and obstacle courses with both the prototype and a white long cane to allow performance comparisons to be drawn between them. The subjects quickly learned how to use the glove and successfully completed all of the trials, though still being slower with it than with the cane. Qualitative interviews revealed a high level of usability and user experience. Overall, the results indicate the general processability of spatial information through sensory substitution using haptic, vibrotactile interfaces. Further research would be required to evaluate the prototype’s capabilities after extensive training and to derive a fully functional navigation aid from its features.

Augmenting perception: How artificial intelligence transforms sensory substitution

What happens when artificial sensors are coupled with the human senses? Using technology to extend the senses is an old human dream, on which sensory substitution and other augmentation technologies have already delivered. Laser tactile canes, corneal implants and magnetic belts can correct or extend what individuals could otherwise perceive. Here we show why accommodating intelligent sensory augmentation devices not just improves but also changes the way of thinking and classifying former sensory augmentation devices. We review the benefits in terms of signal processing and show why non-linear transformation is more than a mere improvement compared to classical linear transformation.

Spatial navigation with horizontally spatialized sounds in early and late blind individuals

Blind individuals often report difficulties to navigate and to detect objects placed outside their peri-personal space. Although classical sensory substitution devices could be helpful in this respect, these devices often give a complex signal which requires intensive training to analyze. New devices that provide a less complex output signal are therefore needed. Here, we evaluate a smartphone-based sensory substitution device that offers navigation guidance based on strictly spatial cues in the form of horizontally spatialized sounds. The system uses multiple sensors to either detect obstacles at a distance directly in front of the user or to create a 3D map of the environment (detection and avoidance mode, respectively), and informs the user with auditory feedback. We tested 12 early blind, 11 late blind and 24 blindfolded-sighted participants for their ability to detect obstacles and to navigate in an obstacle course. The three groups did not differ in the number of objects detected and avoided. However, early blind and late blind participants were faster than their sighted counterparts to navigate through the obstacle course. These results are consistent with previous research on sensory substitution showing that vision can be replaced by other senses to improve performance in a wide variety of tasks in blind individuals. This study offers new evidence that sensory substitution devices based on horizontally spatialized sounds can be used as a navigation tool with a minimal amount of training.

Testing Vision Is Not Testing For Vision

Visual prostheses aim to restore, at least to some extent, vision that leads to the type of perception available for sighted patients. Their effectiveness is almost always evaluated using clinical tests of vision. Clinical vision tests are designed to measure the limits of parameters of a functioning visual system. I argue here that these tests are rarely suited to determine the ability of prosthetic devices and other therapies to restore vision. This paper describes and explains many limitations of these evaluations. Prosthetic vision testing often makes use of multiple-alternative forced-choice (MAFC) procedures. Although these paradigms are suitable for many studies, they are frequently problematic in vision restoration evaluation. Two main types of problems are identified: (1) where nuisance variables provide spurious cues that can be learned in repeated training, which is common in prosthetic vision, and thus defeat the purpose of the test; and (2) even though a test is properly designed and performed, it may not actually measure what the researchers believe, and thus the interpretation of results is wrong. Examples for both types of problems are presented. Additional problems arise from confounding factors in the administration of tests are pointed as limitations of current device evaluation. For example, head tracing of magnified objects enlarged to compensate for the system's low resolution, in distinction from the scanning head (camera) movements with which users of prosthetic devices expand the limited field of view. Because of these problems, the ability to perform satisfactorily on the clinical tests is necessary but insufficient to prove vision restoration, therefore, additional tests are needed. I propose some directions to pursue in such testing.

Translational Relevance

Numerous prosthetic devices are being developed and introduced to the market. Proving the utility of these devices is crucial for regulatory and even for post market acceptance, which so far has largely failed, in my opinion. Potential reasons for the failures despite success in regulatory testing and directions for designing improved testing are provided. It is hoped that improved testing will guide improved designs of future prosthetic systems and other vision restoration approaches.

Spatial Competence and Brain Plasticity in Congenital Blindness via Sensory Substitution Devices

In congenital blindness (CB), tactile, and auditory information can be reinterpreted by the brain to compensate for visual information through mechanisms of brain plasticity triggered by training. Visual deprivation does not cause a cognitive spatial deficit since blind people are able to acquire spatial knowledge about the environment. However, this spatial competence takes longer to achieve but is eventually reached through training-induced plasticity. Congenitally blind individuals can further improve their spatial skills with the extensive use of sensory substitution devices (SSDs), either visual-to-tactile or visual-to-auditory. Using a combination of functional and anatomical neuroimaging techniques, our recent work has demonstrated the impact of spatial training with both visual to tactile and visual to auditory SSDs on brain plasticity, cortical processing, and the achievement of certain forms of spatial competence. The comparison of performances between CB and sighted people using several different sensory substitution devices in perceptual and sensory-motor tasks uncovered the striking ability of the brain to rewire itself during perceptual learning and to interpret novel sensory information even during adulthood. We discuss here the implications of these findings for helping blind people in navigation tasks and to increase their accessibility to both real and virtual environments.

Where Is the Action in Perception? An Exploratory Study With a Haptic Sensory Substitution Device

Enactive cognitive science (ECS) and ecological psychology (EP) agree that active movement is important for perception, but they remain ambiguous regarding the precise role of agency. EP has focused on the notion of sensorimotor invariants, according to which bodily movements play an instrumental role in perception. ECS has focused on the notion of sensorimotor contingencies, which goes beyond an instrumental role because skillfully regulated movements are claimed to play a constitutive role. We refer to these two hypotheses as instrumental agency and constitutive agency, respectively. Evidence comes from a variety of fields, including neural, behavioral, and phenomenological research, but so far with confounds that prevent an experimental distinction between these hypotheses. Here we advance the debate by proposing a novel double-participant setup that aims to isolate agency as the key variable that distinguishes bodily movement in active and passive conditions of perception. We pilot this setup with a psychological study of width discrimination using the Enactive Torch, a haptic sensory substitution device. There was no evidence favoring the stronger hypothesis of constitutive agency over instrumental agency. However, we caution that during debriefing several participants reported using cognitive strategies that did not rely on spatial perception. We conclude that this approach is a viable direction for future research, but that greater care is required to establish and confirm the desired modality of first-person experience.

A case of visuo-auditory sensory substitution in rats

Sensory substitution refers to a process whereby an agent, by means of a removable specialized instrumentation, becomes capable of exploiting an available sensory modality in order to perceive properties of the environment which are normally accessible by means of a different modality. We describe a situation of visual-auditory sensory substitution in the rat. Rats were placed in complete darkness, and trained to follow a virtual path whose position was signalled by a sound activated by a video-tracking device. Our hypothesis was that the rats would be able to succeed in this task of spatial navigation, following the sound contour by means of sensory-motor coupling based on seeking the sound (all-or-none) and mastering the relation between their own actions and the expected sensory feedback. Our results confirm this hypothesis and show the progressive structuring of meaningful exploratory activity, leading from the appearance of stopping behaviour when the sound is lost or acceleration when the sound is discovered, up to a veritable sensory-motor strategy which maximizes the possibilities for discovering and following the sound path. Thus, the animals seem to have developed a new form of perception which translates in particular into motor behaviour adapted to the search for sound.

When the vibrations allow for anticipating the force to be produced: an extend to Pfister et al. (2014)

According to the ideomotor theory, action selection is done by the mental anticipation of its perceptual consequences. If the distal information processed mainly by vision and hearing are considered essential for the representation of the action, the proximal information processed by the sense of touch and proprioception is of less importance. Recent works seem to show the opposite. Nevertheless, it is necessary to complete these results by offering a situation, more ecological, where response and effect can occur on the same effector. So, the goal of our work was to implement a more relevant spatial correspondence because to touch is not the same action that to hear or to see. To do so, participants pressed a specific key after the presentation of a stimulus. The key vibrated depending on the pressure exerted on it. In a compatible condition, high pressure on a key triggered a high vibration, while in an incompatible condition high pressure triggered a low vibration on the same effectors. As expected, the response times were faster in the compatible condition than the incompatible condition. This means that proximal information participates actively in the selection of action.

Sensory Substitution and Multimodal Mental Imagery

Many philosophers use findings about sensory substitution devices in the grand debate about how we should individuate the senses. The big question is this: Is "vision" assisted by (tactile) sensory substitution really vision? Or is it tactile perception? Or some sui generis novel form of perception? My claim is that sensory substitution assisted "vision" is neither vision nor tactile perception, because it is not perception at all. It is mental imagery: visual mental imagery triggered by tactile sensory stimulation. But it is a special form of mental imagery that is triggered by corresponding sensory stimulation in a different sense modality, which I call "multimodal mental imagery."

Designing sensory-substitution devices: Principles, pitfalls and potential1

An exciting possibility for compensating for loss of sensory function is to augment deficient senses by conveying missing information through an intact sense. Here we present an overview of techniques that have been developed for sensory substitution (SS) for the blind, through both touch and audition, with special emphasis on the importance of training for the use of such devices, while highlighting potential pitfalls in their design. One example of a pitfall is how conveying extra information about the environment risks sensory overload. Related to this, the limits of attentional capacity make it important to focus on key information and avoid redundancies. Also, differences in processing characteristics and bandwidth between sensory systems severely constrain the information that can be conveyed. Furthermore, perception is a continuous process and does not involve a snapshot of the environment. Design of sensory substitution devices therefore requires assessment of the nature of spatiotemporal continuity for the different senses. Basic psychophysical and neuroscientific research into representations of the environment and the most effective ways of conveying information should lead to better design of sensory substitution systems. Sensory substitution devices should emphasize usability, and should not interfere with other inter- or intramodal perceptual function. Devices should be task-focused since in many cases it may be impractical to convey too many aspects of the environment. Evidence for multisensory integration in the representation of the environment suggests that researchers should not limit themselves to a single modality in their design. Finally, we recommend active training on devices, especially since it allows for externalization, where proximal sensory stimulation is attributed to a distinct exterior object.

The Perception of Space and Form Recognition in a Simulated Environment: The Case of Minimalist Sensory-Substitution Devices

Whenever we explore a simulated environment, the sensorimotor interactions that underlie our perception of space may be modified. We investigated the conditions under which it is possible to acquire the mastery of new sensorimotor laws and thereby to infer new perceptual spaces. A computer interface, based on the principles of minimalist sensory-substitution devices, was designed to enable different possible links between a user's actions (manipulation of a mouse and/or keys of a keyboard) and the resulting pattern of sensory stimulation (visual or auditory) to be established. The interface generated an all-or-none stimulus whose activation varied as a function of the participant's exploration of a hidden form. In this study we addressed the following questions: What are the conditions necessary for participants to understand their actions as constituting a displacement in a simulated space? What are the conditions required for participants to conceive of sensations as originating from the encounter with an object situated in this space? Finally, what are the conditions required for participants to recognise forms within this space? The results of the two experiments reported here show that, under certain conditions, participants can interpret the new sensorimotor laws as movements in a new perceptual space and can recognise simple geometric forms, and that this occurs no matter whether the sensory stimulation is presented in the visual or auditory modality.

Sensory Substitution: The Spatial Updating of Auditory Scenes "Mimics" the Spatial Updating of Visual Scenes

Visual-to-auditory sensory substitution is used to convey visual information through audition, and it was initially created to compensate for blindness; it consists of software converting the visual images captured by a video-camera into the equivalent auditory images, or “soundscapes”. Here, it was used by blindfolded sighted participants to learn the spatial position of simple shapes depicted in images arranged on the floor. Very few studies have used sensory substitution to investigate spatial representation, while it has been widely used to investigate object recognition. Additionally, with sensory substitution we could study the performance of participants actively exploring the environment through audition, rather than passively localizing sound sources. Blindfolded participants egocentrically learnt the position of six images by using sensory substitution and then a judgment of relative direction task (JRD) was used to determine how this scene was represented. This task consists of imagining being in a given location, oriented in a given direction, and pointing towards the required image. Before performing the JRD task, participants explored a map that provided allocentric information about the scene. Although spatial exploration was egocentric, surprisingly we found that performance in the JRD task was better for allocentric perspectives. This suggests that the egocentric representation of the scene was updated. This result is in line with previous studies using visual and somatosensory scenes, thus supporting the notion that different sensory modalities produce equivalent spatial representation(s). Moreover, our results have practical implications to improve training methods with sensory substitution devices (SSD).

Body-scaled affordances in sensory substitution

The research field on sensory substitution devices has strong implications for theoretical work on perceptual consciousness. One of these implications concerns the extent to which the devices allow distal attribution. The present study applies a classic empirical approach on the perception of affordances to the field of sensory substitution. The reported experiment considers the perception of the stair-climbing affordance. Participants judged the climbability of steps apprehended through a vibrotactile sensory substitution device. If measured with standard metric units, climbability judgments of tall and short participants differed, but if measured in units of leg length, judgments did not differ. These results are similar to paradigmatic results in regular visual perception. We conclude that our sensory substitution device allows the perception of affordances. More generally, we argue that the theory of affordances may enrich theoretical debates concerning sensory substitution to a larger extent than has hitherto been the case.

Is there a future for sensory substitution outside academic laboratories?

Sensory substitution devices (SSDs) have been developed with the ultimate purpose of supporting sensory deprived individuals in their daily activities. However, more than forty years after their first appearance in the scientific literature, SSDs still remain more common in research laboratories than in the daily life of people with sensory deprivation. Here, we seek to identify the reasons behind the limited diffusion of SSDs among the blind community by discussing the ergonomic, neurocognitive and psychosocial issues potentially associated with the use of these systems. We stress that these issues should be considered together when developing future devices or improving existing ones. We provide some examples of how to achieve this by adopting a multidisciplinary and participatory approach. These efforts would contribute not solely to address fundamental theoretical research questions, but also to better understand the everyday needs of blind people and eventually promote the use of SSDs outside laboratories.

The co-constitution of the self and the world: action and proprioceptive coupling

This article proposes a theoretical reflection on the conditions for the constitution of a distinction between the self and the world by a cognitive system. The main hypothesis is the following: proprioception, as a sensory system that is habitually dedicated essentially to experience of the body, is conceived here as a coupling which is necessary for the dual and concomitant constitution of a bodily self and of a distal perceptual field. After recalling the singular characteristics of proprioceptive coupling, three lines of thought are developed. The first, which is notably inspired by research on sensory substitution, aims at emphasizing the indispensable role of action in the context of such perceptual learning. In a second part, this hypothesis is tested against opposing arguments. In particular, we shall discuss, in the context of what Braitenberg called a synthetic psychology, the emergence of oriented behaviors in simple robots that can be regulated by sensory regulations which are strictly external, since these robots do not have any form of "proprioception." In the same vein, this part also provides the opportunity to discuss the argument concerning a bijective relation between action and proprioception; it has been argued by others that because of this strict bijection it is not possible for proprioception to be the basis for the constitution of an exteriority. The third part, which is more prospective, suggests that it is important to take the measure of the phylogenetic history of this exteriority, starting from unicellular organisms. Taking into account the literature which attests the existence of proprioception even amongst the most elementary living organisms, this leads us to propose that the coupling of proprioception to action is very primitive, and that the role we propose for it in the co-constitution of an exteriority and self is probably already at work in the simplest living organisms.

See-Through-Sound

See-Through-Sound is a research project that is aimed at creating an innovative solution for mapping visual information into the auditory realm, enabling a spatial environment and its unique features to be described as organized sonic events. Of particular interest to us has been the creation of a tool for people with vision disabilities to help them perceive and recognize objects and features of their environment through sonic representations of light, color and shapes. Applications for sighted people have also been explored as sonification methods for monitoring changes in color within a broad range of scenarios, as well as advanced motion detection. The benefits and promise of this technology are far reaching; it goes beyond mere medical and scientific applications. Ultimately the main goal of this research project is the attempt to systematize and create a universal vocabulary of sonic events that map visual data into auditory data, both for man and machine use.

Perceiving space and optical cues via a visuo-tactile sensory substitution system: a methodological approach for training of blind subjects for navigation

A methodological approach to perceptual learning was used to allow both early blind subjects (experimental group) and blindfolded sighted subjects (control group) to experience optical information and spatial phenomena, on the basis of visuo-tactile information transmitted by a 64-taxel pneumatic sensory substitution device. The learning process allowed the subjects to develop abilities in spatial localisation, shape recognition (with generalisation to different points of view), and monocular depth cue interpretation. During the training phase, early blind people initially experienced more difficulties than blindfolded sighted subjects (having previous perceptual experience of perspective) with interpreting and using monocular depth cues. The amelioration of the performance for all blind subjects during training sessions and the quite similar level of performance reached by two groups in the final navigation tasks suggested that early blind people were able to develop and apply cognitive understanding of depth cues. Both groups showed generalisation of the learning from the initial phases to cue identification in the maze, and subjectively experienced shapes facing them. Subjects' performance depended not only on their perceptual experience but also on their previous spatial competencies.

The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithms

Sensory substitution is a promising technique for mitigating the loss of a sensory modality. Sensory substitution devices (SSDs) work by converting information from the impaired sense (e.g., vision) into another, intact sense (e.g., audition). However, there are a potentially infinite number of ways of converting images into sounds, and it is important that the conversion takes into account the limits of human perception and other user-related factors (e.g., whether the sounds are pleasant to listen to). The device explored here is termed “polyglot” because it generates a very large set of solutions. Specifically, we adapt a procedure that has been in widespread use in the design of technology but has rarely been used as a tool to explore perception—namely, interactive genetic algorithms. In this procedure, a very large range of potential sensory substitution devices can be explored by creating a set of “genes” with different allelic variants (e.g., different ways of translating luminance into loudness). The most successful devices are then “bred” together, and we statistically explore the characteristics of the selected-for traits after multiple generations. The aim of the present study is to produce design guidelines for a better SSD. In three experiments, we vary the way that the fitness of the device is computed: by asking the user to rate the auditory aesthetics of different devices (Experiment 1), and by measuring the ability of participants to match sounds to images (Experiment 2) and the ability to perceptually discriminate between two sounds derived from similar images (Experiment 3). In each case, the traits selected for by the genetic algorithm represent the ideal SSD for that task. Taken together, these traits can guide the design of a better SSD.

Cortical plasticity and preserved function in early blindness

The "neural Darwinism" theory predicts that when one sensory modality is lacking, as in congenital blindness, the target structures are taken over by the afferent inputs from other senses that will promote and control their functional maturation (Edelman, 1993). This view receives support from both cross-modal plasticity experiments in animal models and functional imaging studies in man, which are presented here.

Reading the World through the Skin and Ears: A New Perspective on Sensory Substitution

Sensory substitution devices aim at replacing or assisting one or several functions of a deficient sensory modality by means of another sensory modality. Despite the numerous studies and research programs devoted to their development and integration, sensory substitution devices have failed to live up to their goal of allowing one to "see with the skin" (White et al., 1970) or to "see with the brain" (Bach-y-Rita et al., 2003). These somewhat peremptory claims, as well as the research conducted so far, are based on an implicit perceptual paradigm. Such perceptual assumption accepts the equivalence between using a sensory substitution device and perceiving through a particular sensory modality. Our aim is to provide an alternative model, which defines sensory substitution as being closer to culturally implemented cognitive extensions of existing perceptual skills such as reading. In this article, we will show why the analogy with reading provides a better explanation of the actual findings, that is, both of the positive results achieved and of the limitations noticed across the field of research on sensory substitution. The parallel with the most recent two-route and interactive models of reading (e.g., Dehaene et al., 2005) generates a radically new way of approaching these results, by stressing the dependence of integration on the existing perceptual-semantic route. In addition, the present perspective enables us to generate innovative research questions and specific predictions which set the stage for future work.

The meaning of embodiment

There is substantial disagreement among philosophers of embodied cognitive science about the meaning of embodiment. In what follows, I describe three different views that can be found in the current literature. I show how this debate centers around the question of whether the science of embodied cognition can retain the computer theory of mind. One view, which I will label body functionalism, takes the body to play the functional role of linking external resources for problem solving with internal biological machinery. Embodiment is thus understood in terms of the role the body plays in supporting the computational circuits that realize cognition. Body enactivism argues by contrast that no computational account of cognition can account for the role of commonsense knowledge in our everyday practical engagement with the world. I will attempt a reconciliation of these seemingly opposed views.

Motor-sensory convergence in object localization: a comparative study in rats and humans

In order to identify basic aspects in the process of tactile perception, we trained rats and humans in similar object localization tasks and compared the strategies used by the two species. We found that rats integrated temporally related sensory inputs ('temporal inputs') from early whisk cycles with spatially related inputs ('spatial inputs') to align their whiskers with the objects; their perceptual reports appeared to be based primarily on this spatial alignment. In a similar manner, human subjects also integrated temporal and spatial inputs, but relied mainly on temporal inputs for object localization. These results suggest that during tactile object localization, an iterative motor-sensory process gradually converges on a stable percept of object location in both species.

The Enactive Torch: A New Tool for the Science of Perception

The cognitive sciences are increasingly coming to terms with the embodied, embedded, extended, and experiential aspects of the mind. Exemplifying this shift, the enactive approach points to an essential role of goal-directed bodily activity in the generation of meaningful perceptual experience, i.e., sense-making. Here, building on recent insights into the transformative effects of practical tool-use, we make use of the enactive approach in order to provide a definition of an enactive interface in terms of augmented sense-making. We introduce such a custom-built interface, the Enactive Torch, and present a study of its experiential effects. The results demonstrate that the user experience is not adequately captured by any standardly assumed perceptual modality; rather, it is a new feeling that is mediated by the design of the device and shaped by the overall situation of the task. Taken together these findings show that there is much to be gained by synergies between engineering and the cognitive sciences in the creation of new experience-centered technology. We suggest that the guiding principle should be the design of interfaces that serve as a transparent medium for augmenting our natural skills of interaction with the world, instead of requiring conscious attention to the interface as an opaque object in the world.

Vision substitution and depth perception: early blind subjects experience visual perspective through their ears

AIM

Sensory substitution (SS) represents a unique opportunity to provide congenitally blind persons with visual-like experience. Although visual experience influences the way we perceive the external world, little is known about the effects of SS experience.

PURPOSE

To investigate the effects of perceptual experience (visual versus sensory substitution) on depth perception through an SS system, object localization abilities of early blind (n = 10), and blindfolded sighted control subjects (n = 20) were assessed before and after a practicing period with a visual-to-auditory SS device.

METHOD

During the pre- and post-test, subjects had to replace, by hand, an object previously localized using the device. The practicing phase consisted of three sessions during which subjects tried to localize and grasp an object using the device. Results. At the pre-test, sighted subjects spontaneously used efficiently different pictorial depth cues to estimate object distance while the blind subjects were affected by their lack of visual experience and were significantly less accurate. Post-test showed that the brief practicing phase sufficed to enable blind subjects to acquire the rules of visual depth and to use them efficiently with the device.

CONCLUSIONS

These results suggest the possibility to compensate for some effects of early and long-lasting blindness by providing visual-like experience via SS. Theoretical implications are discussed.

Distal attribution and distance perception in sensory substitution

In sensory substitution, the user may be directly aware of distal objects, as in everyday perception, or make explicit cognitive inferences based on an awareness of the proximal stimulation. Anecdotal evidence supports the experience of distal attribution, but so far there have been few rigorous experimental tests of the claim. In this study, blindfolded participants observed a target light using a device consisting of a finger-mounted photodiode that drives tactile vibra-tion on the back. With the blindfold off and the target removed, participants moved a reference object to match the perceived egocentric distance of the target. Participants who were instructed to attend to the distal target improved significantly during 2 h of practice, whereas those instructed to attend to proximal variables showed no improvement. Unsigned error increased with ratings of proximal attention, but decreased with ratings of target object solidity, consistent with distal attribution. Performance transferred to the non-dominant arm and to a rotated body orientation, demonstrating that learning did not depend on a joint-specific sensorimotor relationship between target distance and arm configuration. The results experimentally confirm that distal attribution can occur in sensory substitution, based on a perceptual strategy rather than an explicit cognitive strategy. Moreover, they suggest that the informational basis for distal attribution is not a joint-specific sensorimotor relation, but a more abstract spatial invariant.