Functional equivalence of spatial images from touch and vision: evidence from spatial updating in blind and sighted individuals

Evaluating the efficacy of UNav: A computer vision-based navigation aid for persons with blindness or low vision

UNav is a computer-vision-based localization and navigation aid that provides step-by-step route instructions to reach selected destinations without any infrastructure in both indoor and outdoor environments. Despite the initial literature highlighting UNav's potential, clinical efficacy has not yet been rigorously evaluated. Herein, we assess UNav against standard in-person travel directions (SIPTD) for persons with blindness or low vision (PBLV) in an ecologically valid environment using a non-inferiority design. Twenty BLV subjects (age = 38 ± 8.4; nine females) were recruited and asked to navigate to a variety of destinations, over short-range distances (<200 m), in unfamiliar spaces, using either UNav or SIPTD. Navigation performance was assessed with nine dependent variables to assess travel confidence, as well as spatial and temporal performances, including path efficiency, total time, and wrong turns. The results suggest that UNav is not only non-inferior to the standard-of-care in wayfinding (SIPTD) but also superior on 8 out of 9 metrics, as compared to SIPTD. This study highlights the range of benefits computer vision-based aids provide to PBLV in short-range navigation and provides key insights into how users benefit from this systematic form of computer-aided guidance, demonstrating transformative promise for educational attainment, gainful employment, and recreational participation.

Adults' spatial scaling of tactile maps: Insights from studying sighted, early and late blind individuals

The current study investigated spatial scaling of tactile maps among blind adults and blindfolded sighted controls. We were specifically interested in identifying spatial scaling strategies as well as effects of different scaling directions (up versus down) on participants' performance. To this aim, we asked late blind participants (with visual memory, Experiment 1) and early blind participants (without visual memory, Experiment 2) as well as sighted blindfolded controls to encode a map including a target and to place a response disc at the same spot on an empty, constant-sized referent space. Maps had five different sizes resulting in five scaling factors (1:3, 1:2, 1:1, 2:1, 3:1), allowing to investigate different scaling directions (up and down) in a single, comprehensive design. Accuracy and speed of learning about the target location as well as responding served as dependent variables. We hypothesized that participants who can use visual mental representations (i.e., late blind and blindfolded sighted participants) may adopt mental transformation scaling strategies. However, our results did not support this hypothesis. At the same time, we predicted the usage of relative distance scaling strategies in early blind participants, which was supported by our findings. Moreover, our results suggested that tactile maps can be scaled as accurately and even faster by blind participants than by sighted participants. Furthermore, irrespective of the visual status, participants of each visual status group gravitated their responses towards the center of the space. Overall, it seems that a lack of visual imagery does not impair early blind adults' spatial scaling ability but causes them to use a different strategy than sighted and late blind individuals.

Drawing as a tool for investigating the nature of imagery representations of blind people: The case of the canonical size phenomenon

Several studies have shown that blind people, including those with congenital blindness, can use raised-line drawings, both for "reading" tactile graphics and for drawing unassisted. However, research on drawings produced by blind people has mainly been qualitative. The current experimental study was designed to investigate the under-researched issue of the size of drawings created by people with blindness. Participants (N = 59) varied in their visual status. Adventitiously blind people had previous visual experience and might use visual representations (e.g., when visualising objects in imagery/working memory). Congenitally blind people did not have any visual experience. The participant's task was to draw from memory common objects that vary in size in the real world. The findings revealed that both groups of participants produced larger drawings of objects that have larger actual sizes. This means that the size of familiar objects is a property of blind people's mental representations, regardless of their visual status. Our research also sheds light on the nature of the phenomenon of canonical size. Since we have found the canonical size effect in a group of people who are blind from birth, the assumption of the visual nature of this phenomenon - caused by the ocular-centric biases present in studies on drawing performance - should be revised.

Scaling up = scaling down? Children's spatial scaling in different perceptual modalities and scaling directions

The present study examined whether scaling direction and perceptual modality affect children's spatial scaling. Children aged 6-8 years (N = 201) were assigned to a visual, visuo-haptic, and haptic condition in which they were presented with colourful, embossed graphics. In the haptic condition, they were asked to wear a blindfold during the test trials. Across several trials, children were asked to learn about the position of a target in a map and to localize a disc at the same location in a referent space. Scaling factor was manipulated systematically, so that children had to either scale up or scale down spatial information. Their absolute deviations from the correct target location, reversal and signed errors, and response times served as dependent variables. Results revealed higher absolute deviations and response times for the haptic modality as opposed to the visual modality. Children's signed errors, however, showed similar response strategies across the perceptual conditions. Therefore, it seems that a functional equivalence between vision and touch seems to emerge slowly across development for spatial scaling. With respect to scaling directions, findings showed that absolute deviations were affected by scaling factors, with symmetric increases in scaling up and scaling down in the haptic condition. Conversely, children showed an unbalanced pattern in the visual conditions, with higher accuracy in scaling down as opposed to scaling up. Overall, our findings suggest that visibility seems to factor into children's scaling process.

Effects of scaling direction on adults' spatial scaling in different perceptual domains

The current study investigated adults' strategies of spatial scaling from memory in three perceptual conditions (visual, haptic, and visuo-haptic) when scaling up and down. Following previous research, we predicted the usage of mental transformation strategies. In all conditions, participants (N = 90, aged 19-28 years) were presented with tactile, colored graphics which allowed to visually and haptically explore spatial information. Participants were first asked to encode a map including a target. Then, they were instructed to place a response object at the same place on an empty, constant-sized referent space. Maps had five different sizes resulting in five scaling factors (3:1, 2:1, 1:1, 1:2, 1:3). This manipulation also allowed assessing potentially symmetric effects of scaling direction on adults' responses. Response times and absolute errors served as dependent variables. In line with our hypotheses, the changes in these dependent variables were best explained by a quadratic function which suggests the usage of mental transformation strategies for spatial scaling. There were no differences between perceptual conditions concerning the influence of scaling factor on dependent variables. Results revealed symmetric effects of scaling direction on participants' accuracy whereas there were small differences for response times. Our findings highlight the usage of mental transformation strategies in adults' spatial scaling, irrespective of perceptual modality and scaling direction.

Piaget's 3-mountains task with impossible options: sighted, blindfolded, early and late blind participants

In Piaget's 3-mountains task, 3D objects - a cube, cone and sphere - sit on a square tabletop. They are portrayed in 2D pictures as elevations (projections to the sides) such as one with a square on the left, a triangle in the middle and a circle on the right. Three objects offer six elevations, of which four are possible and two impossible. The possibles are elevations from the sides of the table - front, left, right and rear. In the impossibles, an object in the corner of the table is shown in the middle of an elevation. Sighted, sighted-blindfolded, early- and late-blind adults judged the elevations as to side of the table or impossible. The results suggest similar spatial abilities across groups. The impossible options had mid-range accuracy for all groups, with reaction times like possible options. The sighted and blind participants may consider possible and impossible options sequentially, one item at a time.

Early blindness modulates haptic object recognition

Haptic object recognition is usually an efficient process although slower and less accurate than its visual counterpart. The early loss of vision imposes a greater reliance on haptic perception for recognition compared to the sighted. Therefore, we may expect that congenitally blind persons could recognize objects through touch more quickly and accurately than late blind or sighted people. However, the literature provided mixed results. Furthermore, most of the studies on haptic object recognition focused on performance, devoting little attention to the exploration procedures that conducted to that performance. In this study, we used iCube, an instrumented cube recording its orientation in space as well as the location of the points of contact on its faces. Three groups of congenitally blind, late blind and age and gender-matched blindfolded sighted participants were asked to explore the cube faces where little pins were positioned in varying number. Participants were required to explore the cube twice, reporting whether the cube was the same or it differed in pins disposition. Results showed that recognition accuracy was not modulated by the level of visual ability. However, congenitally blind touched more cells simultaneously while exploring the faces and changed more the pattern of touched cells from one recording sample to the next than late blind and sighted. Furthermore, the number of simultaneously touched cells negatively correlated with exploration duration. These findings indicate that early blindness shapes haptic exploration of objects that can be held in hands.

Cognitive map formation supported by auditory, haptic, and multimodal information in persons with blindness

For efficient navigation, the brain needs to adequately represent the environment in a cognitive map. In this review, we sought to give an overview of literature about cognitive map formation based on non-visual modalities in persons with blindness (PWBs) and sighted persons. The review is focused on the auditory and haptic modalities, including research that combines multiple modalities and real-world navigation. Furthermore, we addressed implications of route and survey representations. Taking together, PWBs as well as sighted persons can build up cognitive maps based on non-visual modalities, although the accuracy sometime somewhat differs between PWBs and sighted persons. We provide some speculations on how to deploy information from different modalities to support cognitive map formation. Furthermore, PWBs and sighted persons seem to be able to construct route as well as survey representations. PWBs can experience difficulties building up a survey representation, but this is not always the case, and research suggests that they can acquire this ability with sufficient spatial information or training. We discuss possible explanations of these inconsistencies.

Cognitive map formation through tactile map navigation in visually impaired and sighted persons

The human brain can form cognitive maps of a spatial environment, which can support wayfinding. In this study, we investigated cognitive map formation of an environment presented in the tactile modality, in visually impaired and sighted persons. In addition, we assessed the acquisition of route and survey knowledge. Ten persons with a visual impairment (PVIs) and ten sighted control participants learned a tactile map of a city-like environment. The map included five marked locations associated with different items. Participants subsequently estimated distances between item pairs, performed a direction pointing task, reproduced routes between items and recalled item locations. In addition, we conducted questionnaires to assess general navigational abilities and the use of route or survey strategies. Overall, participants in both groups performed well on the spatial tasks. Our results did not show differences in performance between PVIs and sighted persons, indicating that both groups formed an equally accurate cognitive map. Furthermore, we found that the groups generally used similar navigational strategies, which correlated with performance on some of the tasks, and acquired similar and accurate route and survey knowledge. We therefore suggest that PVIs are able to employ a route as well as survey strategy if they have the opportunity to access route-like as well as map-like information such as on a tactile map.

Size and Quality of Drawings Made by Adults Under Visual and Haptic Control

The aim of this study was twofold. First, our objective was to test the influence of an object’s actual size (size rank) on the drawn size of the depicted object. We tested the canonical size effect (i.e., drawing objects larger in the physical world as larger) in four drawing conditions — two perceptual conditions (blindfolded or sighted) crossed with two materials (paper or special foil for producing embossed drawings). Second, we investigated whether drawing quality (we analysed both the local and global criteria of quality) depends on drawing conditions. We predicted that drawing quality, unlike drawing size, would vary according to drawing conditions — namely, being higher when foil than paper was used for drawing production in the blindfolded condition. We tested these hypotheses with young adults who repeatedly drew eight different familiar objects (differentiated by size in the real world) in four drawing conditions. As expected, drawn size increased linearly with increasing size rank, whatever the drawing condition, thus replicating the canonical size effect and showing that this effect was not dependent on drawing conditions. In line with our hypothesis, in the blindfolded condition drawing quality was better when foil rather than paper was used, suggesting a benefit from haptic feedback on the trace produced. Besides, the quality of drawings produced was still higher in the sighted than the blindfolded condition. In conclusion, canonical size is present under different drawing conditions regardless of whether sight is involved or not, while perceptual control increases drawing quality in adults.

Blindfolded adults use mental transformation strategies for spatial scaling of tactile maps

The current study tested strategies of spatial scaling in the haptic domain. Blindfolded adults (N = 31, aged 20–24 years) were presented with an embossed graphic including a target and asked to encode a target location on this map, imagine this map at a given scale, and to localize a target at the same spot on an empty referent space. Maps varied in three different sizes whereas the referent space had a constant size, resulting in three different scaling factors (1:1, 1:2, 1:4). Participants’ response times and localization errors were measured. Analyses indicated that both response times and errors increased with higher scaling factors, suggesting the usage of mental transformation stratergies for spatial scaling. Overall, the present study provides a suitable, novel methodology to assess spatial scaling in the haptic domain.

Differences in adults' spatial scaling based on visual or haptic information

The present study examined differences in adults' spatial-scaling abilities across three perceptual conditions: (1) visual, (2) haptic, and (3) visual and haptic. Participants were instructed to encode the position of a convex target presented in a simple map without a time limit. Immediately after encoding the map, participants were presented with a referent space and asked to place a disc at the same location from memory. All spaces were designed as tactile graphics. Positions of targets varied along the horizontal dimension. The referent space was constant in size while sizes of maps were systematically varied, resulting in three scaling factor conditions: 1:4, 1:2, 1:1. Sixty adults participated in the study (M = 21.18; SD = 1.05). One-third of them was blindfolded throughout the entire experiment (haptic condition). The second group of participants was allowed to see the graphics (visual condition); the third group were instructed to see and touch the graphics (bimodal condition). An analysis of participants' absolute errors showed that participants produced larger errors in the haptic condition as opposed to the visual and bimodal conditions. There was also a significant interaction effect between scaling factor and perceptual condition. In the visual and bimodal conditions, results showed a linear increase in errors with higher scaling factors (which may suggest that adults adopted mental transformation strategies during the spatial scaling process), whereas, in the haptic condition, this relation was quadratic. Findings imply that adults' spatial-scaling performance decreases when visual information is not available.

Comparing Map Learning between Touchscreen-Based Visual and Haptic Displays: A Behavioral Evaluation with Blind and Sighted Users

The ubiquity of multimodal smart devices affords new opportunities for eyes-free applications for conveying graphical information to both sighted and visually impaired users. Using previously established haptic design guidelines for generic rendering of graphical content on touchscreen interfaces, the current study evaluates the learning and mental representation of digital maps, representing a key real-world translational eyes-free application. Two experiments involving 12 blind participants and 16 sighted participants compared cognitive map development and test performance on a range of spatio-behavioral tasks across three information-matched learning-mode conditions: (1) our prototype vibro-audio map (VAM), (2) traditional hardcopy-tactile maps, and (3) visual maps. Results demonstrated that when perceptual parameters of the stimuli were matched between modalities during haptic and visual map learning, test performance was highly similar (functionally equivalent) between the learning modes and participant groups. These results suggest equivalent cognitive map formation between both blind and sighted users and between maps learned from different sensory inputs, providing compelling evidence supporting the development of amodal spatial representations in the brain. The practical implications of these results include empirical evidence supporting a growing interest in the efficacy of multisensory interfaces as a primary interaction style for people both with and without vision. Findings challenge the long-held assumption that blind people exhibit deficits on global spatial tasks compared to their sighted peers, with results also providing empirical support for the methodological use of sighted participants in studies pertaining to technologies primarily aimed at supporting blind users.

Blindfolded adults' use of geometric cues in haptic-based relocation

Non-visual information is important for navigation in limited visibility conditions. We designed a haptic-based relocation task to examine blindfolded adults' use of geometric cues. Forty-eight participants learned to locate a corner in a parallelogram frame. They were then tested in different transformed frames: (a) a reverse-parallelogram, in which locations predicted by original length information and angle information conflicted, (b) a rectangle, which retained only length information, and (c) a rhombus, which retained only angle information. Results show that access to the environment's geometry through haptic modality is sufficient for relocation. However, adults' performances in the current task were different from that in visual tasks in previous findings. First, compared to previous findings in visual-based tasks, length information lost its priority. Approximately half of the participants relied on angle information in the conflict test and the other half relied on length. Second, though participants encoded both length and angle information in the learning phase, only one cue was relied on after the conflict test. Finally, though participants encoded the target location successfully, they failed to represent the global shape of the environment. We attribute adults' different performances in haptic-based and visual-based tasks to the high cognitive demands in encoding and using haptic spatial cues, especially length information.

Cognitive map formation through haptic and visual exploration of tactile city-like maps

In this study, we compared cognitive map formation of small-scale models of city-like environments presented in visual or tactile/haptic modalities. Previous research often addresses only a limited amount of cognitive map aspects. We wanted to combine several of these aspects to elucidate a more complete view. Therefore, we assessed different types of spatial information, and consider egocentric as well as allocentric perspectives. Furthermore, we compared haptic map learning with visual map learning. In total 18 sighted participants (9 in a haptic condition, 9 visuo-haptic) learned three tactile maps of city-like environments. The maps differed in complexity, and had five marked locations associated with unique items. Participants estimated distances between item pairs, rebuilt the map, recalled locations, and navigated two routes, after learning each map. All participants overall performed well on the spatial tasks. Interestingly, only on the complex maps, participants performed worse in the haptic condition than the visuo-haptic, suggesting no distinct advantage of vision on the simple map. These results support ideas of modality-independent representations of space. Although it is less clear on the more complex maps, our findings indicate that participants using only haptic or a combination of haptic and visual information both form a quite accurate cognitive map of a simple tactile city-like map.

Combining cues to judge distance and direction in an immersive virtual reality environment

When we move, the visual direction of objects in the environment can change substantially. Compared with our understanding of depth perception, the problem the visual system faces in computing this change is relatively poorly understood. Here, we tested the extent to which participants' judgments of visual direction could be predicted by standard cue combination rules. Participants were tested in virtual reality using a head-mounted display. In a simulated room, they judged the position of an object at one location, before walking to another location in the room and judging, in a second interval, whether an object was at the expected visual direction of the first. By manipulating the scale of the room across intervals, which was subjectively invisible to observers, we put two classes of cue into conflict, one that depends only on visual information and one that uses proprioceptive information to scale any reconstruction of the scene. We find that the sensitivity to changes in one class of cue while keeping the other constant provides a good prediction of performance when both cues vary, consistent with the standard cue combination framework. Nevertheless, by comparing judgments of visual direction with those of distance, we show that judgments of visual direction and distance are mutually inconsistent. We discuss why there is no need for any contradiction between these two conclusions.

An Important Step toward Understanding the Role of Body-based Cues on Human Spatial Memory for Large-Scale Environments

Moving our body through space is fundamental to human navigation; however, technical and physical limitations have hindered our ability to study the role of these body-based cues experimentally. We recently designed an experiment using novel immersive virtual-reality technology, which allowed us to tightly control the availability of body-based cues to determine how these cues influence human spatial memory [Huffman, D. J., & Ekstrom, A. D. A modality-independent network underlies the retrieval of large-scale spatial environments in the human brain. Neuron, 104, 611-622, 2019]. Our analysis of behavior and fMRI data revealed a similar pattern of results across a range of body-based cues conditions, thus suggesting that participants likely relied primarily on vision to form and retrieve abstract, holistic representations of the large-scale environments in our experiment. We ended our paper by discussing a number of caveats and future directions for research on the role of body-based cues in human spatial memory. Here, we reiterate and expand on this discussion, and we use a commentary in this issue by A. Steel, C. E. Robertson, and J. S. Taube (Current promises and limitations of combined virtual reality and functional magnetic resonance imaging research in humans: A commentary on Huffman and Ekstrom (2019). Journal of Cognitive Neuroscience, 2020) as a helpful discussion point regarding some of the questions that we think will be the most interesting in the coming years. We highlight the exciting possibility of taking a more naturalistic approach to study the behavior, cognition, and neuroscience of navigation. Moreover, we share the hope that researchers who study navigation in humans and nonhuman animals will synergize to provide more rapid advancements in our understanding of cognition and the brain.

Mental Maps and the Use of Sensory Information by Blind and Partially Sighted People

This article aims to fill an important gap in the literature by reporting on blind and partially sighted people's use of spatial representations (mental maps) from their perspective and when travelling on real routes. The results presented here were obtained from semi-structured interviews with 100 blind and partially sighted people in five different countries. They are intended to answer three questions about the representation of space by blind and partially sighted people, how these representations are used to support travel, and the implications for the design of travel aids and orientation and mobility training. They show that blind and partially sighted people do have spatial representations and that a number of them explicitly use the term mental map. This article discusses the variety of approaches to spatial representations, including the sensory modalities used, the use of global or local representations, and the applications to support travel. The conclusions summarize the answers to the three questions and include a two-level preliminary classification of the spatial representations of blind and partially sighted people.

Cognitive Mapping Without Vision: Comparing Wayfinding Performance After Learning From Digital Touchscreen-Based Multimodal Maps vs. Embossed Tactile Overlays

This article starts by discussing the state of the art in accessible interactive maps for use by blind and visually impaired (BVI) people. It then describes a behavioral experiment investigating the efficacy of a new type of low-cost, touchscreen-based multimodal interface, called a vibro-audio map (VAM), for supporting environmental learning, cognitive map development, and wayfinding behavior on the basis of nonvisual sensing. In the study, eight BVI participants learned two floor-maps of university buildings, one using the VAM and the other using an analogous hardcopy tactile map (HTM) overlaid on the touchscreen. They were asked to freely explore each map, with the task of learning the entire layout and finding three hidden target locations. After meeting a learning criterion, participants performed an environmental transfer test, where they were brought to the corresponding physical layout and were asked to plan/navigate routes between learned target locations from memory, i.e., without access to the map used at learning. The results using Bayesian analyses aimed at assessing equivalence showed highly similar target localization accuracy and route efficiency performance between conditions, suggesting that the VAM supports the same level of environmental learning, cognitive map development, and wayfinding performance as is possible from interactive displays using traditional tactile map overlays. These results demonstrate the efficacy of the VAM for supporting complex spatial tasks without vision using a commercially available, low-cost interface and open the door to a new era of mobile interactive maps for spatial learning and wayfinding by BVI navigators.

Spatial reorientation with a geometric array of auditory cues

A visuocentric bias has dominated the literature on spatial navigation and reorientation. Studies on visually accessed environments indicate that, during reorientation, human and non-human animals encode the geometric shape of the environment, even if this information is unnecessary and insufficient for the task. In an attempt to extend our limited knowledge on the similarities and differences between visual and non-visual navigation, here we examined whether the same phenomenon would be observed during auditory-guided reorientation. Provided with a rectangular array of four distinct auditory landmarks, blindfolded, sighted participants had to learn the location of a target object situated on a panel of an octagonal arena. Subsequent test trials were administered to understand how the task was acquired. Crucially, in a condition in which the auditory cues were indistinguishable (same sound sample), participants could still identify the correct target location, suggesting that the rectangular array of auditory landmarks was encoded as a geometric configuration. This is the first evidence of incidental encoding of geometric information with auditory cues and, consistent with the theory of functional equivalence, it supports the generalisation of mechanisms of spatial learning across encoding modalities.

Adults' spatial scaling: evidence from the haptic domain

The current study investigated adults' spatial-scaling abilities using a haptic localization task. As a first aim, we examined the strategies used to solve this haptic task. Secondly, we explored whether irrelevant visual information influenced adults' spatial-scaling performance. Thirty-two adults were asked to locate targets as presented in maps on a larger or same-sized referent space. Maps varied in size in accordance with different scaling factors (1:4, 1:2, 1:1), whereas the referent space was constant in size throughout the experimental session. The availability of irrelevant, non-informative vision was manipulated by blindfolding half of the participants prior to the experiment (condition without non-informative vision), whereas the other half were able to see their surroundings with the stimuli being hidden behind a curtain (condition with non-informative vision). Analyses with absolute errors (after correcting for reversal errors) as the dependent variable revealed a significant interaction of the scaling factor and non-informative vision condition. Adults in the blindfolded condition showed constant errors and response times irrespective of scaling factor. Such a response pattern indicates the usage of relative strategies. Adults in the curtain condition showed a linear increase in errors with higher scaling factors, whereas their response times remained constant. This pattern of results supports the usage of absolute strategies or mental transformation strategies. Overall, our results indicate different scaling strategies depending on the availability of non-informative vision, highlighting the strong influence of (even irrelevant) vision on adults' haptic processing.

A Modality-Independent Network Underlies the Retrieval of Large-Scale Spatial Environments in the Human Brain

In humans, the extent to which body-based cues, such as vestibular, somatosensory, and motoric cues, are necessary for normal expression of spatial representations remains unclear. Recent breakthroughs in immersive virtual reality technology allowed us to test how body-based cues influence spatial representations of large-scale environments in humans. Specifically, we manipulated the availability of body-based cues during navigation using an omnidirectional treadmill and a head-mounted display, investigating brain differences in levels of activation (i.e., univariate analysis), patterns of activity (i.e., multivariate pattern analysis), and putative network interactions between spatial retrieval tasks using fMRI. Our behavioral and neuroimaging results support the idea that there is a core, modality-independent network supporting spatial memory retrieval in the human brain. Thus, for well-learned spatial environments, at least in humans, primarily visual input may be sufficient for expression of complex representations of spatial environments. VIDEO ABSTRACT.

The format of mental imagery: from a critical review to an integrated embodied representation approach

The issue of the format of mental imagery is still an open debate. The classical analogue (depictive)-propositional (descriptive) debate has not provided definitive conclusions. Over the years, the debate has shifted within the frame of the embodied cognition approach, which focuses on the interdependence of perception, cognition and action. Although the simulation approach still retains the concept of representation, the more radical line of the embodied cognition approach emphasizes the importance of action and clearly disregards the concept of representation. In particular, the enactive approach focuses on motor procedures that allow the body to interact with the environment, whereas the sensorimotor approach focuses on the possession and exercise of sensorimotor knowledge about how the sensory input changes as a function of movement. In this review, the embodied approaches are presented and critically discussed. Then, in an attempt to show that the format of mental imagery varies according to the ability and the strategy used to represent information, the role of individual differences in imagery ability (e.g., vividness and expertise) and imagery strategy (e.g., object vs. spatial imagers) is reviewed. Since vividness is mainly associated with perceptual information, reflecting the activation level of specific imagery systems, whereas the preferred strategy used is mainly associated with perceptual (e.g., object imagery) or amodal and motor information (e.g., spatial imagery), the format of mental imagery appears to be based on dynamic embodied representations, depending on imagery abilities and imagery strategies.

Unidirectional influence of vision on locomotion in multimodal spatial representations acquired from navigation

Visual and idiothetic information is coupled in forming multimodal spatial representations during navigation (Tcheang et al. in Proc Natl Acad Sci USA 108(3):1152-1157, 2011). We investigated whether idiothetic representations activate visual representations but not vice versa (unidirectional coupling) or whether these two representations activate each other (bidirectional coupling). In a virtual reality environment, participants actively rotated in place to face certain orientations to become adapted to a new vision-locomotion relationship (gain). In particular, the visual turning angle was equal to 0.7 times the physical turning angle. After adaptation, participants walked a path with a turn in darkness (idiothetic input only) or watched a video of the traversed path (visual input only). Then, the participants pointed to the origin of the path. The participants who were presented with only idiothetic input showed that their pointing responses were influenced by the new gain (adaptation effect). By contrast, the participants who were presented with only visual input did not show any adaptation effect. These results suggest that idiothetic input contributed to spatial representations indirectly via the coupling, which resulted in the adaptation effect, whereas vision alone contributed to spatial representations directly, which did not result in the adaptation effect. Hence, the coupling between vision and locomotion is unidirectional.

Balance in Blind Subjects: Cane and Fingertip Touch Induce Similar Extent and Promptness of Stance Stabilization

Subjects with low vision often use a cane when standing and walking autonomously in everyday life. One aim of this study was to assess differences in the body stabilizing effect produced by the contact of the cane with the ground or by the fingertip touch of a firm surface. Another aim was to estimate the promptness of balance stabilization (or destabilization) on adding (or withdrawing) the haptic input from cane or fingertip. Twelve blind subjects and two subjects with severe visual impairment participated in two experimental protocols while maintaining the tandem Romberg posture on a force platform. In one protocol, subjects lowered the cane to a second platform on the ground and lifted it in sequence at their own pace. In the other protocol, they touched an instrumented pad with the index finger and withdrew the finger from the pad in sequence. In both protocols, subjects were asked to exert a force not granting mechanical stabilization. Under steady-state condition, the finger touch or the contact of the cane with the ground significantly reduced (to ∼78% and ∼86%, respectively) the amplitude of medio-lateral oscillation of the centre of foot pressure (CoP). Oscillation then increased when haptic information was removed. The delay to the change in body oscillation after the haptic shift was longer for addition than withdrawal of the haptic information (∼1.4 s and ∼0.7 s, respectively; p < 0.001), but was not different between the two haptic conditions (finger and cane). Similar stabilizing effects of input from cane on the ground and from fingertip touch, and similar latencies to integrate haptic cue from both sources, suggest that the process of integration of the input for balance control is initiated by the haptic stimulus at the interface cane-hand. Use of a tool is as helpful as the fingertip input, and does not produce different stabilization. Further, the latencies to haptic cue integration (from fingertip or cane) are similar to those previously found in a group of sighted subjects, suggesting that integration delays for automatic balance stabilization are not modified by visual impairment. Haptic input from a tool is easily exploited by the neural circuits subserving automatic balance stabilization in blind people, and its use should be enforced by sensory-enhancing devices and appropriate training.

Interacting networks of brain regions underlie human spatial navigation: a review and novel synthesis of the literature

Navigation is an inherently dynamic and multimodal process, making isolation of the unique cognitive components underlying it challenging. The assumptions of much of the literature on human spatial navigation are that 1) spatial navigation involves modality independent, discrete metric representations (i.e., egocentric vs. allocentric), 2) such representations can be further distilled to elemental cognitive processes, and 3) these cognitive processes can be ascribed to unique brain regions. We argue that modality-independent spatial representations, instead of providing exact metrics about our surrounding environment, more often involve heuristics for estimating spatial topology useful to the current task at hand. We also argue that egocentric (body centered) and allocentric (world centered) representations are better conceptualized as involving a continuum rather than as discrete. We propose a neural model to accommodate these ideas, arguing that such representations also involve a continuum of network interactions centered on retrosplenial and posterior parietal cortex, respectively. Our model thus helps explain both behavioral and neural findings otherwise difficult to account for with classic models of spatial navigation and memory, providing a testable framework for novel experiments.

Linguistic asymmetry, egocentric anchoring, and sensory modality as factors for the observed association between time and space perception

Temporal and spatial representations have been consistently shown to be inextricably intertwined. However, the exact nature of time-space mapping remains unknown. On the one hand, the conceptual metaphor theory postulates unilateral, asymmetric mapping of time onto space, that is, time is perceived in spatial terms but the perception of space is relatively independent of time. On the other hand, a theory of magnitude assumes bilateral and symmetric interactions between temporal and spatial perceptions. In the present paper, we argue that the concepts of linguistic asymmetry, egocentric anchoring, and sensory modality provide potential explanations for why evidences favoring both asymmetry and symmetry have been obtained. We first examine the asymmetry model and suggest that language plays a critical role in it. Next, we discuss the symmetry model in relation to egocentric anchoring and sensory modality. We conclude that since these three factors may jointly account for some conflicting past results regarding the strength and directionality of time-space mapping, they should be taken into serious consideration in future test designs.

SPATIAL UPDATING OF HAPTIC ARRAYS ACROSS THE LIFE SPAN

Background/Study Context: Aging research addressing spatial learning, representation, and action is almost exclusively based on vision as the input source. Much less is known about how spatial abilities from nonvisual inputs, particularly from haptic information, may change during life-span spatial development. This research studied whether learning and updating of haptic target configurations differs as a function of age.

METHODS

Three groups of participants, ranging from 20 to 80 years old, felt four-target table-top circular arrays and then performed several tasks to assess life-span haptic spatial cognition. Measures evaluated included egocentric pointing, allocentric pointing, and array reconstruction after physical or imagined spatial updating.

RESULTS

All measures revealed reliable differences between the oldest and youngest participant groups. The age effect for egocentric pointing contrasts with previous findings showing preserved egocentric spatial abilities. Error performance on allocentric pointing and map reconstruction tasks showing a clear age effect, with the oldest participants exhibiting the greatest error, is in line with other studies in the visual domain. Postupdating performance sharply declined with age but did not reliably differ between physical and imagined updating.

CONCLUSION

Results suggest that there is a general trend for age-related degradation of spatial abilities after haptic learning, with the greatest declines manifesting in all measures in people over 60 years of age. Results are interpreted in terms of a spatial aging effect on mental transformations of three-dimensional representations of space in working memory.

Principles for Designing Large-Format Refreshable Haptic Graphics Using Touchscreen Devices

Touchscreen devices, such as smartphones and tablets, represent a modern solution for providing graphical access to people with blindness and visual impairment (BVI). However, a significant problem with these solutions is their limited screen real estate, which necessitates panning or zooming operations for accessing large-format graphical materials such as maps. Non-visual interfaces cannot directly employ traditional panning or zooming techniques due to various perceptual and cognitive limitations (e.g., constraints of the haptic field of view and disorientation due to loss of one's reference point after performing these operations). This article describes the development of four novel non-visual panning methods designed from the onset with consideration of these perceptual and cognitive constraints. Two studies evaluated the usability of these panning methods in comparison with a non-panning control condition. Results demonstrated that the exploration, learning, and subsequent spatial behaviors were similar between panning and non-panning conditions, with one panning mode, based on a two-finger drag technique, revealing the overall best performance. Findings provide compelling evidence that incorporating panning operations on touchscreen devices -- the fastest growing computational platform among the BVI demographic -- is a viable, low-cost, and immediate solution for providing BVI people with access to a broad range of large-format digital graphical information.

Spatial updating of map-acquired representation

In the present study, we examined whether people can update a map-acquired spatial representation. The participants learned a spatial layout from a map displayed on a computer screen, and then performed spatial judgments at a novel position either in the same room (Exp. 1) or in a distal room (Exp. 2). They were required to imagine the spatial layout from a perspective aligned with the learning direction, aligned with their facing direction during testing, or toward a novel direction misaligned with the two directions mentioned above. In both the immediate and nonimmediate environments, the participants performed better from the learning direction than from the novel direction, and also performed better from the facing direction than from the novel direction. These results reveal that people establish an orientation-specific spatial representation through map learning, and that they can update a map-acquired representation during locomotion.

Indoor Spatial Updating With Impaired Vision

Purpose

Spatial updating is the ability to keep track of position and orientation while moving through an environment. We asked how normally sighted and visually impaired subjects compare in spatial updating and in estimating room dimensions.

Methods

Groups of 32 normally sighted, 16 low-vision, and 16 blind subjects estimated the dimensions of six rectangular rooms. Updating was assessed by guiding the subjects along three-segment paths in the rooms. At the end of each path, they estimated the distance and direction to the starting location, and to a designated target. Spatial updating was tested in five conditions ranging from free viewing to full auditory and visual deprivation.

Results

The normally sighted and low-vision groups did not differ in their accuracy for judging room dimensions. Correlations between estimated size and physical size were high. Accuracy of low-vision performance was not correlated with acuity, contrast sensitivity, or field status. Accuracy was lower for the blind subjects. The three groups were very similar in spatial-updating performance, and exhibited only weak dependence on the nature of the viewing conditions.

Conclusions

People with a wide range of low-vision conditions are able to judge room dimensions as accurately as people with normal vision. Blind subjects have difficulty in judging the dimensions of quiet rooms, but some information is available from echolocation. Vision status has little impact on performance in simple spatial updating; proprioceptive and vestibular cues are sufficient.

Touching and Hearing Unseen Objects: Multisensory Effects on Scene Recognition

In three experiments, we investigated the influence of object-specific sounds on haptic scene recognition without vision. Blindfolded participants had to recognize, through touch, spatial scenes comprising six objects that were placed on a round platform. Critically, in half of the trials, object-specific sounds were played when objects were touched (bimodal condition), while sounds were turned off in the other half of the trials (unimodal condition). After first exploring the scene, two objects were swapped and the task was to report, which of the objects swapped positions. In Experiment 1, geometrical objects and simple sounds were used, while in Experiment 2, the objects comprised toy animals that were matched with semantically compatible animal sounds. In Experiment 3, we replicated Experiment 1, but now a tactile-auditory object identification task preceded the experiment in which the participants learned to identify the objects based on tactile and auditory input. For each experiment, the results revealed a significant performance increase only after the switch from bimodal to unimodal. Thus, it appears that the release of bimodal identification, from audio-tactile to tactile-only produces a benefit that is not achieved when having the reversed order in which sound was added after having experience with haptic-only. We conclude that task-related factors other than mere bimodal identification cause the facilitation when switching from bimodal to unimodal conditions.

A review of principles in design and usability testing of tactile technology for individuals with visual impairments

To lay the groundwork for devising, improving, and implementing new technologies to meet the needs of individuals with visual impairments, a systematic literature review was conducted to: a) describe hardware platforms used in assistive devices, b) identify their various applications, and c) summarize practices in user testing conducted with these devices. A search in relevant EBSCO databases for articles published between 1980 and 2014 with terminology related to visual impairment, technology, and tactile sensory adaptation yielded 62 articles that met the inclusion criteria for final review. It was found that while earlier hardware development focused on pin matrices, the emphasis then shifted toward force feedback haptics and accessible touch screens. The inclusion of interactive and multimodal features has become increasingly prevalent. The quantity and consistency of research on navigation, education, and computer accessibility suggest that these are pertinent areas of need for the visually impaired community. Methodologies for usability testing ranged from case studies to larger cross-sectional studies. Many studies used blindfolded sighted users to draw conclusions about design principles and usability. Altogether, the findings presented in this review provide insight on effective design strategies and user testing methodologies for future research on assistive technology for individuals with visual impairments.

Sensory Substitution: The Spatial Updating of Auditory Scenes "Mimics" the Spatial Updating of Visual Scenes

Visual-to-auditory sensory substitution is used to convey visual information through audition, and it was initially created to compensate for blindness; it consists of software converting the visual images captured by a video-camera into the equivalent auditory images, or “soundscapes”. Here, it was used by blindfolded sighted participants to learn the spatial position of simple shapes depicted in images arranged on the floor. Very few studies have used sensory substitution to investigate spatial representation, while it has been widely used to investigate object recognition. Additionally, with sensory substitution we could study the performance of participants actively exploring the environment through audition, rather than passively localizing sound sources. Blindfolded participants egocentrically learnt the position of six images by using sensory substitution and then a judgment of relative direction task (JRD) was used to determine how this scene was represented. This task consists of imagining being in a given location, oriented in a given direction, and pointing towards the required image. Before performing the JRD task, participants explored a map that provided allocentric information about the scene. Although spatial exploration was egocentric, surprisingly we found that performance in the JRD task was better for allocentric perspectives. This suggests that the egocentric representation of the scene was updated. This result is in line with previous studies using visual and somatosensory scenes, thus supporting the notion that different sensory modalities produce equivalent spatial representation(s). Moreover, our results have practical implications to improve training methods with sensory substitution devices (SSD).

Systematic Movements in Haptic Search: Spirals, Zigzags, and Parallel Sweeps

Movement strategies were investigated in a one-handed haptic search task in which blindfolded-sighted participants used either one or five fingers to find a landmark on an unstructured tactile map. Search theory predicts that systematic strategies, such as spirals, zigzags, and parallel sweeps, should be more prevalent when the searcher's detection radius is small (one finger) and less common when the detection radius is large (five fingers). As predicted, systematic strategies were more common in one-finger than five-finger search. Participants were able to exploit the larger detection radius during five-finger searches to detect targets with any of their fingers, and in one-finger search used more systematic strategies. For the most part, participants' fingers moved together during five-finger search, expanding and moving quickly when looking for search targets/distractors, and contracting and moving slowly when examining search targets/distractors. There was no evidence of fingers being used as spatial anchors or other independent finger movements in five-finger search. While targets could be found with any fingers, examination was primarily accomplished using the index and middle fingers. Overall, these results indicate that untrained sighted participants will use optimal systematic strategies during haptic search, and this behavior is appropriately modulated by detection radius.

Spatial navigation by congenitally blind individuals

Spatial navigation in the absence of vision has been investigated from a variety of perspectives and disciplines. These different approaches have progressed our understanding of spatial knowledge acquisition by blind individuals, including their abilities, strategies, and corresponding mental representations. In this review, we propose a framework for investigating differences in spatial knowledge acquisition by blind and sighted people consisting of three longitudinal models (i.e., convergent, cumulative, and persistent). Recent advances in neuroscience and technological devices have provided novel insights into the different neural mechanisms underlying spatial navigation by blind and sighted people and the potential for functional reorganization. Despite these advances, there is still a lack of consensus regarding the extent to which locomotion and wayfinding depend on amodal spatial representations. This challenge largely stems from methodological limitations such as heterogeneity in the blind population and terminological ambiguity related to the concept of cognitive maps. Coupled with an over‐reliance on potential technological solutions, the field has diffused into theoretical and applied branches that do not always communicate. Here, we review research on navigation by congenitally blind individuals with an emphasis on behavioral and neuroscientific evidence, as well as the potential of technological assistance. Throughout the article, we emphasize the need to disentangle strategy choice and performance when discussing the navigation abilities of the blind population. WIREs Cogn Sci 2016, 7:37–58. doi: 10.1002/wcs.1375

For further resources related to this article, please visit the WIREs website.

Navigation using sensory substitution in real and virtual mazes

Under certain specific conditions people who are blind have a perception of space that is equivalent to that of sighted individuals. However, in most cases their spatial perception is impaired. Is this simply due to their current lack of access to visual information or does the lack of visual information throughout development prevent the proper integration of the neural systems underlying spatial cognition? Sensory Substitution devices (SSDs) can transfer visual information via other senses and provide a unique tool to examine this question. We hypothesize that the use of our SSD (The EyeCane: a device that translates distance information into sounds and vibrations) can enable blind people to attain a similar performance level as the sighted in a spatial navigation task. We gave fifty-six participants training with the EyeCane. They navigated in real life-size mazes using the EyeCane SSD and in virtual renditions of the same mazes using a virtual-EyeCane. The participants were divided into four groups according to visual experience: congenitally blind, low vision & late blind, blindfolded sighted and sighted visual controls. We found that with the EyeCane participants made fewer errors in the maze, had fewer collisions, and completed the maze in less time on the last session compared to the first. By the third session, participants improved to the point where individual trials were no longer significantly different from the initial performance of the sighted visual group in terms of errors, time and collision.

Visual, haptic and bimodal scene perception: evidence for a unitary representation

Participants studied seven meaningful scene-regions bordered by removable boundaries (30s each). In Experiment 1 (N = 80) participants used visual or haptic exploration and then minutes later, reconstructed boundary position using the same or the alternate modality. Participants in all groups shifted boundary placement outward (boundary extension), but visual study yielded the greater error. Critically, this modality-specific difference in boundary extension transferred without cost in the cross-modal conditions, suggesting a functionally unitary scene representation. In Experiment 2 (N = 20), bimodal study led to boundary extension that did not differ from haptic exploration alone, suggesting that bimodal spatial memory was constrained by the more "conservative" haptic modality. In Experiment 3 (N = 20), as in picture studies, boundary memory was tested 30s after viewing each scene-region and as with pictures, boundary extension still occurred. Results suggest that scene representation is organized around an amodal spatial core that organizes bottom-up information from multiple modalities in combination with top-down expectations about the surrounding world.

Effects of using multiple hands and fingers on haptic performance in individuals who are blind

In a previous paper we documented that sighted participants complete haptic tasks faster with two hands and multiple fingers, but that these benefits are task specific. The present study investigates whether these effects are the same for participants who are blind. We compared the performance of fourteen blind participants on seven tactile-map tasks using seven finger conditions. As with sighted participants, blind participants performed all tasks faster with multiple fingers. Line-tracing tasks were faster with fingers added to an already in-use hand, and sometimes when added to the second hand. Local and global search tasks were faster with multiple fingers and two hands. Distance comparison tasks were performed faster with multiple fingers, but not two hands. Lastly, moving in a straight line was faster with multiple fingers. These results reinforce our previous finding that the haptic system performs best when it can exploit the independence of multiple fingers. Furthermore, in every instance that an effect was different between sighted and blind participants, the blind participants benefitted more from two hands or multiple fingers than the sighted participants. This indicates that the blind participants have learned, through experience or training, how to best take advantage of multiple fingers during haptic tasks.

Loss of form vision impairs spatial imagery

Previous studies have reported inconsistent results when comparing spatial imagery performance in the blind and the sighted, with some, but not all, studies demonstrating deficits in the blind. Here, we investigated the effect of visual status and individual preferences ("cognitive style") on performance of a spatial imagery task. Participants with blindness resulting in the loss of form vision at or after age 6, and age- and gender-matched sighted participants, performed a spatial imagery task requiring memorization of a 4 × 4 lettered matrix and subsequent mental construction of shapes within the matrix from four-letter auditory cues. They also completed the Santa Barbara Sense of Direction Scale (SBSoDS) and a self-evaluation of cognitive style. The sighted participants also completed the Object-Spatial Imagery and Verbal Questionnaire (OSIVQ). Visual status affected performance on the spatial imagery task: the blind performed significantly worse than the sighted, independently of the age at which form vision was completely lost. Visual status did not affect the distribution of preferences based on self-reported cognitive style. Across all participants, self-reported verbalizer scores were significantly negatively correlated with accuracy on the spatial imagery task. There was a positive correlation between the SBSoDS score and accuracy on the spatial imagery task, across all participants, indicating that a better sense of direction is related to a more proficient spatial representation and that the imagery task indexes ecologically relevant spatial abilities. Moreover, the older the participants were, the worse their performance was, indicating a detrimental effect of age on spatial imagery performance. Thus, spatial skills represent an important target for rehabilitative approaches to visual impairment, and individual differences, which can modulate performance, should be taken into account in such approaches.

Up by upwest: Is slope like north?

Terrain slope can be used to encode the location of a goal. However, this directional information may be encoded using a conceptual north (i.e., invariantly with respect to the environment), or in an observer-relative fashion (i.e., varying depending on the direction one faces when learning the goal). This study examines which representation is used, whether the sensory modality in which slope is encoded (visual, kinaesthetic, or both) influences representations, and whether use of slope varies for men and women. In a square room, with a sloped floor explicitly pointed out as the only useful cue, participants encoded the corner in which a goal was hidden. Without direct sensory access to slope cues, participants used a dial to point to the goal. For each trial, the goal was hidden uphill or downhill, and the participants were informed whether they faced uphill or downhill when pointing. In support of observer-relative representations, participants pointed more accurately and quickly when facing concordantly with the hiding position. There was no effect of sensory modality, providing support for functional equivalence. Sex did not interact with the findings on modality or reference frame, but spatial measures correlated with success on the slope task differently for each sex.

The Role of Landmark Modality and Familiarity in Human Wayfinding

What characteristics constitute a “helpful” landmark for wayfinding and how are they represented in the human brain? Experiment 1 compared recognition and wayfinding performance for visual, verbal, and acoustic landmarks (animals) learned in our virtual environment SQUARELAND. Experiment 2 investigated landmark semantics, namely, famous versus unfamiliar buildings. The results showed that, first, the best recognition performance was observed for words (verbal condition) followed by sounds. Performance was worst for the pictorial landmark information. In the wayfinding phase, a similar level of performance was observed for all three modalities. Second, famous buildings were better recognized than unfamiliar ones, indicating a semantic influence. We conclude that nonvisual information may successfully constitute a landmark and discuss this within the context of current research on landmarks and human wayfinding.

Intrinsic frames of reference in haptic spatial learning

It has been proposed that spatial reference frames with which object locations are specified in memory are intrinsic to a to-be-remembered spatial layout (intrinsic reference theory). Although this theory has been supported by accumulating evidence, it has only been collected from paradigms in which the entire spatial layout was simultaneously visible to observers. The present study was designed to examine the generality of the theory by investigating whether the geometric structure of a spatial layout (bilateral symmetry) influences selection of spatial reference frames when object locations are sequentially learned through haptic exploration. In two experiments, participants learned the spatial layout solely by touch and performed judgments of relative direction among objects using their spatial memories. Results indicated that the geometric structure can provide a spatial cue for establishing reference frames as long as it is accentuated by explicit instructions (Experiment 1) or alignment with an egocentric orientation (Experiment 2). These results are entirely consistent with those from previous studies in which spatial information was encoded through simultaneous viewing of all object locations, suggesting that the intrinsic reference theory is not specific to a type of spatial memory acquired by the particular learning method but instead generalizes to spatial memories learned through a variety of encoding conditions. In particular, the present findings suggest that spatial memories that follow the intrinsic reference theory function equivalently regardless of the modality in which spatial information is encoded.

Real world navigation independence in the early blind correlates with differential brain activity associated with virtual navigation

Navigating is a complex cognitive task that places high demands on spatial abilities, particularly in the absence of sight. Significant advances have been made in identifying the neural correlates associated with various aspects of this skill; however, how the brain is able to navigate in the absence of visual experience remains poorly understood. Furthermore, how neural network activity relates to the wide variability in navigational independence and skill in the blind population is also unknown. Using functional magnetic resonance imaging, we investigated the neural correlates of audio-based navigation within a large scale, indoor virtual environment in early profoundly blind participants with differing levels of spatial navigation independence (assessed by the Santa Barbara Sense of Direction scale). Performing path integration tasks in the virtual environment was associated with activation within areas of a core network implicated in navigation. Furthermore, we found a positive relationship between Santa Barbara Sense of Direction scores and activation within right temporal parietal junction during the planning and execution phases of the task. These findings suggest that differential navigational ability in the blind may be related to the utilization of different brain network structures. Further characterization of the factors that influence network activity may have important implications regarding how this skill is taught in the blind community.