Active and passive scene recognition across views

Path integration, rather than being suppressed, is used to update spatial views in familiar environments with constantly available landmarks

This project tested three hypotheses conceptualizing the interaction between path integration based on self-motion and piloting based on landmarks in a familiar environment with persistent landmarks. The first hypothesis posits that path integration functions automatically, as in environments lacking persistent landmarks (environment-independent hypothesis). The second hypothesis suggests that persistent landmarks suppress path integration (suppression hypothesis). The third hypothesis proposes that path integration updates the spatial views of the environment (updating-spatial-views hypothesis). Participants learned a specific object's location. Subsequently, they undertook an outbound path originating from the object and then indicated the object's location (homing). In Experiments 1&1b, there were landmarks throughout the first 9 trials. On some later trials, the landmarks were presented during the outbound path but unexpectedly removed during homing (catch trials). On the last trials, there were no landmarks throughout (baseline trials). Experiments 2-3 were similar but added two identical objects (the original one and a rotated distractor) during homing on the catch and baseline trials. Experiment 4 replaced two identical objects with two groups of landmarks. The results showed that in Experiments 1&1b, homing angular error on the first catch trial was significantly larger than the matched baseline trial, undermining the environment-independent hypothesis. Conversely, in Experiment 2-4, the proportion of participants who recognized the original object or landmarks was similar between the first catch and the matched baseline trial, favoring the updating-spatial-views hypothesis over the suppression hypothesis. Therefore, while mismatches between updated spatial views and actual views of unexpected removal of landmarks impair homing performance, the updated spatial views help eliminate ambiguous targets or landmarks within the familiar environment.

The psychophysics of human three-dimensional active visuospatial problem-solving

Our understanding of how visual systems detect, analyze and interpret visual stimuli has advanced greatly. However, the visual systems of all animals do much more; they enable visual behaviours. How well the visual system performs while interacting with the visual environment and how vision is used in the real world is far from fully understood, especially in humans. It has been suggested that comparison is the most primitive of psychophysical tasks. Thus, as a probe into these active visual behaviours, we use a same-different task: Are two physical 3D objects visually the same? This task is a fundamental cognitive ability. We pose this question to human subjects who are free to move about and examine two real objects in a physical 3D space. The experimental design is such that all behaviours are directed to viewpoint change. Without any training, our participants achieved a mean accuracy of 93.82%. No learning effect was observed on accuracy after many trials, but some effect was seen for response time, number of fixations and extent of head movement. Our probe task, even though easily executed at high-performance levels, uncovered a surprising variety of complex strategies for viewpoint control, suggesting that solutions were developed dynamically and deployed in a seemingly directed hypothesize-and-test manner tailored to the specific task. Subjects need not acquire task-specific knowledge; instead, they formulate effective solutions right from the outset, and as they engage in a series of attempts, those solutions progressively refine, becoming more efficient without compromising accuracy.

Objects are selected for attention based upon meaning during passive scene viewing

While object meaning has been demonstrated to guide attention during active scene viewing and object salience guides attention during passive viewing, it is unknown whether object meaning predicts attention in passive viewing tasks and whether attention during passive viewing is more strongly related to meaning or salience. To answer this question, we used a mixed modeling approach where we computed the average meaning and physical salience of objects in scenes while statistically controlling for the roles of object size and eccentricity. Using eye-movement data from aesthetic judgment and memorization tasks, we then tested whether fixations are more likely to land on high-meaning objects than low-meaning objects while controlling for object salience, size, and eccentricity. The results demonstrated that fixations are more likely to be directed to high meaning objects than low meaning objects regardless of these other factors. Further analyses revealed that fixation durations were positively associated with object meaning irrespective of the other object properties. Overall, these findings provide the first evidence that objects are, in part, selected by meaning for attentional selection during passive scene viewing.

Virtual Reality for Spatial Navigation

Immersive virtual reality (VR) allows its users to experience physical space in a non-physical world. It has developed into a powerful research tool to investigate the neural basis of human spatial navigation as an embodied experience. The task of wayfinding can be carried out by using a wide range of strategies, leading to the recruitment of various sensory modalities and brain areas in real-life scenarios. While traditional desktop-based VR setups primarily focus on vision-based navigation, immersive VR setups, especially mobile variants, can efficiently account for motor processes that constitute locomotion in the physical world, such as head-turning and walking. When used in combination with mobile neuroimaging methods, immersive VR affords a natural mode of locomotion and high immersion in experimental settings, designing an embodied spatial experience. This in turn facilitates ecologically valid investigation of the neural underpinnings of spatial navigation.

VR for Cognition and Memory

This chapter will provide a review of research into human cognition through the lens of VR-based paradigms for studying memory. Emphasis is placed on why VR increases the ecological validity of memory research and the implications of such enhancements.

Merging familiar and new senses to perceive and act in space

Our experience of the world seems to unfold seamlessly in a unitary 3D space. For this to be possible, the brain has to merge many disparate cognitive representations and sensory inputs. How does it do so? I discuss work on two key combination problems: coordinating multiple frames of reference (e.g. egocentric and allocentric), and coordinating multiple sensory signals (e.g. visual and proprioceptive). I focus on two populations whose spatial processing we can observe at a crucial stage of being configured and optimised: children, whose spatial abilities are still developing significantly, and naïve adults learning new spatial skills, such as sensing distance using auditory cues. The work uses a model-based approach to compare participants’ behaviour with the predictions of alternative information processing models. This lets us see when and how—during development, and with experience—the perceptual-cognitive computations underpinning our experiences in space change. I discuss progress on understanding the limits of effective spatial computation for perception and action, and how lessons from the developing spatial cognitive system can inform approaches to augmenting human abilities with new sensory signals provided by technology.

Alignment Effects in Spatial Perspective Taking from an External Vantage Point

In three experiments, we examined, using a perceptual task, the difficulties of spatial perspective taking. Participants imagined adopting perspectives around a table and pointed from them towards the positions of a target. Depending on the condition, the scene was presented on a virtual screen in Virtual Reality or projected on an actual screen in the real world (Experiment 1), or viewed as immediate in Virtual Reality (Experiment 2). Furthermore, participants pointed with their arm (Experiments 1 and 2) vs. a joystick (Experiment 3). Results showed a greater alignment effect (i.e., a larger difference in performance between trials with imagined perspectives that were aligned vs. misaligned with the orientation of the participant) when executing the task in a virtual rather than in the real environment, suggesting that visual access to body information and room geometry, which is typically lacking in Virtual Reality, influences perspective taking performance. The alignment effect was equal across the Virtual Reality conditions of Experiment 1 and Experiment 2, suggesting that being an internal (compared to an external) observer to the scene induces no additional difficulties for perspective taking. Equal alignment effects were also found when pointing with the arm vs. a joystick, indicating that a body-dependent response mode such as pointing with the arm creates no further difficulties for reasoning from imagined perspectives.

The relationship between mental and physical space and its impact on topographical disorientation

We generate mental representations of space to facilitate our ability to remember things and navigate our environment. Many studies implicitly assume that these representations simply reflect the environments that they represent without considering other factors that influence the extent to which this is the case. Here, we bring together findings from cognitive psychology, environmental psychology, geography, urban planning, and neuroscience to discuss how internalizing the environment involves a complex interplay between bottom-up and top-down mental processes and depends on key characteristics of the physical environment itself. We describe how mental space is structured, the ways in which mental and physical space converge and diverge, and the disparate but complementary techniques used to assess these relationships. Finally, we contextualize this knowledge in the clinical populations affected by acquired and developmental topographical disorientation, exploring mechanisms that cause these patients to get lost in familiar surroundings.

Updating perception and action across real-world viewpoint changes

A growing body of research suggests that performing actions can distort the perception of size, distance, and other visual information. These distortions have been observed under a variety of circumstances, and appear to persist in both perception and memory. However, it is unclear whether these distortions persist as observers move to new viewpoints. To address this issue, the present study assessed whether action-specific distortions persist across changes in viewpoint. Participants viewed an object that was projected onto a table, then reached for it with their index finger or a reach-extending tool. After reaching for the object, participants remained stationary or moved to a new viewpoint, then estimated the object's distance from their current viewpoint. When participants remained stationary, using a reach-extending tool led them to report shorter distance estimates. However, when participants moved to a new viewpoint, these distortions were eliminated. Similar effects were observed when participants produced different types of movement, including when participants rotated in place, moved to a new location, or simply walked in place. Together, these findings suggest that action-specific distortions are eliminated when observers move and perform other actions.

Mental representations of recently learned nested environments

Two experiments investigated the mental representations of objects' location in a virtual nested environment. In Experiment 1, participants learned the locations of objects (buildings or related accessories) in an exterior environment and then learned the locations of objects inside one of the centrally located buildings (interior environment). Participants completed judgments of relative direction in which the imagined heading was established by pairs of objects from the interior environment and the target was one of the objects in the exterior environment. Performance was best for the imagined heading and allocentric target direction parallel to the learning heading of the exterior environment, but the effect of allocentric target direction was only significant for the imagined headings aligned with the reference axes of both environments; in addition, performance was best along the front-back egocentric axis (parallel to the imagined heading). Experiment 2 used the same learning procedure. After learning, the viewpoint was moved from the exterior environment along a smooth path into a side entrance of the building/interior environment. There participants saw the array of interior objects in the orientation consistent with their movement (correct cue), the array of objects in an orientation inconsistent with their movement (misleading cue), or no array of objects (no cue), and then pointed to objects in the exterior environment. Pointing performance was best for the correct-cue condition. Collectively the results indicated that memories of nested spaces are segregated by spatial conceptual level, and that spatial relations between levels are specified in terms of the dominant reference directions.

The Difficulty of Effectively Using Allocentric Prior Information in a Spatial Recall Task

Prior information represents the long-term statistical structure of an environment. For example, colds develop more often than throat cancer, making the former a more likely diagnosis for a sore throat. There is ample evidence for effective use of prior information during a variety of perceptual tasks, including the ability to recall locations using an egocentric (self-based) frame. However, it is not yet known if people can use prior information effectively when using an allocentric (world-based) frame. Forty-eight adults were shown sixty sets of three target locations in a sparse virtual environment with three beacons. The targets were drawn from one of four prior distributions. They were then asked to point to the targets after a delay and a change in perspective. While searches were biased towards the beacons, we did not find any evidence that participants successfully exploited the prior distributions of targets. These results suggest that allocentric reasoning does not conform to normative Bayesian models: we saw no evidence for use of priors in our cognitively-complex (allocentric) task, unlike in previous, simpler (egocentric) recall tasks. It is possible that this reflects the high biological cost of processing precise allocentric information.

Egocentric and Allocentric Spatial Memory in Korsakoff's Amnesia

The goal of the present study was to investigate spatial memory in a group of patients with amnesia due to Korsakoff’s syndrome (KS). We used a virtual spatial memory task that allowed us to separate the use of egocentric and allocentric spatial reference frames to determine object locations. Research investigating the ability of patients with Korsakoff’s amnesia to use different reference frames is scarce and it remains unclear whether these patients are impaired in using ego- and allocentric reference frames to the same extent. Twenty Korsakoff patients and 24 matched controls watched an animation of a bird flying in one of three trees standing in a virtual environment. After the bird disappeared, the camera turned around, by which the trees were briefly out of sight and then turned back to the center of the environment. Participants were asked in which tree the bird was hiding. In half of the trials, a landmark was shown. Half of the trials required an immediate response whereas in the other half a delay of 10 s was present. Patients performed significantly worse than controls. For all participants trials with a landmark were easier than without a landmark and trials without a delay were easier than with a delay. While controls were above chance on all trials patients were at chance in allocentric trials without a landmark present and with a memory delay. Patients showed no difference in the ego- and the allocentric condition. Together the findings suggest that despite the amnesia, spatial memory and especially the use of ego- and allocentric reference frames in Korsakoff patients are spared.

Virtual Walking Sensation by Prerecorded Oscillating Optic Flow and Synchronous Foot Vibration

This article reports the first psychological evidence that the combination of oscillating optic flow and synchronous foot vibration evokes a walking sensation. In this study, we first captured a walker's first-person-view scenes with footstep timings. Participants observed the naturally oscillating scenes on a head-mounted display with vibrations on their feet and rated walking-related sensations using a Visual Analogue Scale. They perceived stronger sensations of self-motion, walking, leg action, and telepresence from the oscillating visual flow with foot vibrations than with randomized-timing vibrations or without vibrations. The artificial delay of foot vibrations with respect to the scenes diminished the walking-related sensations. These results suggest that the oscillating visual scenes and synchronous foot vibrations are effective for creating virtual walking sensations.

No single, stable 3D representation can explain pointing biases in a spatial updating task

People are able to keep track of objects as they navigate through space, even when objects are out of sight. This requires some kind of representation of the scene and of the observer’s location but the form this might take is debated. We tested the accuracy and reliability of observers’ estimates of the visual direction of previously-viewed targets. Participants viewed four objects from one location, with binocular vision and small head movements then, without any further sight of the targets, they walked to another location and pointed towards them. All conditions were tested in an immersive virtual environment and some were also carried out in a real scene. Participants made large, consistent pointing errors that are poorly explained by any stable 3D representation. Any explanation based on a 3D representation would have to posit a different layout of the remembered scene depending on the orientation of the obscuring wall at the moment the participant points. Our data show that the mechanisms for updating visual direction of unseen targets are not based on a stable 3D model of the scene, even a distorted one.

Alternative spin on phylogenetically inherited spatial reference frames

People make use of different frames of reference (north-south; left-right) to talk about space. To explore the cognitive capacity that children bring to learning spatial language, Haun, Rapold, Call, Janzen, and Levinson (2006) examined children's ability to notice and abstract invariant frames of references across instances. They found that 4-year-olds and non-human great apes often noticed environment-defined allocentric relations and not body-defined egocentric ones, leading them to conclude that preschoolers are ready to learn environment-defined terms (e.g. "uphill"), but not body-defined ones (e.g., "left"). However, such a conclusion may be premature. In four new experiments we demonstrate that the previous findings could be an artifact of specific task constraints. With minor experiment modifications, similar-aged children readily noticed egocentric relations. Reviewing additional research, we provide an account of what makes acquiring frames of reference easy or difficult, and why full mastery of terms like "left" and "right" may take many years under normal circumstances.

Flexibility in adopting relative frames of reference in dorsal and lateral settings

The relative frame of reference (FoR) is used to describe spatial relations between two objects from an observer's perspective. Standard, frontal referencing situations with objects located in the observer's visual field afford three well-established variants: translation, reflection, and rotation. Here, we focus on references in non-standard situations with objects located at the back or at the side of an observer (dorsal and lateral, respectively). We scrutinise the consistency assumption, which was introduced to infer the covert strategy used in dorsal tasks from an ambiguous overt response: that, when confronted with a non-standard situation, people adopt a strategy consistent with how they construct the relative FoR in frontal situations. Lateral tasks enable us to disentangle the ambiguous response. The results of a study in Norway and Germany support the consistency assumption in part: Nearly all participants with a preference for translation in frontal tasks applied translation in lateral tasks, and some participants with a preference for reflection in frontal tasks turned towards the objects before applying reflection in lateral tasks. Most other participants with a preference for reflection in frontal tasks, however, switched to translation in lateral tasks. The latter may be due to a specific affordance of the lateral arrangements, which invite translation as the easier strategy compared to the alternative derived from reflection. Our findings indicate that people do not apply their preferred variant of the relative FoR to all kinds of situations, but rather flexibly adapt their strategy when it is more convenient to do so.

Don't worry, be active: how to facilitate the detection of errors in immersive virtual environments

The current research aims to study the link between the type of vision experienced in a collaborative immersive virtual environment (active vs. multiple passive), the type of error one looks for during a cooperative multi-user exploration of a design project (affordance vs. perceptual violations), and the type of setting in which multi-user perform (field in Experiment 1 vs. laboratory in Experiment 2). The relevance of this link is backed by the lack of conclusive evidence on an active vs. passive vision advantage in cooperative search tasks within software based on immersive virtual reality (IVR). Using a yoking paradigm based on the mixed usage of simultaneous active and multiple passive viewings, we found that the likelihood of error detection in a complex 3D environment was characterized by an active vs. multi-passive viewing advantage depending on: (1) the degree of knowledge dependence of the type of error the passive/active observers were looking for (low for perceptual violations, vs. high for affordance violations), as the advantage tended to manifest itself irrespectively from the setting for affordance, but not for perceptual violations; and (2) the degree of social desirability possibly induced by the setting in which the task was performed, as the advantage occurred irrespectively from the type of error in the laboratory (Experiment 2) but not in the field (Experiment 1) setting. Results are relevant to future development of cooperative software based on IVR used for supporting the design review. A multi-user design review experience in which designers, engineers and end-users all cooperate actively within the IVR wearing their own head mounted display, seems more suitable for the detection of relevant errors than standard systems characterized by a mixed usage of active and passive viewing.

Face recognition of full-bodied avatars by active observers in a virtual environment

Viewing faces in motion or attached to a body instead of isolated static faces improves their subsequent recognition. Here we enhanced the ecological validity of face encoding by having observers physically moving in a virtual room populated by life-size avatars. We compared the recognition performance of this active group to two control groups. The first control group watched a passive reenactment of the visual experience of the active group. The second control group saw static screenshots of the avatars. All groups performed the same old/new recognition task after learning. Half of the learned faces were shown at test in an orientation close to that experienced during learning while the others were viewed from a new viewing angle. All observers found novel views more difficult to recognize than familiar ones. Overall, the active group performed better than both other groups. Furthermore, the group learning faces from static images was the only one to be at chance level in the novel-view condition. These findings suggest that active exploration combined with a dynamic experience of the faces to learn allow for more robust face recognition and point out the value of such techniques for integrating facial visual information and enhancing recognition from novel viewpoints.

Does the Motor System Facilitate Spatial Imagery?

Abstract. Recent studies have ascertained a link between the motor system and imagery in children. A motor effect on imagery is demonstrated by the influence of stimuli-related movement constraints (i. e., constraints defined by the musculoskeletal system) on mental rotation, or by interference effects due to participants’ own body movements or body postures. This link is usually seen as qualitatively different or stronger in children as opposed to adults. In the present research, we put this interpretation to further scrutiny using a new paradigm: In a motor condition we asked our participants (kindergartners and third-graders) to manually rotate a circular board with a covered picture on it. This condition was compared with a perceptual condition where the board was rotated by an experimenter. Additionally, in a pure imagery condition, children were instructed to merely imagine the rotation of the board. The children’s task was to mark the presumed end position of a salient detail of the respective picture. The children’s performance was clearly the worst in the pure imagery condition. However, contrary to what embodiment theories would suggest, there was no difference in participants’ performance between the active rotation (i. e., motor) and the passive rotation (i. e., perception) condition. Control experiments revealed that this was also the case when, in the perception condition, gaze shifting was controlled for and when the board was rotated mechanically rather than by the experimenter. Our findings indicate that young children depend heavily on external support when imagining physical events. Furthermore, they indicate that motor-assisted imagery is not generally superior to perceptually driven dynamic imagery.

Representing Spatial Layout According to Intrinsic Frames of Reference: Limitations From Position Regularity and Instructions

Mou and McNamara have suggested that object locations are represented according to intrinsic reference frames. In three experiments, we investigated the limitations of intrinsic reference frames as a mean to represent object locations in spatial memory. Participants learned the locations of seven or eight common objects in a rectangular room and then made judgments of relative direction based on their memory of the layout. The results of all experiments showed that when all objects were positioned regularly, judgments of relative direction were faster or more accurate for novel headings that were aligned with the primary intrinsic structure than for other novel headings; however, when one irregularly positioned object was added to the layout, this advantage was eliminated. The experiments further indicated that with a single view at study, participants could represent the layout from either an egocentric orientation or a different orientation, according to experimental instructions. Together, these results suggest that environmental reference frames and intrinsic axes can influence performance for novel headings, but their role in spatial memory depends on egocentric experience, layout regularity, and instructions.

Peripersonal versus extrapersonal visual scene information for egocentric direction and position perception

When perceiving the visual environment, people simultaneously perceive their own direction and position in the environment (i.e., egocentric spatial perception). This study investigated what visual information in a scene is necessary for egocentric spatial perceptions. In two perception tasks (the egocentric direction and position perception tasks), observers viewed two static road images presented sequentially. In Experiment 1, the critical manipulation involved an occluded region in the road image, an extrapersonal region (far-occlusion) and a peripersonal region (near-occlusion). Egocentric direction perception was worse in the far-occlusion condition than in the no-occlusion condition, and egocentric position perceptions were worse in the far- and near-occlusion conditions than in the no-occlusion condition. In Experiment 2, we conducted the same tasks manipulating the observers' gaze location in a scene-an extrapersonal region (far-gaze), a peripersonal region (near-gaze) and the intermediate region between the former two (middle-gaze). Egocentric direction perception performance was the best in the far-gaze condition, and egocentric position perception performances were not different among gaze location conditions. These results suggest that egocentric direction perception is based on fine visual information about the extrapersonal region in a road landscape, and egocentric position perception is based on information about the entire visual scene.

Medial Prefrontal-Medial Temporal Theta Phase Coupling in Dynamic Spatial Imagery

Hippocampal-medial prefrontal interactions are thought to play a crucial role in mental simulation. Notably, the frontal midline/medial pFC (mPFC) theta rhythm in humans has been linked to introspective thought and working memory. In parallel, theta rhythms have been proposed to coordinate processing in the medial temporal cortex, retrosplenial cortex (RSc), and parietal cortex during the movement of viewpoint in imagery, extending their association with physical movement in rodent models. Here, we used noninvasive whole-head MEG to investigate theta oscillatory power and phase-locking during the 18-sec postencoding delay period of a spatial working memory task, in which participants imagined previously learned object sequences either on a blank background (object maintenance), from a first-person viewpoint in a scene (static imagery), or moving along a path past the objects (dynamic imagery). We found increases in 4- to 7-Hz theta power in mPFC when comparing the delay period with a preencoding baseline. We then examined whether the mPFC theta rhythm was phase-coupled with ongoing theta oscillations elsewhere in the brain. The same mPFC region showed significantly higher theta phase coupling with the posterior medial temporal lobe/RSc for dynamic imagery versus either object maintenance or static imagery. mPFC theta phase coupling was not observed with any other brain region. These results implicate oscillatory coupling between mPFC and medial temporal lobe/RSc theta rhythms in the dynamic mental exploration of imagined scenes.

Building a cognitive map by assembling multiple path integration systems

Path integration and cognitive mapping are two of the most important mechanisms for navigation. Path integration is a primitive navigation system which computes a homing vector based on an animal's self-motion estimation, while cognitive map is an advanced spatial representation containing richer spatial information about the environment that is persistent and can be used to guide flexible navigation to multiple locations. Most theories of navigation conceptualize them as two distinctive, independent mechanisms, although the path integration system may provide useful information for the integration of cognitive maps. This paper demonstrates a fundamentally different scenario, where a cognitive map is constructed in three simple steps by assembling multiple path integrators and extending their basic features. The fact that a collection of path integration systems can be turned into a cognitive map suggests the possibility that cognitive maps may have evolved directly from the path integration system.

Implicit coding of location and direction in a familiar, real-world "vista" space

Keeping oriented in the surrounding space requires an accurate representation of one's spatial position and facing direction. Although previous studies provided evidence of specific spatial codes for position and direction within room-sized and large-scale navigational environments, little is known about the mechanisms by which these spatial quantities are represented in a real small-scale environment. Here, we used two spatial tasks requiring participants to encode their own position and facing direction on a series of pictures taken from a familiar circular square. Crucially, directions and positions were incidentally manipulated, so that when participants were required to encode their current position in the square, the perceived direction across consecutive trials was the same, and vice versa. We found a behavioral advantage (priming effect: reduced reaction times and increased accuracy) for repeated directions and positions, even in the absence of any explicit demand to encode either of them. The advantage was higher for repeated directions, indicating that representation of one's own direction is more automatic than representation of one's own location. Furthermore, priming effects were partially mediated by gender: females (but not males) showed a stronger priming effect for repeated directions than for repeated positions. Finally, although priming effects were not linearly related to the physical distances between consecutive positions and directions, they revealed a rough preservation of real-world distance relationships.

The Effects of Visual Discriminability and Rotation Angle on 30-Month-Olds' Search Performance in Spatial Rotation Tasks

Tracking objects that are hidden and then moved is a crucial ability related to object permanence, which develops across several stages in early childhood. In spatial rotation tasks, children observe a target object that is hidden in one of two or more containers before the containers are rotated around a fixed axis. Usually, 30-month-olds fail to find the hidden object after it was rotated by 180°. We examined whether visual discriminability of the containers improves 30-month-olds’ success in this task and whether children perform better after 90° than after 180° rotations. Two potential hiding containers with same or different colors were placed on a board that was rotated by 90° or 180° in a within-subjects design. Children (N = 29) performed above chance level in all four conditions. Their overall success in finding the object did not improve by differently colored containers. However, different colors prevented children from showing an inhibition bias in 90° rotations, that is, choosing the empty container more often when it was located close to them than when it was farther away: This bias emerged in the same colors condition but not in the different colors condition. Results are discussed in view of particular challenges that might facilitate or deteriorate spatial rotation tasks for young children.

Touching and Hearing Unseen Objects: Multisensory Effects on Scene Recognition

In three experiments, we investigated the influence of object-specific sounds on haptic scene recognition without vision. Blindfolded participants had to recognize, through touch, spatial scenes comprising six objects that were placed on a round platform. Critically, in half of the trials, object-specific sounds were played when objects were touched (bimodal condition), while sounds were turned off in the other half of the trials (unimodal condition). After first exploring the scene, two objects were swapped and the task was to report, which of the objects swapped positions. In Experiment 1, geometrical objects and simple sounds were used, while in Experiment 2, the objects comprised toy animals that were matched with semantically compatible animal sounds. In Experiment 3, we replicated Experiment 1, but now a tactile-auditory object identification task preceded the experiment in which the participants learned to identify the objects based on tactile and auditory input. For each experiment, the results revealed a significant performance increase only after the switch from bimodal to unimodal. Thus, it appears that the release of bimodal identification, from audio-tactile to tactile-only produces a benefit that is not achieved when having the reversed order in which sound was added after having experience with haptic-only. We conclude that task-related factors other than mere bimodal identification cause the facilitation when switching from bimodal to unimodal conditions.

Scene Recognition following Locomotion around a Scene

Effects of locomotion on scene-recognition reaction time (RT) and accuracy were studied. In experiment 1, observers memorized an 11-object scene and made scene-recognition judgments on subsequently presented scenes from the encoded view or different views (ie scenes were rotated or observers moved around the scene, both from 40° to 360°). In experiment 2, observers viewed different 5-object scenes on each trial and made scene-recognition judgments from the encoded view or after moving around the scene, from 36° to 180°. Across experiments, scene-recognition RT increased (in experiment 2 accuracy decreased) with angular distance between encoded and judged views, regardless of how the viewpoint changes occurred. The findings raise questions about conditions in which locomotion produces spatially updated representations of scenes.

Transformations of Visuospatial Images

Transformations of visuospatial mental images are important for action, navigation, and reasoning. They depend on representations in multiple spatial reference frames, implemented in the posterior parietal cortex and other brain regions. The multiple systems framework proposes that different transformations can be distinguished in terms of which spatial reference frame is updated. In an object-based transformation, the reference frame of an object moves relative to those of the observer and the environment. In a perspective transformation, the observer's egocentric reference frame moves relative to those of the environment and of salient objects. These two types of spatial reference frame updating rely on distinct neural processing resources in the parietal, occipital, and temporal cortex. They are characterized by different behavioral patterns and unique individual differences. Both object-based transformations and perspective transformations interact with posterior frontal cortical regions subserving the simulation of body movements. These interactions indicate that multiple systems coordinate to support everyday spatial problem solving.

Environmental layout complexity affects neural activity during navigation in humans

Navigating large-scale surroundings is a fundamental ability. In humans, it is commonly assumed that navigational performance is affected by individual differences, such as age, sex, and cognitive strategies adopted for orientation. We recently showed that the layout of the environment itself also influences how well people are able to find their way within it, yet it remains unclear whether differences in environmental complexity are associated with changes in brain activity during navigation. We used functional magnetic resonance imaging to investigate how the brain responds to a change in environmental complexity by asking participants to perform a navigation task in two large-scale virtual environments that differed solely in interconnection density, a measure of complexity defined as the average number of directional choices at decision points. The results showed that navigation in the simpler, less interconnected environment was faster and more accurate relative to the complex environment, and such performance was associated with increased activity in a number of brain areas (i.e. precuneus, retrosplenial cortex, and hippocampus) known to be involved in mental imagery, navigation, and memory. These findings provide novel evidence that environmental complexity not only affects navigational behaviour, but also modulates activity in brain regions that are important for successful orientation and navigation.

Children's Spatial Representations: 3- and 4-Year-Olds are Affected by Irrelevant Peripheral References

Children as young as 3 years can remember an object’s location within an arrangement and can retrieve it from a novel viewpoint (Nardini et al., 2006). However, this ability is impaired if the arrangement is rotated to compensate for the novel viewpoint, or, if the arrangement is rotated and children stand still. There are two dominant explanations for this phenomenon: self-motion induces an automatic spatial updating process which is beneficial if children move around the arrangement, but misleading if the children’s movement is matched by the arrangement and not activated if children stand still and only the arrangement is moved (see spatial updating; Simons and Wang, 1998). Another explanation concerns reference frames: spatial representations might depend on peripheral spatial relations concerning the surrounding room instead on proximal relations within the arrangement, even if these proximal relations are sufficient or more informative. To evaluate these possibilities, we rotated children (N = 120) aged between 3 and 6 years with an occluded arrangement. When the arrangement was in misalignment to the surrounding room, 3- and 4-year-olds’ spatial memory was impaired and 5-year-olds’ was lightly impaired suggesting that they relied on peripheral references of the surrounding room for retrieval. In contrast, 6-years-olds’ spatial representation seemed robust against misalignment indicating a successful integration of spatial representations.

Role of the human retrosplenial cortex/parieto-occipital sulcus in perspective priming

The ability to imagine the world from a different viewpoint is a fundamental competence for spatial reorientation and for imagining what another individual sees in the environment. Here, we investigated the neural bases of such an ability using functional magnetic resonance imaging. Healthy participants detected target displacements across consecutive views of a familiar virtual room, either from the perspective of an avatar (primed condition) or in the absence of such a prime (unprimed condition). In the primed condition, the perspective at test always corresponded to the avatar's perspective, while in the unprimed condition it was randomly chosen as 0, 45 or 135deg of viewpoint rotation. We observed a behavioral advantage in performing a perspective transformation during the primed condition as compared to an equivalent amount of unprimed perspective change. Although many cortical regions (dorsal parietal, parieto-temporo-occipital junction, precuneus and retrosplenial cortex/parieto-occipital sulcus or RSC/POS) were involved in encoding and retrieving target location from different perspectives and were modulated by the amount of viewpoint rotation, the RSC/POS was the only area showing decreased activity in the primed as compared to the unprimed condition, suggesting that this region anticipates the upcoming perspective change. The retrosplenial cortex/parieto-occipital sulcus appears to play a special role in the allocentric coding of heading directions.

Visual memory performance for color depends on spatiotemporal context

Performance on visual short-term memory for features has been known to depend on stimulus complexity, spatial layout, and feature context. However, with few exceptions, memory capacity has been measured for abruptly appearing, single-instance displays. In everyday life, objects often have a spatiotemporal history as they or the observer move around. In three experiments, we investigated the effect of spatiotemporal history on explicit memory for color. Observers saw a memory display emerge from behind a wall, after which it disappeared again. The test display then emerged from either the same side as the memory display or the opposite side. In the first two experiments, memory improved for intermediate set sizes when the test display emerged in the same way as the memory display. A third experiment then showed that the benefit was tied to the original motion trajectory and not to the display object per se. The results indicate that memory for color is embedded in a richer episodic context that includes the spatiotemporal history of the display.

Spatial scale, rather than nature of task or locomotion, modulates the spatial reference frame of attention

Visuospatial attention is strongly biased to locations that had frequently contained a search target before. However, the function of this bias depends on the reference frame in which attended locations are coded. Previous research has shown a striking difference between tasks administered on a computer monitor and those administered in a large environment, with the former inducing viewer-centered learning and the latter environment-centered learning. Why does environment-centered learning fail on a computer? Here, we tested 3 possibilities: differences in spatial scale, the nature of task, and locomotion may each influence the reference frame of attention. Participants searched for a target on a monitor placed flat on a stand. On each trial, they stood at a different location around the monitor. The target was frequently located in a fixed area of the monitor, but changes in participants' perspective rendered this area random relative to the participants. Under incidental learning conditions, participants failed to acquire environment-centered learning even when (a) the task and display resembled those of a large-scale task and (b) the search task required locomotion. The difficulty in inducing environment-centered learning on a computer underscores the egocentric nature of visual attention. It supports the idea that spatial scale modulates the reference frame of attention.

Visual, haptic and bimodal scene perception: evidence for a unitary representation

Participants studied seven meaningful scene-regions bordered by removable boundaries (30s each). In Experiment 1 (N = 80) participants used visual or haptic exploration and then minutes later, reconstructed boundary position using the same or the alternate modality. Participants in all groups shifted boundary placement outward (boundary extension), but visual study yielded the greater error. Critically, this modality-specific difference in boundary extension transferred without cost in the cross-modal conditions, suggesting a functionally unitary scene representation. In Experiment 2 (N = 20), bimodal study led to boundary extension that did not differ from haptic exploration alone, suggesting that bimodal spatial memory was constrained by the more "conservative" haptic modality. In Experiment 3 (N = 20), as in picture studies, boundary memory was tested 30s after viewing each scene-region and as with pictures, boundary extension still occurred. Results suggest that scene representation is organized around an amodal spatial core that organizes bottom-up information from multiple modalities in combination with top-down expectations about the surrounding world.

Deficits in egocentric-updating and spatial context memory in a case of developmental amnesia

Patients with developmental amnesia usually suffer from both episodic and spatial memory deficits. DM, a developmental amnesic, was impaired in her ability to process self-motion (i.e., idiothetic) information while her ability to process external stable landmarks (i.e., allothetic) was preserved when no self-motion processing was required. On a naturalistic and incidental episodic task, DM was severely and predictably impaired on both free and cued recall tasks. Interestingly, when cued, she was more impaired at recalling spatial context than factual or temporal information. Theoretical implications of that co-occurrence of deficits and those dissociations are discussed and testable cerebral hypothesis are proposed.

Learning with interactive computer graphics in the undergraduate neuroscience classroom

Instruction of neuroanatomy depends on graphical representation and extended self-study. As a consequence, computer-based learning environments that incorporate interactive graphics should facilitate instruction in this area. The present study evaluated such a system in the undergraduate neuroscience classroom. The system used the method of adaptive exploration, in which exploration in a high fidelity graphical environment is integrated with immediate testing and feedback in repeated cycles of learning. The results of this study were that students considered the graphical learning environment to be superior to typical classroom materials used for learning neuroanatomy. Students managed the frequency and duration of study, test, and feedback in an efficient and adaptive manner. For example, the number of tests taken before reaching a minimum test performance of 90 % correct closely approximated the values seen in more regimented experimental studies. There was a wide range of student opinion regarding the choice between a simpler and a more graphically compelling program for learning sectional anatomy. Course outcomes were predicted by individual differences in the use of the software that reflected general work habits of the students, such as the amount of time committed to testing. The results of this introduction into the classroom are highly encouraging for development of computer-based instruction in biomedical disciplines.

Perspective image comprehension depends on both visual and proprioceptive information

Proprioceptive information can supplement visual information in the comprehension of ambiguous perspective images. The importance of proprioceptive information in unambiguous perspective image comprehension is untested, however. We explored the role of proprioception in perspective image comprehension using three experiments in which participants took or imagined taking an upward- or downward-oriented posture and then made judgments about images viewed from below or viewed from above. Participants were faster and more accurate in their judgments when their actual or simulated posture was consistent with the posture implied by the perspective of the image they were judging. These results support a role for proprioception in the comprehension of unambiguous perspective images as well as ambiguous perspective images.

Picturing perspectives: development of perspective-taking abilities in 4- to 8-year-olds

Although the development of perspective taking has been well researched, there is no uniform methodology for assessing this ability across a wide age span when frames of reference conflict. To address this gap, we created scenes of toy photographers taking pictures of layouts of objects from different angles, and presented them to 4- to 8-year-olds (N = 80). Children were asked to choose which one of four pictures could have been taken from a specific viewpoint. Results showed that this new technique confirmed the classic pattern of developmental progress on this kind of spatial skill: (1) 4-year-olds responded near chance level, regardless of layout complexity, (2) there was a growing ability to inhibit egocentric choices around age 6 with layouts of low complexity (one object), (3) performance increased and egocentric responses decreased dramatically around age 7, (4) even at age 8, children still showed considerable individual variability. This perspective taking task can thus be used to address important questions about the supports for early spatial development and the structure of early intellect.

Visual spatial cue use for guiding orientation in two-to-three-year-old children

In spatial development representations of the environment and the use of spatial cues change over time. To date, the influence of individual differences in skills relevant for orientation and navigation has not received much attention. The current study investigated orientation abilities on the basis of visual spatial cues in 2-3-year-old children, and assessed factors that possibly influence spatial task performance. Thirty-month and 35-month-olds performed an on-screen Virtual Reality (VR) orientation task searching for an animated target in the presence of visual self-movement cues and landmark information. Results show that, in contrast to 30-month-old children, 35-month-olds were successful in using visual spatial cues for maintaining orientation. Neither age group benefited from landmarks present in the environment, suggesting that successful task performance relied on the use of optic flow cues, rather than object-to-object relations. Analysis of individual differences revealed that 2-year-olds who were relatively more independent in comparison to their peers, as measured by the daily living skills scale of the parental questionnaire Vineland-Screener were most successful at the orientation task. These results support previous findings indicating that the use of various spatial cues gradually improves during early childhood. Our data show that a developmental transition in spatial cue use can be witnessed within a relatively short period of 5 months only. Furthermore, this study indicates that rather than chronological age, individual differences may play a role in successful use of visual cues for spatial updating in an orientation task. Future studies are necessary to assess the exact nature of these individual differences.

Is navigation in virtual reality with FMRI really navigation?

Identifying the neural mechanisms underlying spatial orientation and navigation has long posed a challenge for researchers. Multiple approaches incorporating a variety of techniques and animal models have been used to address this issue. More recently, virtual navigation has become a popular tool for understanding navigational processes. Although combining this technique with functional imaging can provide important information on many aspects of spatial navigation, it is important to recognize some of the limitations these techniques have for gaining a complete understanding of the neural mechanisms of navigation. Foremost among these is that, when participants perform a virtual navigation task in a scanner, they are lying motionless in a supine position while viewing a video monitor. Here, we provide evidence that spatial orientation and navigation rely to a large extent on locomotion and its accompanying activation of motor, vestibular, and proprioceptive systems. Researchers should therefore consider the impact on the absence of these motion-based systems when interpreting virtual navigation/functional imaging experiments to achieve a more accurate understanding of the mechanisms underlying navigation.