Spatial updating: how the brain keeps track of changing object locations during observer motion

The Human Retrosplenial Cortex and Thalamus Code Head Direction in a Global Reference Frame

Spatial navigation is a multisensory process involving integration of visual and body-based cues. In rodents, head direction (HD) cells, which are most abundant in the thalamus, integrate these cues to code facing direction. Human fMRI studies examining HD coding in virtual environments (VE) have reported effects in retrosplenial complex and (pre-)subiculum, but not the thalamus. Furthermore, HD coding appeared insensitive to global landmarks. These tasks, however, provided only visual cues for orientation, and attending to global landmarks did not benefit task performance. In the present study, participants explored a VE comprising four separate locales, surrounded by four global landmarks. To provide body-based cues, participants wore a head-mounted display so that physical rotations changed facing direction in the VE. During subsequent MRI scanning, subjects saw stationary views of the environment and judged whether their orientation was the same as in the preceding trial. Parameter estimates extracted from retrosplenial cortex and the thalamus revealed significantly reduced BOLD responses when HD was repeated. Moreover, consistent with rodent findings, the signal did not continue to adapt over repetitions of the same HD. These results were supported by a whole-brain analysis showing additional repetition suppression in the precuneus. Together, our findings suggest that: (1) consistent with the rodent literature, the human thalamus may integrate visual and body-based, orientation cues; (2) global reference frame cues can be used to integrate HD across separate individual locales; and (3) immersive training procedures providing full body-based cues may help to elucidate the neural mechanisms supporting spatial navigation.

SIGNIFICANCE STATEMENT

In rodents, head direction (HD) cells signal facing direction in the environment via increased firing when the animal assumes a certain orientation. Distinct brain regions, the retrosplenial cortex (RSC) and thalamus, code for visual and vestibular cues of orientation, respectively. Putative HD signals have been observed in human RSC but not the thalamus, potentially because body-based cues were not provided. Here, participants encoded HD in a novel virtual environment while wearing a head-mounted display to provide body-based cues for orientation. In subsequent fMRI scanning, we found evidence of an HD signal in RSC, thalamus, and precuneus. These findings harmonize rodent and human data, and suggest that immersive training procedures provide a viable way to examine the neural basis of navigation.

Spatial updating of multiple targets: Comparison of younger and older adults

When walking without vision, people mentally keep track of the directions and distances of previously viewed objects, a process called spatial updating. The current experiment indicates that while people across a large age range are able to update multiple targets in memory without perceptual support, aging negatively affects accuracy, precision, and decision time. Participants (20 to 80 years of age) viewed one, three, or six targets (colored lights) on the floor of a dimly lit room. Then, without vision, they walked to a target designated by color, either directly or indirectly (via a forward turning point). The younger adults' final stopping points were both accurate (near target) and precise (narrowly dispersed), but updating performance did degrade slightly with the number of targets. Older adults' performance was consistently worse than the younger group, but the lack of interaction between age and memory load indicates that the effect of age on performance was not further exacerbated by a greater number of targets. The number of targets also significantly increased the latency required to turn toward the designated target for both age groups. Taken together, results extend previous work showing impressive updating performance by younger adults, with novel findings showing that older adults manifest small but consistent degradation of updating performance of multitarget arrays.

Reduced multimodal integration of memory features following continuous theta burst stimulation of angular gyrus

BACKGROUND

Lesions of the angular gyrus (AnG) region of human parietal cortex do not cause amnesia, but appear to be associated with reduction in the ability to consciously experience the reliving of previous events.

OBJECTIVES/HYPOTHESIS

We used continuous theta burst stimulation to test the hypothesis that the cognitive mechanism implicated in this memory deficit might be the integration of retrieved sensory event features into a coherent multimodal memory representation.

METHODS

Healthy volunteers received stimulation to AnG or a vertex control site after studying stimuli that each comprised a visual object embedded in a scene, with the name of the object presented auditorily. Participants were then asked to make memory judgments about the studied stimuli that involved recollection of single event features (visual or auditory), or required integration of event features within the same modality, or across modalities.

RESULTS

Participants' ability to retrieve context features from across multiple modalities was significantly reduced after AnG stimulation compared to stimulation of the vertex. This effect was observed only for the integration of cross-modal context features but not for integration of features within the same modality, and could not be accounted for by task difficulty as performance was matched across integration conditions following vertex stimulation.

CONCLUSION

These results support the hypothesis that AnG is necessary for the multimodal integration of distributed cortical episodic features into a unified conscious representation that enables the experience of remembering.

Spatial Navigation in Preclinical Alzheimer's Disease

Although several previous studies have demonstrated navigational deficits in early-stage symptomatic Alzheimer's disease (AD), navigational abilities in preclinical AD have not been examined. The present investigation examined the effects of preclinical AD and early-stage symptomatic AD on spatial navigation performance. Performance on tasks of wayfinding and route learning in a virtual reality environment were examined. Comparisons were made across the following three groups: Clinically normal without preclinical AD (n = 42), clinically normal with preclinical AD (n = 13), and early-stage symptomatic AD (n = 16) groups. Preclinical AD was defined based on cerebrospinal fluid Aβ42 levels below 500 pg/ml. Preclinical AD was associated with deficits in the use of a wayfinding strategy, but not a route learning strategy. Moreover, post-hoc analyses indicated that wayfinding performance had moderate sensitivity and specificity. Results also confirmed early-stage symptomatic AD-related deficits in the use of both wayfinding and route learning strategies. The results of this study suggest that aspects of spatial navigation may be particularly sensitive at detecting the earliest cognitive deficits of AD.

Cortical Correlates of the Simulated Viewpoint Oscillation Advantage for Vection

Behavioural studies have consistently found stronger vection responses for oscillating, compared to smooth/constant, patterns of radial flow (the simulated viewpoint oscillation advantage for vection). Traditional accounts predict that simulated viewpoint oscillation should impair vection by increasing visual–vestibular conflicts in stationary observers (as this visual oscillation simulates self-accelerations that should strongly stimulate the vestibular apparatus). However, support for increased vestibular activity during accelerating vection has been mixed in the brain imaging literature. This fMRI study examined BOLD activity in visual (cingulate sulcus visual area — CSv; medial temporal complex — MT+; V6; precuneus motion area — PcM) and vestibular regions (parieto-insular vestibular cortex — PIVC/posterior insular cortex — PIC; ventral intraparietal region — VIP) when stationary observers were exposed to vection-inducing optic flow (i.e., globally coherent oscillating and smooth self-motion displays) as well as two suitable control displays. In line with earlier studies in which no vection occurred, CSv and PIVC/PIC both showed significantly increased BOLD activity during oscillating global motion compared to the other motion conditions (although this effect was found for fewer subjects in PIVC/PIC). The increase in BOLD activity in PIVC/PIC during prolonged exposure to the oscillating (compared to smooth) patterns of global optical flow appears consistent with vestibular facilitation.

Shifting visual perspective during retrieval shapes autobiographical memories

The dynamic and flexible nature of memories is evident in our ability to adopt multiple visual perspectives. Although autobiographical memories are typically encoded from the visual perspective of our own eyes they can be retrieved from the perspective of an observer looking at our self. Here, we examined the neural mechanisms of shifting visual perspective during long-term memory retrieval and its influence on online and subsequent memories using functional magnetic resonance imaging (fMRI). Participants generated specific autobiographical memories from the last five years and rated their visual perspective. In a separate fMRI session, they were asked to retrieve the memories across three repetitions while maintaining the same visual perspective as their initial rating or by shifting to an alternative perspective. Visual perspective shifting during autobiographical memory retrieval was supported by a linear decrease in neural recruitment across repetitions in the posterior parietal cortices. Additional analyses revealed that the precuneus, in particular, contributed to both online and subsequent changes in the phenomenology of memories. Our findings show that flexibly shifting egocentric perspective during autobiographical memory retrieval is supported by the precuneus, and suggest that this manipulation of mental imagery during retrieval has consequences for how memories are retrieved and later remembered.

Rapid and independent memory formation in the parietal cortex

Previous evidence indicates that the brain stores memory in two complementary systems, allowing both rapid plasticity and stable representations at different sites. For memory to be established in a long-lasting neocortical store, many learning repetitions are considered necessary after initial encoding into hippocampal circuits. To elucidate the dynamics of hippocampal and neocortical contributions to the early phases of memory formation, we closely followed changes in human functional brain activity while volunteers navigated through two different, initially unknown virtual environments. In one condition, they were able to encode new information continuously about the spatial layout of the maze. In the control condition, no information could be learned because the layout changed constantly. Our results show that the posterior parietal cortex (PPC) encodes memories for spatial locations rapidly, beginning already with the first visit to a location and steadily increasing activity with each additional encounter. Hippocampal activity and connectivity between the PPC and hippocampus, on the other hand, are strongest during initial encoding, and both decline with additional encounters. Importantly, stronger PPC activity related to higher memory-based performance. Compared with the nonlearnable control condition, PPC activity in the learned environment remained elevated after a 24-h interval, indicating a stable change. Our findings reflect the rapid creation of a memory representation in the PPC, which belongs to a recently proposed parietal memory network. The emerging parietal representation is specific for individual episodes of experience, predicts behavior, and remains stable over offline periods, and must therefore hold a mnemonic function.

Introducing a new age-and-cognition-sensitive measurement for assessing spatial orientation using a landmark-less virtual reality navigational task

Age-related impairments during spatial navigation have been widely reported in egocentric and allocentric paradigms. However, the effect of age on more specific navigational components such as the ability to drive or update directional information has not received enough attention. In this study we investigated the effect of age on spatial updating of a visual target after a series of whole-body rotations and transitions using a novel landmark-less virtual reality (VR) environment. Moreover, a significant number of previous studies focused on measures susceptible to a general decline in motor skills such as the spent time navigating, the distance traversed. The current paper proposes a new compound spatial measure to assess navigational performance, examines its reliability and compares its power with those of the measures of duration and traversed distance in predicting participants' age and cognitive groups assessed by Montreal Cognitive Assessment (MoCA) scores. Using data from 319 adults (20-83 years), our results confirm the reliability, the age sensitivity, and the cognitive validity of the designed spatial measure as well as its superiority to the measures of duration and traversed distance in predicting age and MoCA score. In addition, the results show the significant effect of age cognitive status on spatial updating.

Children's Spatial Representations: 3- and 4-Year-Olds are Affected by Irrelevant Peripheral References

Children as young as 3 years can remember an object’s location within an arrangement and can retrieve it from a novel viewpoint (Nardini et al., 2006). However, this ability is impaired if the arrangement is rotated to compensate for the novel viewpoint, or, if the arrangement is rotated and children stand still. There are two dominant explanations for this phenomenon: self-motion induces an automatic spatial updating process which is beneficial if children move around the arrangement, but misleading if the children’s movement is matched by the arrangement and not activated if children stand still and only the arrangement is moved (see spatial updating; Simons and Wang, 1998). Another explanation concerns reference frames: spatial representations might depend on peripheral spatial relations concerning the surrounding room instead on proximal relations within the arrangement, even if these proximal relations are sufficient or more informative. To evaluate these possibilities, we rotated children (N = 120) aged between 3 and 6 years with an occluded arrangement. When the arrangement was in misalignment to the surrounding room, 3- and 4-year-olds’ spatial memory was impaired and 5-year-olds’ was lightly impaired suggesting that they relied on peripheral references of the surrounding room for retrieval. In contrast, 6-years-olds’ spatial representation seemed robust against misalignment indicating a successful integration of spatial representations.

Motion parallax links visual motion areas and scene regions

When we move, the retinal velocities of objects in our surrounding differ according to their relative distances and give rise to a powerful three-dimensional visual cue referred to as motion parallax. Motion parallax allows us to infer our surrounding's 3D structure as well as self-motion based on 2D retinal information. However, the neural substrates mediating the link between visual motion and scene processing are largely unexplored. We used fMRI in human observers to study motion parallax by means of an ecologically relevant yet highly controlled stimulus that mimicked the observer's lateral motion past a depth-layered scene. We found parallax selective responses in parietal regions IPS3 and IPS4, and in a region lateral to scene selective occipital place area (OPA). The traditionally defined scene responsive regions OPA, the para-hippocampal place area (PPA) and the retrosplenial cortex (RSC) did not respond to parallax. During parallax processing, the occipital parallax selective region entertained highly specific functional connectivity with IPS3 and with scene selective PPA. These results establish a network linking dorsal motion and ventral scene processing regions specifically during parallax processing, which may underlie the brain's ability to derive 3D scene information from motion parallax.

Role of the human retrosplenial cortex/parieto-occipital sulcus in perspective priming

The ability to imagine the world from a different viewpoint is a fundamental competence for spatial reorientation and for imagining what another individual sees in the environment. Here, we investigated the neural bases of such an ability using functional magnetic resonance imaging. Healthy participants detected target displacements across consecutive views of a familiar virtual room, either from the perspective of an avatar (primed condition) or in the absence of such a prime (unprimed condition). In the primed condition, the perspective at test always corresponded to the avatar's perspective, while in the unprimed condition it was randomly chosen as 0, 45 or 135deg of viewpoint rotation. We observed a behavioral advantage in performing a perspective transformation during the primed condition as compared to an equivalent amount of unprimed perspective change. Although many cortical regions (dorsal parietal, parieto-temporo-occipital junction, precuneus and retrosplenial cortex/parieto-occipital sulcus or RSC/POS) were involved in encoding and retrieving target location from different perspectives and were modulated by the amount of viewpoint rotation, the RSC/POS was the only area showing decreased activity in the primed as compared to the unprimed condition, suggesting that this region anticipates the upcoming perspective change. The retrosplenial cortex/parieto-occipital sulcus appears to play a special role in the allocentric coding of heading directions.

Cortical connectivity maps reveal anatomically distinct areas in the parietal cortex of the rat

A central feature of theories of spatial navigation involves the representation of spatial relationships between objects in complex environments. The parietal cortex has long been linked to the processing of spatial visual information and recent evidence from single unit recording in rodents suggests a role for this region in encoding egocentric and world-centered frames. The rat parietal cortex can be subdivided into four distinct rostral-caudal and medial-lateral regions, which includes a zone previously characterized as secondary visual cortex. At present, very little is known regarding the relative connectivity of these parietal subdivisions. Thus, we set out to map the connectivity of the entire anterior-posterior and medial-lateral span of this region. To do this we used anterograde and retrograde tracers in conjunction with open source neuronal segmentation and tracer detection tools to generate whole brain connectivity maps of parietal inputs and outputs. Our present results show that inputs to the parietal cortex varied significantly along the medial-lateral, but not the rostral-caudal axis. Specifically, retrosplenial connectivity is greater medially, but connectivity with visual cortex, though generally sparse, is more significant laterally. Finally, based on connection density, the connectivity between parietal cortex and hippocampus is indirect and likely achieved largely via dysgranular retrosplenial cortex. Thus, similar to primates, the parietal cortex of rats exhibits a difference in connectivity along the medial-lateral axis, which may represent functionally distinct areas.

Parallax-sensitive remapping of visual space in occipito-parietal alpha-band activity during whole-body motion

Despite the constantly changing retinal image due to eye, head, and body movements, we are able to maintain a stable representation of the visual environment. Various studies on retinal image shifts caused by saccades have suggested that occipital and parietal areas correct for these perturbations by a gaze-centered remapping of the neural image. However, such a uniform, rotational, remapping mechanism cannot work during translations when objects shift on the retina in a more complex, depth-dependent fashion due to motion parallax. Here we tested whether the brain's activity patterns show parallax-sensitive remapping of remembered visual space during whole-body motion. Under continuous recording of electroencephalography (EEG), we passively translated human subjects while they had to remember the location of a world-fixed visual target, briefly presented in front of or behind the eyes' fixation point prior to the motion. Using a psychometric approach we assessed the quality of the memory update, which had to be made based on vestibular feedback and other extraretinal motion cues. All subjects showed a variable amount of parallax-sensitive updating errors, i.e., the direction of the errors depended on the depth of the target relative to fixation. The EEG recordings show a neural correlate of this parallax-sensitive remapping in the alpha-band power at occipito-parietal electrodes. At parietal electrodes, the strength of these alpha-band modulations correlated significantly with updating performance. These results suggest that alpha-band oscillatory activity reflects the time-varying updating of gaze-centered spatial information during parallax-sensitive remapping during whole-body motion.

Examining the influence of working memory on updating mental models

The ability to accurately build and update mental representations of our environment depends on our ability to integrate information over a variety of time scales and detect changes in the regularity of events. As such, the cognitive mechanisms that support model building and updating are likely to interact with those involved in working memory (WM). To examine this, we performed three experiments that manipulated WM demands concurrently with the need to attend to regularities in other stimulus properties (i.e., location and shape). That is, participants completed a prediction task while simultaneously performing an n-back WM task with either no load or a moderate load. The distribution of target locations (Experiment 1) or shapes (Experiments 2 and 3) included some level of probabilistic regularity, which, unbeknown to participants, changed abruptly within each block. Moderate WM load hampered the ability to benefit from target regularities and to adapt to changes in those regularities (i.e., the prediction task). This was most pronounced when both prediction and WM requirements shared the same target feature. Our results show that representational updating depends on free WM resources in a domain-specific fashion.

Functional Dissociations Within Posterior Parietal Cortex During Scene Integration and Viewpoint Changes

The posterior parietal cortex (PPC) is an anatomically heterogeneous brain region implicated in a wide range of cognitive operations, including egocentric spatial processing and both short- and long-term memory. Here, we report functional specificities of cytoarchitectonically defined subregions of PPC during the processing of scenes across changes in viewpoint. Participants (n = 16) saw photographs of familiar and unfamiliar places while undergoing functional magnetic resonance imaging (fMRI). On each trial, 4 viewpoints of the same place were presented, with either a plausible sequence of viewpoints (SEQ) or a scrambled order (SCRA). Distinct response profiles were observed within PPC. Area 7A showed increased activity for SEQ versus SCRA order, regardless of place familiarity, whereas the rostral inferior parietal lobule showed preferential increases for unfamiliar versus familiar places in SEQ series. In contrast, more posterior subregions in both superior and inferior PPC exhibited increases for familiar versus unfamiliar places at the end of the sequence, regardless of order. The data highlight the distinctive contribution of several subregions of PPC during the processing of scenes, with specific cortical areas involved in the progressive integration of spatial information across viewpoint changes, and others involved in the retrieval and maintenance of scene information in memory.

Challenges for identifying the neural mechanisms that support spatial navigation: the impact of spatial scale

Spatial navigation is a fascinating behavior that is essential for our everyday lives. It involves nearly all sensory systems, it requires numerous parallel computations, and it engages multiple memory systems. One of the key problems in this field pertains to the question of reference frames: spatial information such as direction or distance can be coded egocentrically—relative to an observer—or allocentrically—in a reference frame independent of the observer. While many studies have associated striatal and parietal circuits with egocentric coding and entorhinal/hippocampal circuits with allocentric coding, this strict dissociation is not in line with a growing body of experimental data. In this review, we discuss some of the problems that can arise when studying the neural mechanisms that are presumed to support different spatial reference frames. We argue that the scale of space in which a navigation task takes place plays a crucial role in determining the processes that are being recruited. This has important implications, particularly for the inferences that can be made from animal studies in small scale space about the neural mechanisms supporting human spatial navigation in large (environmental) spaces. Furthermore, we argue that many of the commonly used tasks to study spatial navigation and the underlying neuronal mechanisms involve different types of reference frames, which can complicate the interpretation of neurophysiological data.

Balancing the mind: vestibular induced facilitation of egocentric mental transformations

The body schema is a key component in accomplishing egocentric mental transformations, which rely on bodily reference frames. These reference frames are based on a plurality of different cognitive and sensory cues among which the vestibular system plays a prominent role. We investigated whether a bottom-up influence of vestibular stimulation modulates the ability to perform egocentric mental transformations. Participants were significantly faster to make correct spatial judgments during vestibular stimulation as compared to sham stimulation. Interestingly, no such effects were found for mental transformation of hand stimuli or during mental transformations of letters, thus showing a selective influence of vestibular stimulation on the rotation of whole-body reference frames. Furthermore, we found an interaction with the angle of rotation and vestibular stimulation demonstrating an increase in facilitation during mental body rotations in a direction congruent with rightward vestibular afferents. We propose that facilitation reflects a convergence in shared brain areas that process bottom-up vestibular signals and top-down imagined whole-body rotations, including the precuneus and tempero-parietal junction. Ultimately, our results show that vestibular information can influence higher-order cognitive processes, such as the body schema and mental imagery.

Medial temporal lobe roles in human path integration

Path integration is a process in which observers derive their location by integrating self-motion signals along their locomotion trajectory. Although the medial temporal lobe (MTL) is thought to take part in path integration, the scope of its role for path integration remains unclear. To address this issue, we administered a variety of tasks involving path integration and other related processes to a group of neurosurgical patients whose MTL was unilaterally resected as therapy for epilepsy. These patients were unimpaired relative to neurologically intact controls in many tasks that required integration of various kinds of sensory self-motion information. However, the same patients (especially those who had lesions in the right hemisphere) walked farther than the controls when attempting to walk without vision to a previewed target. Importantly, this task was unique in our test battery in that it allowed participants to form a mental representation of the target location and anticipate their upcoming walking trajectory before they began moving. Thus, these results put forth a new idea that the role of MTL structures for human path integration may stem from their participation in predicting the consequences of one's locomotor actions. The strengths of this new theoretical viewpoint are discussed.

Scene analysis in the natural environment

The problem of scene analysis has been studied in a number of different fields over the past decades. These studies have led to important insights into problems of scene analysis, but not all of these insights are widely appreciated, and there remain critical shortcomings in current approaches that hinder further progress. Here we take the view that scene analysis is a universal problem solved by all animals, and that we can gain new insight by studying the problems that animals face in complex natural environments. In particular, the jumping spider, songbird, echolocating bat, and electric fish, all exhibit behaviors that require robust solutions to scene analysis problems encountered in the natural environment. By examining the behaviors of these seemingly disparate animals, we emerge with a framework for studying scene analysis comprising four essential properties: (1) the ability to solve ill-posed problems, (2) the ability to integrate and store information across time and modality, (3) efficient recovery and representation of 3D scene structure, and (4) the use of optimal motor actions for acquiring information to progress toward behavioral goals.

Differential hippocampal and retrosplenial involvement in egocentric-updating, rotation, and allocentric processing during online spatial encoding: an fMRI study

The way new spatial information is encoded seems to be crucial in disentangling the role of decisive regions within the spatial memory network (i.e., hippocampus, parahippocampal, parietal, retrosplenial,…). Several data sources converge to suggest that the hippocampus is not always involved or indeed necessary for allocentric processing. Hippocampal involvement in spatial coding could reflect the integration of new information generated by “online” self-related changes. In this fMRI study, the participants started by encoding several object locations in a virtual reality environment and then performed a pointing task. Allocentric encoding was maximized by using a survey perspective and an object-to-object pointing task. Two egocentric encoding conditions were used, involving self-related changes processed under a first-person perspective and implicating a self-to-object pointing task. The Egocentric-updating condition involved navigation whereas the Egocentric with rotation only condition involved orientation changes only. Conjunction analysis of spatial encoding conditions revealed a wide activation of the occipito-parieto-frontal network and several medio-temporal structures. Interestingly, only the cuneal areas were significantly more recruited by the allocentric encoding in comparison to other spatial conditions. Moreover, the enhancement of hippocampal activation was found during Egocentric-updating encoding whereas the retrosplenial activation was observed during the Egocentric with rotation only condition. Hence, in some circumstances, hippocampal and retrosplenial structures—known for being involved in allocentric environmental coding—demonstrate preferential involvement in the egocentric coding of space. These results indicate that the raw differentiation between allocentric versus egocentric representation seems to no longer be sufficient in understanding the complexity of the mechanisms involved during spatial encoding.

Neural correlates of spatial navigation changes in mild cognitive impairment and Alzheimer's disease

Although the memory impairment is a hallmark of Alzheimer's disease (AD), AD has also been characterized by spatial disorientation, which is present from its early stages. Spatial disorientation in AD manifests itself in getting lost in familiar and unfamiliar places and have been characterized more specifically using spatial navigation tests in both real space and virtual environments as an impairment in multiple spatial abilities, including allocentric and egocentric navigation strategies, visuo-spatial perception, or selection of relevant information for successful navigation. Patients suffering mild cognitive impairment (MCI), who are at a high risk of development of dementia, show impairment in a subset of these abilities, mainly connected with allocentric and egocentric processing. While spatial disorientation in typical AD patients probably reflects neurodegenerative changes in medial and posterior temporal, parietal, and frontal lobes, and retrosplenial cortex, the impairment of spatial navigation in MCI seem to be connected mainly with the medial temporal and also parietal brain changes. In this review, we will summarize the signs of brain disease in most MCI and AD patients showing in various tasks of spatial memory and navigation.

Do we have an internal model of the outside world?

Our phenomenal world remains stationary in spite of movements of the eyes, head and body. In addition, we can point or turn to objects in the surroundings whether or not they are in the field of view. In this review, I argue that these two features of experience and behaviour are related. The ability to interact with objects we cannot see implies an internal memory model of the surroundings, available to the motor system. And, because we maintain this ability when we move around, the model must be updated, so that the locations of object memories change continuously to provide accurate directional information. The model thus contains an internal representation of both the surroundings and the motions of the head and body: in other words, a stable representation of space. Recent functional MRI studies have provided strong evidence that this egocentric representation has a location in the precuneus, on the medial surface of the superior parietal cortex. This is a region previously identified with 'self-centred mental imagery', so it seems likely that the stable egocentric representation, required by the motor system, is also the source of our conscious percept of a stable world.

Spatial linear navigation: is vision necessary?

In order to analyze spatial linear navigation through a task of self-controlled reproduction, healthy participants were passively transported on a mobile robot at constant velocity, and then had to reproduce the imposed distance of 2-8m in two conditions: "with vision" and "without vision". Our hypothesis was that the reproduction of distances would be longer with than without visual information. Indeed, with visual information the reproduction of all distances was overshot. In the "without vision" condition the reproduced distances were quite close to the imposed ones, but only for the shortest distances (2 and 4m) as the longest ones were clearly undershot. With vision the reproduction error was less than 10% for all distances; however the error could be smaller without vision at short distances, and therefore vision was not necessary.

Priorities for selection and representation in natural tasks

Selecting and remembering visual information is an active and competitive process. In natural environments, representations are tightly coupled to task. Objects that are task-relevant are remembered better due to a combination of increased selection for fixation and strategic control of encoding and/or retaining viewed information. However, it is not understood how physically manipulating objects when performing a natural task influences priorities for selection and memory. In this study, we compare priorities for selection and memory when actively engaged in a natural task with first-person observation of the same object manipulations. Results suggest that active manipulation of a task-relevant object results in a specific prioritization for object position information compared with other properties and compared with action observation of the same manipulations. Experiment 2 confirms that this spatial prioritization is likely to arise from manipulation rather than differences in spatial representation in real environments and the movies used for action observation. Thus, our findings imply that physical manipulation of task relevant objects results in a specific prioritization of spatial information about task-relevant objects, possibly coupled with strategic de-prioritization of colour memory for irrelevant objects.

Multifaceted roles for low-frequency oscillations in bottom-up and top-down processing during navigation and memory

A prominent and replicated finding is the correlation between running speed and increases in low-frequency oscillatory activity in the hippocampal local field potential. A more recent finding concerns low-frequency oscillations that increase in coherence between the hippocampus and neocortical brain areas such as prefrontal cortex during memory-related behaviors (i.e., remembering the correct location to visit). In this review, we tie together movement-related and memory-related low-frequency oscillations in the rodent with similar findings in humans. We argue that although movement-related low-frequency oscillations, in particular, may have slightly different characteristics in humans than rodents, placing important constraints on our thinking about this issue, both phenomena have similar functional foundations. We review four prominent theoretical models that provide partially conflicting accounts of movement-related low-frequency oscillations. We attempt to tie together these theoretical proposals, and existing data in rodents and humans, with memory-related low-frequency oscillations. We propose that movement-related low-frequency oscillations and memory-related low-frequency oscillatory activity, both of which show significant coherence with oscillations in other brain regions, represent different facets of "spectral fingerprints," or different resonant frequencies within the same brain networks underlying different cognitive processes. Together, movement-related and memory-related low-frequency oscillatory coupling may be linked by their distinct contributions to bottom-up, sensorimotor driven processing and top-down, controlled processing characterizing aspects of memory encoding and retrieval.

The eye of the self: precuneus volume and visual perspective during autobiographical memory retrieval

Visual perspective (i.e. first-person versus third-person perspective) during autobiographical memory (AM) retrieval plays a role in both emotional regulation and self-related processes. However, its neural underpinnings remain mostly unexplored. Visual perspective during AM retrieval was assessed in two independent datasets of 45 and 20 healthy young adults with two different AM retrieval tasks. Diffeomorphic anatomical registration using exponentiated lie algebra and voxel-based morphometry were used to assess individual differences in the precuneus grey matter volume. The spontaneous tendency to recall memories from a first-person perspective was positively correlated with the right precuneus volume among the two independent datasets. Whole-brain analyses revealed that these results were relatively specific to the anterior part of the right precuneus. Our results provide first evidence for the role of the precuneus in egocentric spatial processing in the context of AM retrieval among healthy subjects.

The operation of the visual system in relation to action

Neurophysiologists studying the visual representation of the world in the parietal lobe generally find that it is based in a gaze-centred (retinotopic) frame. Students of orientation, however, find that the brain also contains a more panoramic egocentric representation that allows appropriate motor actions to take place independent of the orientation of the eyes and head. This representation can operate temporarily without visual input, but is updated from the vestibular system and from other modalities. In this minireview, I shall consider how these two representations are kept aligned with each other, and how they relate to the organisation of motor actions and to the phenomenal world that we see.

Obstacle avoidance and smooth trajectory control: neural areas highlighted during improved locomotor performance

Visual control of locomotion typically involves both detection of current egomotion as well as anticipation of impending changes in trajectory. To determine if there are distinct neural systems involved in these aspects of steering control we used a slalom paradigm, which required participants to steer around objects in a computer simulated environment using a joystick. In some trials the whole slalom layout was visible (steering “preview” trials) so planning of the trajectory around future waypoints was possible, whereas in other trials the slalom course was only revealed one object at a time (steering “near” trials) so that future planning was restricted. In order to control for any differences in the motor requirements and visual properties between “preview” and “near” trials, we also interleaved control trials which replayed a participants' previous steering trials, with the task being to mimic the observed steering. Behavioral and fMRI results confirmed previous findings of superior parietal lobe (SPL) recruitment during steering trials, with a more extensive parietal and sensorimotor network during steering “preview” compared to steering “near” trials. Correlational analysis of fMRI data with respect to individual behavioral performance revealed that there was increased activation in the SPL in participants who exhibited smoother steering performance. These findings indicate that there is a role for the SPL in encoding path defining targets or obstacles during forward locomotion, which also provides a potential neural underpinning to explain improved steering performance on an individual basis.

The functional role of the medial motion area V6

In macaque, several visual areas are devoted to analyze motion in the visual field, and V6 is one of these areas. In macaque, area V6 occupies the ventral part of the anterior bank of the parieto-occipital sulcus (POs), is retinotopically-organized and contains a point-to-point representation of the retinal surface. V6 is a motion sensitive area that largely represents the peripheral part of the visual field and whose cells are very sensitive to translational motion. Based on the fact that macaque V6 contains many real-motion cells, it has been suggested that V6 is involved in object-motion recognition. Recently, area V6 has been recognized also in the human brain by neuroimaging and electrophysiological methods. Like macaque V6, human V6 is located in the POs, is retinotopically organized, and represents the entire contralateral hemifield up to the far periphery. Human V6, like macaque V6, is a motion area that responds to unidirectional motion. It has a strong preference for coherent motion and a recent combined VEPs/fMRI work has shown that area V6 is even one of the most early stations coding the motion coherence. Human V6 is highly sensitive to flow field and is also able to distinguish between different 3D flow fields being selective to translational egomotion. This suggests that this area processes visual egomotion signals to extract information about the relative distance of objects, likely in order to act on them, or to avoid them. The view that V6 is involved in the estimation of egomotion has been tested also in other recent fMRI studies. Thus, taken together, human and macaque data suggest that V6 is involved in both object and self-motion recognition. Specifically, V6 could be involved in "subtracting out" self-motion signals across the whole visual field and in providing information about moving objects, particularly during self-motion in a complex and dynamically unstable environment.

Neural mechanisms of attentional reorienting in three-dimensional space

How the human brain reconstructs the three-dimensional (3D) world from two-dimensional (2D) retinal images has received a great deal of interest as has how we shift attention in 2D space. In contrast, it remains poorly understood how visuospatial attention is shifted in depth. In this fMRI study, by constructing a virtual 3D environment in the MR scanner and by presenting targets either close to or far from the participants in an adapted version of the Posner spatial-cueing paradigm, we investigated the behavioral and neural mechanisms underlying visuospatial orienting/reorienting in depth. At the behavioral level, although covering the same spatial distance, attentional reorienting to objects unexpectedly appearing closer to the observer and in the unattended hemispace was faster than reorienting to unexpected objects farther away. At the neural level, we found that in addition to the classical attentional reorienting system in the right temporoparietal junction, two additional brain networks were differentially involved in aspects of attentional reorienting in depth. First, bilateral premotor cortex reoriented visuospatial attention specifically along the third dimension of visual space (i.e., from close to far or vice versa), compared with attentional reorienting within the same depth plane. Second, a network of areas reminiscent of the human "default-mode network," including posterior cingulate cortex, orbital prefrontal cortex, and left angular gyrus, was involved in the neural interaction between depth and attentional orienting, by boosting attentional reorienting to unexpected objects appearing both closer to the observer and in the unattended hemispace.

Dissociable influences of auditory object vs. spatial attention on visual system oscillatory activity

Given that both auditory and visual systems have anatomically separate object identification (“what”) and spatial (“where”) pathways, it is of interest whether attention-driven cross-sensory modulations occur separately within these feature domains. Here, we investigated how auditory “what” vs. “where” attention tasks modulate activity in visual pathways using cortically constrained source estimates of magnetoencephalograpic (MEG) oscillatory activity. In the absence of visual stimuli or tasks, subjects were presented with a sequence of auditory-stimulus pairs and instructed to selectively attend to phonetic (“what”) vs. spatial (“where”) aspects of these sounds, or to listen passively. To investigate sustained modulatory effects, oscillatory power was estimated from time periods between sound-pair presentations. In comparison to attention to sound locations, phonetic auditory attention was associated with stronger alpha (7–13 Hz) power in several visual areas (primary visual cortex; lingual, fusiform, and inferior temporal gyri, lateral occipital cortex), as well as in higher-order visual/multisensory areas including lateral/medial parietal and retrosplenial cortices. Region-of-interest (ROI) analyses of dynamic changes, from which the sustained effects had been removed, suggested further power increases during Attend Phoneme vs. Location centered at the alpha range 400–600 ms after the onset of second sound of each stimulus pair. These results suggest distinct modulations of visual system oscillatory activity during auditory attention to sound object identity (“what”) vs. sound location (“where”). The alpha modulations could be interpreted to reflect enhanced crossmodal inhibition of feature-specific visual pathways and adjacent audiovisual association areas during “what” vs. “where” auditory attention.

Frames of reference and their neural correlates within navigation in a 3D environment

The goal of this study was an administration of the navigation task in a three-dimensional virtual environment to localize the electroencephalogram (EEG) features responsible for egocentric and allocentric reference frame processing in a horizontal and also in a vertical plane. We recorded the EEG signal of a traverse through a virtual tunnel to search for the best signal features that discriminate between specific strategies in particular plane. We identified intrahemispheric coherences in occipital-parietal and temporal-parietal areas as the most discriminative features. They have 10% lower error rate compared to single electrode features adopted in previous studies. The behavioral analysis revealed that 11% of participants switched from egocentric to allocentric strategy in a vertical plane, while 24% of participants consistently adopted egocentric strategy in both planes.

Dynamic visuomotor transformation involved with remote flying of a plane utilizes the 'Mirror Neuron' system

Brain regions involved with processing dynamic visuomotor representational transformation are investigated using fMRI. The perceptual-motor task involved flying (or observing) a plane through a simulated Red Bull Air Race course in first person and third person chase perspective. The third person perspective is akin to remote operation of a vehicle. The ability for humans to remotely operate vehicles likely has its roots in neural processes related to imitation in which visuomotor transformation is necessary to interpret the action goals in an egocentric manner suitable for execution. In this experiment for 3^rd person perspective the visuomotor transformation is dynamically changing in accordance to the orientation of the plane. It was predicted that 3^rd person remote flying, over 1^st, would utilize brain regions composing the ‘Mirror Neuron’ system that is thought to be intimately involved with imitation for both execution and observation tasks. Consistent with this prediction differential brain activity was present for 3^rd person over 1^st person perspectives for both execution and observation tasks in left ventral premotor cortex, right dorsal premotor cortex, and inferior parietal lobule bilaterally (Mirror Neuron System) (Behaviorally: 1^st>3^rd). These regions additionally showed greater activity for flying (execution) over watching (observation) conditions. Even though visual and motor aspects of the tasks were controlled for, differential activity was also found in brain regions involved with tool use, motion perception, and body perspective including left cerebellum, temporo-occipital regions, lateral occipital cortex, medial temporal region, and extrastriate body area. This experiment successfully demonstrates that a complex perceptual motor real-world task can be utilized to investigate visuomotor processing. This approach (Aviation Cerebral Experimental Sciences ACES) focusing on direct application to lab and field is in contrast to standard methodology in which tasks and conditions are reduced to their simplest forms that are remote from daily life experience.

Brain activation during spatial updating and attentive tracking of moving targets

Keeping aware of the locations of objects while one is moving requires the updating of spatial representations. As long as the objects are visible, attentional tracking is sufficient, but knowing where objects out of view went in relation to one's own body involves an updating of spatial working memory. Here, multiple object tracking was employed to study spatial updating and its neural correlates. In a dynamic 3D-scene, targets moved among visually indistinguishable distractors. The targets and distractors either stayed visible during continuous viewpoint changes or they turned invisible. The parametric variation of tracking load revealed load-dependent activations of the intraparietal sulcus, the superior parietal lobule, and the lateral occipital cortex in response to the attentive tracking task. Viewpoint changes with invisible objects that demanded retention and updating produced load-dependent activation only in the precuneus in line with its presumed involvement in updating spatial working memory.

Sensitivity of human visual cortical area V6 to stereoscopic depth gradients associated with self-motion

The principal visual cue to self-motion (egomotion) is optic flow, which is specified in terms of local 2D velocities in the retinal image without reference to depth cues. However, in general, points near the center of expansion of natural flow fields are distant, whereas those in the periphery are closer, creating gradients of horizontal binocular disparity. To assess whether the brain combines disparity gradients with optic flow when encoding egomotion, stereoscopic gradients were applied to expanding dot patterns presented to observers during functional MRI scanning. The gradients were radially symmetrical, disparity changing as a function of eccentricity. The depth cues were either consistent with egomotion (peripheral dots perceived as near and central dots perceived as far) or inconsistent (the reverse gradient, central dots near, peripheral dots far). The BOLD activity generated by these stimuli was compared in a range of predefined visual regions in 13 participants with good stereoacuity. Visual area V6, in the parieto-occipital sulcus, showed a unique pattern of results, responding well to all optic flow patterns but much more strongly when they were paired with consistent rather than inconsistent or zero-disparity gradients. Of the other areas examined, a region of the precuneus and parietoinsular vestibular cortex also differentiate between consistent and inconsistent gradients, but with weak or suppressive responses. V3A, V7, MT, and ventral intraparietal area responded more strongly in the presence of a depth gradient but were indifferent to its depth-flow congruence. The results suggest that depth and flow cues are integrated in V6 to improve estimation of egomotion.

Vision and the representation of the surroundings in spatial memory

One of the paradoxes of vision is that the world as it appears to us and the image on the retina at any moment are not much like each other. The visual world seems to be extensive and continuous across time. However, the manner in which we sample the visual environment is neither extensive nor continuous. How does the brain reconcile these differences? Here, we consider existing evidence from both static and dynamic viewing paradigms together with the logical requirements of any representational scheme that would be able to support active behaviour. While static scene viewing paradigms favour extensive, but perhaps abstracted, memory representations, dynamic settings suggest sparser and task-selective representation. We suggest that in dynamic settings where movement within extended environments is required to complete a task, the combination of visual input, egocentric and allocentric representations work together to allow efficient behaviour. The egocentric model serves as a coding scheme in which actions can be planned, but also offers a potential means of providing the perceptual stability that we experience.

Spatial constancy mechanisms in motor control

The success of the human species in interacting with the environment depends on the ability to maintain spatial stability despite the continuous changes in sensory and motor inputs owing to movements of eyes, head and body. In this paper, I will review recent advances in the understanding of how the brain deals with the dynamic flow of sensory and motor information in order to maintain spatial constancy of movement goals. The first part summarizes studies in the saccadic system, showing that spatial constancy is governed by a dynamic feed-forward process, by gaze-centred remapping of target representations in anticipation of and across eye movements. The subsequent sections relate to other oculomotor behaviour, such as eye-head gaze shifts, smooth pursuit and vergence eye movements, and their implications for feed-forward mechanisms for spatial constancy. Work that studied the geometric complexities in spatial constancy and saccadic guidance across head and body movements, distinguishing between self-generated and passively induced motion, indicates that both feed-forward and sensory feedback processing play a role in spatial updating of movement goals. The paper ends with a discussion of the behavioural mechanisms of spatial constancy for arm motor control and their physiological implications for the brain. Taken together, the emerging picture is that the brain computes an evolving representation of three-dimensional action space, whose internal metric is updated in a nonlinear way, by optimally integrating noisy and ambiguous afferent and efferent signals.

Visual stability

Our vision remains stable even though the movements of our eyes, head and bodies create a motion pattern on the retina. One of the most important, yet basic, feats of the visual system is to correctly determine whether this retinal motion is owing to real movement in the world or rather our own self-movement. This problem has occupied many great thinkers, such as Descartes and Helmholtz, at least since the time of Alhazen. This theme issue brings together leading researchers from animal neurophysiology, clinical neurology, psychophysics and cognitive neuroscience to summarize the state of the art in the study of visual stability. Recently, there has been significant progress in understanding the limits of visual stability in humans and in identifying many of the brain circuits involved in maintaining a stable percept of the world. Clinical studies and new experimental methods, such as transcranial magnetic stimulation, now make it possible to test the causal role of different brain regions in creating visual stability and also allow us to measure the consequences when the mechanisms of visual stability break down.

Dissociable cognitive mechanisms underlying human path integration

Path integration is a fundamental mechanism of spatial navigation. In non-human species, it is assumed to be an online process in which a homing vector is updated continuously during an outward journey. In contrast, human path integration has been conceptualized as a configural process in which travelers store working memory representations of path segments, with the computation of a homing vector only occurring when required. To resolve this apparent discrepancy, we tested whether humans can employ different path integration strategies in the same task. Using a triangle completion paradigm, participants were instructed either to continuously update the start position during locomotion (continuous strategy) or to remember the shape of the outbound path and to calculate home vectors on basis of this representation (configural strategy). While overall homing accuracy was superior in the configural condition, participants were quicker to respond during continuous updating, strongly suggesting that homing vectors were computed online. Corroborating these findings, we observed reliable differences in head orientation during the outbound path: when participants applied the continuous updating strategy, the head deviated significantly from straight ahead in direction of the start place, which can be interpreted as a continuous motor expression of the homing vector. Head orientation-a novel online measure for path integration-can thus inform about the underlying updating mechanism already during locomotion. In addition to demonstrating that humans can employ different cognitive strategies during path integration, our two-systems view helps to resolve recent controversies regarding the role of the medial temporal lobe in human path integration.

Interruption of visually perceived forward motion in depth evokes a cortical activation shift from spatial to intentional motor regions

Forward locomotion generates a radially expanding flow of visual motion which supports goal-directed walking. In stationary mode, wide-field visual presentation of optic flow stimuli evokes the illusion of forward self-motion. These effects illustrate an intimate relation between visual and motor processing. In the present fMRI study, we applied optic flow to identify distinct interfaces between circuitries implicated in vision and movement. The dorsal premotor cortex (PMd) was expected to contribute to wide-field forward motion flow (FFw), reflecting a pathway for externally triggered motor control. Medial prefrontal activation was expected to follow interrupted optic flow urging internally generated action. Data of 15 healthy subjects were analyzed with Statistical Parametric Mapping and confirmed this hypothesis. Right PMd activation was seen in FFw, together with activations of posterior parietal cortex, ventral V5, and the right fusiform gyrus. Conjunction analysis of the transition from wide to narrow forward flow and reversed wide-field flow revealed selective dorsal medial prefrontal activation. These findings point at equivalent visuomotor transformations in locomotion and goal-directed hand movement, in which parietal-premotor circuitry is crucially implicated. Possible implications of an activation shift from spatial to intentional motor regions for understanding freezing of gait in Parkinson's disease are discussed: impaired medial prefrontal function in Parkinson's disease may reflect an insufficient internal motor drive when visual support from optic flow is reduced at the entrance of a narrow corridor.

Spatiotemporal phase-scrambling increases visual cortex activity

The hemodynamic response of the visual cortex to continuously moving spatial stimuli of virtual tunnels and phase-scrambled versions thereof was examined using functional magnetic resonance imaging. Earlier functional magnetic resonance imaging studies found either no difference or less early visual cortex (VC) activation when presenting normal versus phase-manipulated static natural images. Here we describe an increase in VC activation while viewing phase-scrambled films compared with normal films, although basic image statistics and average local flow were the same. The normal films, in contrast, resulted in an increased lateral occipital and precuneus activity sparing VC. In summary, our results show that earlier findings for scrambling of static images no longer hold for spatiotemporal stimuli.