Current Promises and Limitations of Combined Virtual Reality and Functional Magnetic Resonance Imaging Research in Humans: A Commentary on Huffman and Ekstrom (2019)

Electrophysiological signatures of veridical head direction in humans

Information about heading direction is critical for navigation as it provides the means to orient ourselves in space. However, given that veridical head-direction signals require physical rotation of the head and most human neuroimaging experiments depend upon fixing the head in position, little is known about how the human brain is tuned to such heading signals. Here we adress this by asking 52 healthy participants undergoing simultaneous electroencephalography and motion tracking recordings (split into two experiments) and 10 patients undergoing simultaneous intracranial electroencephalography and motion tracking recordings to complete a series of orientation tasks in which they made physical head rotations to target positions. We then used a series of forward encoding models and linear mixed-effects models to isolate electrophysiological activity that was specifically tuned to heading direction. We identified a robust posterior central signature that predicts changes in veridical head orientation after regressing out confounds including sensory input and muscular activity. Both source localization and intracranial analysis implicated the medial temporal lobe as the origin of this effect. Subsequent analyses disentangled head-direction signatures from signals relating to head rotation and those reflecting location-specific effects. Lastly, when directly comparing head direction and eye-gaze-related tuning, we found that the brain maintains both codes while actively navigating, with stronger tuning to head direction in the medial temporal lobe. Together, these results reveal a taxonomy of population-level head-direction signals within the human brain that is reminiscent of those reported in the single units of rodents.

Neural compass in the human brain during naturalistic virtual navigation

Humans and animals maintain a consistent representation of their facing direction during spatial navigation. In rodents, head direction cells are believed to support this "neural compass", but identifying a similar mechanism in humans during dynamic naturalistic navigation has been challenging. To address this issue, we acquired fMRI data while participants freely navigated through a virtual reality city. Encoding model analyses revealed voxel clusters in retrosplenial complex and superior parietal lobule that exhibited reliable tuning as a function of facing direction. Crucially, these directional tunings were consistent across perceptually different versions of the city, spatially separated locations within the city, and motivationally distinct phases of the behavioral task. Analysis of the model weights indicated that these regions may represent facing direction relative to the principal axis of the environment. These findings reveal specific mechanisms in the human brain that allow us to maintain a sense of direction during naturalistic, dynamic navigation.

iVR-fNIRS: studying brain functions in a fully immersive virtual environment

Immersive virtual reality (iVR) employs head-mounted displays or cave-like environments to create a sensory-rich virtual experience that simulates the physical presence of a user in a digital space. The technology holds immense promise in neuroscience research and therapy. In particular, virtual reality (VR) technologies facilitate the development of diverse tasks and scenarios closely mirroring real-life situations to stimulate the brain within a controlled and secure setting. It also offers a cost-effective solution in providing a similar sense of interaction to users when conventional stimulation methods are limited or unfeasible. Although combining iVR with traditional brain imaging techniques may be difficult due to signal interference or instrumental issues, recent work has proposed the use of functional near infrared spectroscopy (fNIRS) in conjunction with iVR for versatile brain stimulation paradigms and flexible examination of brain responses. We present a comprehensive review of current research studies employing an iVR-fNIRS setup, covering device types, stimulation approaches, data analysis methods, and major scientific findings. The literature demonstrates a high potential for iVR-fNIRS to explore various types of cognitive, behavioral, and motor functions in a fully immersive VR (iVR) environment. Such studies should set a foundation for adaptive iVR programs for both training (e.g., in novel environments) and clinical therapeutics (e.g., pain, motor and sensory disorders and other psychiatric conditions).

Repetition Suppression Reveals Cue-Specific Spatial Representations for Landmarks and Self-Motion Cues in the Human Retrosplenial Cortex

The efficient use of various spatial cues within a setting is crucial for successful navigation. Two fundamental forms of spatial navigation, landmark-based and self-motion-based, engage distinct cognitive mechanisms. The question of whether these modes invoke shared or separate spatial representations in the brain remains unresolved. While nonhuman animal studies have yielded inconsistent results, human investigation is limited. In our previous work (Chen et al., 2019), we introduced a novel spatial navigation paradigm utilizing ultra-high field fMRI to explore neural coding of positional information. We found that different entorhinal subregions in the right hemisphere encode positional information for landmarks and self-motion cues. The present study tested the generalizability of our previous finding with a modified navigation paradigm. Although we did not replicate our previous finding in the entorhinal cortex, we identified adaptation-based allocentric positional codes for both cue types in the retrosplenial cortex (RSC), which were not confounded by the path to the spatial location. Crucially, the multi-voxel patterns of these spatial codes differed between the cue types, suggesting cue-specific positional coding. The parahippocampal cortex exhibited positional coding for self-motion cues, which was not dissociable from path length. Finally, the brain regions involved in successful navigation differed from our previous study, indicating overall distinct neural mechanisms recruited in our two studies. Taken together, the current findings demonstrate cue-specific allocentric positional coding in the human RSC in the same navigation task for the first time and that spatial representations in the brain are contingent on specific experimental conditions.

Changes in spatial self-consciousness elicit grid cell-like representation in the entorhinal cortex

Grid cells in the entorhinal cortex (EC) encode an individual's location in space, integrating both environmental and multisensory bodily cues. Notably, body-derived signals are also primary signals for the sense of self. While studies have demonstrated that continuous application of visuo-tactile bodily stimuli can induce perceptual shifts in self-location, it remains unexplored whether these illusory changes suffice to trigger grid cell-like representation (GCLR) within the EC, and how this compares to GCLR during conventional virtual navigation. To address this, we systematically induced illusory drifts in self-location toward controlled directions using visuo-tactile bodily stimulation, while maintaining the subjects' visual viewpoint fixed (absent conventional virtual navigation). Subsequently, we evaluated the corresponding GCLR in the EC through functional MRI analysis. Our results reveal that illusory changes in perceived self-location (independent of changes in environmental navigation cues) can indeed evoke entorhinal GCLR, correlating in strength with the magnitude of perceived self-location, and characterized by similar grid orientation as during conventional virtual navigation in the same virtual room. These data demonstrate that the same grid-like representation is recruited when navigating based on environmental, mainly visual cues, or when experiencing illusory forward drifts in self-location, driven by perceptual multisensory bodily cues.

Enhancing spatial navigation skills in mild cognitive impairment patients: a usability study of a new version of ANTaging software

Introduction

Mild Cognitive Impairment (MCI) often presents challenges related to spatial navigation and retention of spatial information. Navigating space involves intricate integration of bodily and environmental cues. Spatial memory is dependent on two distinct frame of reference systems for organizing this information: egocentric and allocentric frames of reference. Virtual Reality (VR) has emerged as a promising technology for enhancing spatial navigation skills and spatial memory by facilitating the manipulation of bodily, environmental, and cognitive cues.

Methods

This usability study was based on a fully within-subjects design in which seven MCI patients underwent two kinds of VR conditions: participants were required to complete the ANTaging demo both in Oculus Rift S (immersive condition) and in Samsung UHD 4K monitor (semi-immersive condition). Participants were seated and they had to use a foot-motion pad to navigate and explore the environment to collect and relocate some objects in the virtual environment. Post-interaction, users provided feedback on their experiences. Additionally, usability, potential side effects, data analysis feasibility, and user preferences with immersive and semi-immersive technologies were assessed through questionnaires.

Results

Results indicated higher usability ratings for the semi-immersive setup, with fewer negative effects reported compared to the immersive counterpart. According to qualitative analyses of the interviews, patients do seem to like both VR apparatuses even though the semi-immersive condition was perceived as the most suitable choice because of the size of the screen. Patients generally found it difficult to remember object locations. Participants expressed the need for more practice with the foot-motion pad, despite an overall positive experience. They generally would like to use this system to improve their memory.

Discussion

Identifying these key aspects was crucial for refining the system before the upcoming clinical trial. This study sheds light on the potential of semi-immersive VR in aiding individuals with MCI, paving the way for enhanced spatial navigation interventions.

Embodied Spatial Navigation Training in Mild Cognitive Impairment: A Proof-of-Concept Trial

Background

Egocentric and allocentric spatial memory impairments affect the navigation abilities of older adults with mild cognitive impairment (MCI). Embodied cognition research hints that specific aids can be implemented into virtual reality (VR) training to enhance spatial memory.

Objective

In this study, we preliminarily tested 'ANTaging', an embodied-based immersive VR training for egocentric and allocentric memory, compared to treatment as usual (TAU) spatial training in MCI.

Methods

MCI patients were recruited for this controlled trial. A cognitive battery was administered at pre-test, after ten sessions of ANTaging or TAU intervention, and at 3-month follow-up (FU). The primary outcomes were spatial cognition tests (Corsi supra-span, CSS; Manikin test, MT). VR egocentric and allocentric performance was also collected.

Results

We found that ANTaging significantly improved MT scores at FU compared to TAU. CSS slightly improved in both groups. Concerning secondary outcomes, auditory-verbal forgetting significantly improved at post-test in the ANTaging but not TAU group and significantly declined at FU in the TAU but not in the ANTaging group. Global cognition significantly improved at FU for TAU and remained stable for ANTaging. Other tests showed no improvement or deterioration. Clinical significance showed that ANTaging is effective for CSS. Virtual egocentric and allocentric memory performance improved across ANTaging sessions.

Conclusions

ANTaging holds the potential to be superior for improving spatial cognition in MCI compared to TAU. Embodied cognition research provides insights for designing effective spatial navigation rehabilitation in aging.

Bodily and Visual-Cognitive Navigation Aids to Enhance Spatial Recall in Mild Cognitive Impairment

Background

Individuals with mild cognitive impairment (MCI) syndrome often report navigation difficulties, accompanied by impairments in egocentric and allocentric spatial memory. However, studies have shown that both bodily (e.g., motor commands, proprioception, vestibular information) and visual-cognitive (e.g., maps, directional arrows, attentional markers) cues can support spatial memory in MCI.

Objective

We aimed to assess navigation cues for innovative spatial training in aging.

Methods

Fifteen MCI patients were recruited for this study. Their egocentric and allocentric memory recall performances were tested through a navigation task with five different virtual reality (VR) assistive encoding navigation procedures (bodily, vision only, interactive allocentric map, reduced executive load, free navigation without cues). Bodily condition consisted of an immersive VR setup to engage self-motion cues, vision only condition consisted of passive navigation without interaction, in the interactive allocentric map condition patients could use a bird-view map, in the reduced executive load condition directional cues and attentional markers were employed, and during free navigation no aid was implemented.

Results

Bodily condition improved spatial memory compared to vision only and free navigation without cues. In addition, the interactive allocentric map was superior to the free navigation without cues. Surprisingly, the reduced executive load was comparable to vison only condition. Moreover, a detrimental impact of free navigation was observed on allocentric memory across testing trials.

Conclusions

These findings challenge the notion of an amodal representation of space in aging, suggesting that spatial maps can be influenced by the modality in which the environment was originally encoded.

Physical Body Orientation Impacts Virtual Navigation Experience and Performance

Most human navigation studies in MRI rely on virtual navigation. However, the necessary supine position in MRI makes it fundamentally different from daily ecological navigation. Nonetheless, until now, no study has assessed whether differences in physical body orientation (BO) affect participants' experienced BO during virtual navigation. Here, combining an immersive virtual reality navigation task with subjective BO measures and implicit behavioral measures, we demonstrate that physical BO (either standing or supine) modulates experienced BO. Also, we show that standing upright BO is preferred during spatial navigation: participants were more likely to experience a standing BO and were better at spatial navigation when standing upright. Importantly, we report that showing a supine virtual agent reduces the conflict between the preferred BO and physical supine BO. Our study provides critical, but missing, information regarding experienced BO during virtual navigation, which should be considered cautiously when designing navigation studies, especially in MRI.

Mobile cognition: imaging the human brain in the 'real world'

Cognitive neuroscience studies in humans have enabled decades of impactful discoveries but have primarily been limited to recording the brain activity of immobile participants in a laboratory setting. In recent years, advances in neuroimaging technologies have enabled recordings of human brain activity to be obtained during freely moving behaviours in the real world. Here, we propose that these mobile neuroimaging methods can provide unique insights into the neural mechanisms of human cognition and contribute to the development of novel treatments for neurological and psychiatric disorders. We further discuss the challenges associated with studying naturalistic human behaviours in complex real-world settings as well as strategies for overcoming them. We conclude that mobile neuroimaging methods have the potential to bring about a new era of cognitive neuroscience in which neural mechanisms can be studied with increased ecological validity and with the ability to address questions about natural behaviour and cognitive processes in humans engaged in dynamic real-world experiences.

Contribution of cognitive and bodily navigation cues to egocentric and allocentric spatial memory in hallucinations due to Parkinson's disease: A case report

Parkinson's disease (PD) manifestations can include visual hallucinations and illusions. Recent findings suggest that the coherent integration of bodily information within an egocentric representation could play a crucial role in these phenomena. Egocentric processing is a key aspect of spatial navigation and is supported by the striatum. Due to the deterioration of the striatal and motor systems, PD mainly impairs the egocentric rather than the allocentric spatial frame of reference. However, it is still unclear the interplay between spatial cognition and PD hallucinations and how different navigation mechanisms can influence such spatial frames of reference. We report the case of A.A., a patient that suffers from PD with frequent episodes of visual hallucinations and illusions. We used a virtual reality (VR) navigation task to assess egocentric and allocentric spatial memory under five navigation conditions (passive, immersive, map, path decision, and attentive cues) in A.A. and a PD control group without psychosis. In general, A.A. exhibited a statistically significant classical dissociation between the egocentric and allocentric performance with a greater deficit for the former. In particular, the dissociation was statistically significant in the “passive” and “attentive cues” conditions. Interestingly in the “immersive” condition, the dissociation was not significant and, in contrast to the other conditions, trends showed better performance for egocentric than allocentric memory. Within the theories of embodiment, we suggest that body-based information, as assessed with VR navigation tasks, could play an important role in PD hallucinations. In addition, the possible neural underpinnings and the usefulness of VR are discussed.

Building Embodied Spaces for Spatial Memory Neurorehabilitation with Virtual Reality in Normal and Pathological Aging

Along with deficits in spatial cognition, a decline in body-related information is observed in aging and is thought to contribute to impairments in navigation, memory, and space perception. According to the embodied cognition theories, bodily and environmental information play a crucial role in defining cognitive representations. Thanks to the possibility to involve body-related information, manipulate environmental stimuli, and add multisensory cues, virtual reality is one of the best candidates for spatial memory rehabilitation in aging for its embodied potential. However, current virtual neurorehabilitation solutions for aging and neurodegenerative diseases are in their infancy. Here, we discuss three concepts that could be used to improve embodied representations of the space with virtual reality. The virtual bodily representation is the combination of idiothetic information involved during virtual navigation thanks to input/output devices; the spatial affordances are environmental or symbolic elements used by the individual to act in the virtual environment; finally, the virtual enactment effect is the enhancement on spatial memory provided by actively (cognitively and/or bodily) interacting with the virtual space and its elements. Theoretical and empirical findings will be presented to propose innovative rehabilitative solutions in aging for spatial memory and navigation.

An Important Step toward Understanding the Role of Body-based Cues on Human Spatial Memory for Large-Scale Environments

Moving our body through space is fundamental to human navigation; however, technical and physical limitations have hindered our ability to study the role of these body-based cues experimentally. We recently designed an experiment using novel immersive virtual-reality technology, which allowed us to tightly control the availability of body-based cues to determine how these cues influence human spatial memory [Huffman, D. J., & Ekstrom, A. D. A modality-independent network underlies the retrieval of large-scale spatial environments in the human brain. Neuron, 104, 611-622, 2019]. Our analysis of behavior and fMRI data revealed a similar pattern of results across a range of body-based cues conditions, thus suggesting that participants likely relied primarily on vision to form and retrieve abstract, holistic representations of the large-scale environments in our experiment. We ended our paper by discussing a number of caveats and future directions for research on the role of body-based cues in human spatial memory. Here, we reiterate and expand on this discussion, and we use a commentary in this issue by A. Steel, C. E. Robertson, and J. S. Taube (Current promises and limitations of combined virtual reality and functional magnetic resonance imaging research in humans: A commentary on Huffman and Ekstrom (2019). Journal of Cognitive Neuroscience, 2020) as a helpful discussion point regarding some of the questions that we think will be the most interesting in the coming years. We highlight the exciting possibility of taking a more naturalistic approach to study the behavior, cognition, and neuroscience of navigation. Moreover, we share the hope that researchers who study navigation in humans and nonhuman animals will synergize to provide more rapid advancements in our understanding of cognition and the brain.