Landmarks: A solution for spatial navigation and memory experiments in virtual reality

Age differences in spatial memory are mitigated during naturalistic navigation

Spatial navigation deficits are often observed among older adults on tasks that require navigating virtual reality (VR) environments on a computer screen. We investigated whether these age differences are attenuated when tested in more naturalistic and ambulatory virtual environments. In Experiment 1, young and older adults navigated a variant of the Morris Water Maze task in each of two VR conditions: a desktop VR condition which required using a mouse and keyboard to navigate, and an ambulatory VR condition which permitted unrestricted locomotion. In Experiment 2, we examined whether age- and VR-related differences in spatial performance were affected by the inclusion of additional spatial cues. In both experiments, older adults navigated to target locations less precisely than younger individuals in the desktop condition. Age differences were significantly attenuated, however, when tested in the ambulatory VR environment. These findings underscore the importance of developing naturalistic assessments of spatial memory and navigation.

Neural networks need real-world behavior

Bowers et al. propose to use controlled behavioral experiments when evaluating deep neural networks as models of biological vision. We agree with the sentiment and draw parallels to the notion that "neuroscience needs behavior." As a promising path forward, we suggest complementing image recognition tasks with increasingly realistic and well-controlled task environments that engage real-world object recognition behavior.

Frontal-midline theta and posterior alpha oscillations index early processing of spatial representations during active navigation

Previous research has demonstrated that humans combine multiple sources of spatial information such as self-motion and landmark cues while navigating through an environment. However, it is unclear whether this involves comparing multiple representations obtained from different sources during navigation (parallel hypothesis) or building a representation first based on self-motion cues and then combining with landmarks later (serial hypothesis). We tested these two hypotheses (parallel vs serial) in an active navigation task using wireless mobile scalp EEG recordings. Participants walked through an immersive virtual hallway with or without conflicts between self-motion and landmarks (i.e., intersections) and pointed toward the starting position of the hallway. We employed the oscillatory signals recorded during mobile wireless scalp EEG as a means of identifying when participant representations based on self-motion versus landmark cues might have first emerged. We found that path segments, including intersections present early during navigation, were more strongly associated with later pointing error, regardless of when they appeared during encoding. We also found that there was sufficient information contained within the frontal-midline theta and posterior alpha oscillatory signals in the earliest segments of navigation involving intersections to decode condition (i.e., conflicting vs not conflicting). Together, these findings suggest that intersections play a pivotal role in the early development of spatial representations, suggesting that memory representations for the geometry of walked paths likely develop early during navigation, in support of the parallel hypothesis.

The Virtual Navigation Toolbox: Providing tools for virtual navigation experiments

Spatial navigation research in humans increasingly relies on experiments using virtual reality (VR) tools, which allow for the creation of highly flexible, and immersive study environments, that can react to participant interaction in real time. Despite the popularity of VR, tools simplifying the creation and data management of such experiments are rare and often restricted to a specific scope-limiting usability and comparability. To overcome those limitations, we introduce the Virtual Navigation Toolbox (VNT), a collection of interchangeable and independent tools for the development of spatial navigation VR experiments using the popular Unity game engine. The VNT's features are packaged in loosely coupled and reusable modules, facilitating convenient implementation of diverse experimental designs. Here, we depict how the VNT fulfils feature requirements of different VR environments and experiments, guiding through the implementation and execution of a showcase study using the toolbox. The presented showcase study reveals that homing performance in a classic triangle completion task is invariant to translation velocity of the participant's avatar, but highly sensitive to the number of landmarks. The VNT is freely available under a creative commons license, and we invite researchers to contribute, extending and improving tools using the provided repository.

A virtual reality paradigm with dynamic scene stimuli for use in memory research

Episodic memory may essentially be memory for one's place within a temporally unfolding scene from a first-person perspective. Given this, pervasively used static stimuli may only capture one small part of episodic memory. A promising approach for advancing the study of episodic memory is immersing participants within varying scenes from a first-person perspective. We present a pool of distinct scene stimuli for use in virtual environments and a paradigm that is implementable across varying levels of immersion on multiple virtual reality (VR) platforms and adaptable to studying various aspects of scene and episodic memory. In our task, participants are placed within a series of virtual environments from a first-person perspective and guided through a virtual tour of scenes during a study phase and a test phase. In the test phase, some scenes share a spatial layout with studied scenes; others are completely novel. In three experiments with varying degrees of immersion, we measure scene recall, scene familiarity-detection during recall failure, the subjective experience of déjà vu, the ability to predict the next turn on a tour, the subjective sense of being able to predict the next turn on a tour, and the factors that influence memory search and the inclination to generate candidate recollective information. The level of first-person immersion mattered to multiple facets of episodic memory. The paradigm presents a useful means of advancing mechanistic understanding of how memory operates in realistic dynamic scene environments, including in combination with cognitive neuroscience methods such as functional magnetic resonance imaging and electrophysiology.

Age differences in spatial memory are mitigated during naturalistic navigation

Spatial navigation deficits in older adults are well documented. These findings are often based on experimental paradigms that require using a joystick or keyboard to navigate a virtual desktop environment. In the present study, we investigated whether age differences in spatial memory are attenuated when tested in a more naturalistic and ambulatory virtual environment. In Experiment 1, cognitively normal young and older adults navigated a virtual variant of the Morris Water Maze task in each of two virtual reality (VR) conditions: a desktop VR condition which required using a mouse and keyboard to navigate and an immersive and ambulatory VR condition which permitted unrestricted locomotion. In Experiment 2, we examined whether age- and VR-related differences in spatial performance were affected by the inclusion of additional spatial cues in an independent sample of young and older adults. In both experiments, older adults navigated to target locations less precisely than did younger individuals in the desktop condition, replicating numerous prior studies. These age differences were significantly attenuated, however, when tested in the fully immersive and ambulatory environment. These findings underscore the importance of developing naturalistic and ecologically valid measures of spatial memory and navigation, especially when performing cross-sectional studies of cognitive aging.

Frontal-midline theta and posterior alpha oscillations index early processing of spatial representations during active navigation

Previous research has demonstrated that humans combine multiple sources of spatial information such as self-motion and landmark cues, while navigating through an environment. However, it is unclear whether this involves comparing multiple representations obtained from different sources during navigation (parallel hypothesis) or building a representation first based on self-motion cues and then combining with landmarks later (serial hypothesis). We tested these two hypotheses (parallel vs. serial) in an active navigation task using wireless mobile scalp EEG recordings. Participants walked through an immersive virtual hallway with or without conflicts between self-motion and landmarks (i.e., intersections) and pointed toward the starting position of the hallway. We employed the oscillatory signals recorded during mobile wireless scalp EEG as means of identifying when participant representations based on self-motion vs. landmark cues might have first emerged. We found that path segments, including intersections present early during navigation, were more strongly associated with later pointing error, regardless of when they appeared during encoding. We also found that there was sufficient information contained within the frontal-midline theta and posterior alpha oscillatory signals in the earliest segments of navigation involving intersections to decode condition (i.e., conflicting vs. not conflicting). Together, these findings suggest that intersections play a pivotal role in the early development of spatial representations, suggesting that memory representations for the geometry of walked paths likely develop early during navigation, in support of the parallel hypothesis.

Spatial memory distortions for the shapes of walked paths occur in violation of physically experienced geometry

An important question regards the nature of our spatial memories for the paths that we have walked and, in particular, whether such distortions might violate the topological properties of the shape of the paths (i.e., creating an intersection when two paths did not intersect or vice versa). To investigate whether and how this might occur, we tested humans in situations in which they walked simple paths and idiothetic and visual cues either matched or mismatched, with the mismatching cues creating the greatest potential for topological distortions. Participants walked four-segment paths with 90° turns in immersive virtual reality and pointed to their start location when they arrived at the end of the path. In paths with a crossing, when the intersection was not presented, participants pointed to a novel start location suggesting a topological distortion involving non-crossed paths. In paths without a crossing, when a false intersection was presented, participants pointed to a novel start location suggesting a topological distortion involving crossed paths. In paths without crossings and without false intersections, participants showed reduced pointing errors that typically did not involve topological distortions. Distortions more generally, as indicated by pointing errors to the start location, were significantly reduced for walked paths involving primarily idiothetic cues with limited visual cues; conversely, distortions were significantly increased when idiothetic cues were diminished and navigation relied primarily on visual cues. Our findings suggest that our spatial memories for walked paths sometimes involve topological distortions, particularly when resolving the competition between idiothetic and visual cues.

vexptoolbox: A software toolbox for human behavior studies using the Vizard virtual reality platform

Virtual reality (VR) is a powerful tool for researchers due to its potential to study dynamic human behavior in highly naturalistic environments while retaining full control over the presented stimuli. Due to advancements in consumer hardware, VR devices are now very affordable and have also started to include technologies such as eye tracking, further extending potential research applications. Rendering engines such as Unity, Unreal, or Vizard now enable researchers to easily create complex VR environments. However, implementing the experimental design can still pose a challenge, and these packages do not provide out-of-the-box support for trial-based behavioral experiments. Here, we present a Python toolbox, designed to facilitate common tasks when developing experiments using the Vizard VR platform. It includes functionality for common tasks like creating, randomizing, and presenting trial-based experimental designs or saving results to standardized file formats. Moreover, the toolbox greatly simplifies continuous recording of eye and body movements using any hardware supported in Vizard. We further implement and describe a simple goal-directed reaching task in VR and show sample data recorded from five volunteers. The toolbox, example code, and data are all available on GitHub under an open-source license. We hope that our toolbox can simplify VR experiment development, reduce code duplication, and aid reproducibility and open-science efforts.

Virtual Reality for Spatial Navigation

Immersive virtual reality (VR) allows its users to experience physical space in a non-physical world. It has developed into a powerful research tool to investigate the neural basis of human spatial navigation as an embodied experience. The task of wayfinding can be carried out by using a wide range of strategies, leading to the recruitment of various sensory modalities and brain areas in real-life scenarios. While traditional desktop-based VR setups primarily focus on vision-based navigation, immersive VR setups, especially mobile variants, can efficiently account for motor processes that constitute locomotion in the physical world, such as head-turning and walking. When used in combination with mobile neuroimaging methods, immersive VR affords a natural mode of locomotion and high immersion in experimental settings, designing an embodied spatial experience. This in turn facilitates ecologically valid investigation of the neural underpinnings of spatial navigation.

A virtual reality platform for memory evaluation: Assessing effects of spatial strategies

Human spatial memories are usually evaluated using computer screens instead of real arenas or landscapes where subjects could move voluntarily and use allocentric cues to guide their behavior. A possible approach to fill this gap is the adoption of virtual reality, which provides the opportunity to create spatial memory tasks closer to real-life experience. Here we present and evaluate a new software to create experiments using this technology. Specifically, we have developed a spatial memory task that is carried out in a computer-assisted virtual environment where participants walk around a virtual arena using a joystick. This spatial memory task provides an immersive environment where the spatial component is constantly present without the use of virtual reality goggles. The design is similar to that of tasks used for animal studies, allowing a direct comparison across species. We found that only participants who reported using spatial cues to guide their behavior showed significant learning and performed significantly better during a memory test. This tool allows evaluation of human spatial memory in an ecological environment and will be useful to develop a wide range of other tasks to assess spatial cognition.

Combining egoformative and alloformative cues in a novel tabletop navigation task

Previous work has shown how different interfaces (i.e., route navigation, maps, or a combination of the two) influence spatial knowledge and recollection. To test for the existence of intermediate representations along an egocentric-to-allocentric continuum, we developed a novel task, tabletop navigation, to provide a mixture of cues that inform the emergence of egocentric and allocentric representations or strategies. In this novel tabletop task, participants navigated a remote-controlled avatar through a tabletop scale model of the virtual city. Participants learned virtual cities from either navigating routes, studying maps, or our new tabletop navigation task. We interleaved these learning tasks with either an in situ pointing task (the scene- and orientation-dependent pointing [SOP] task) or imagined judgements of relative direction (JRD) pointing. In Experiment 1, performance on each memory task was similar across learning tasks and performance on the route and map learning tasks correlated with more precise spatial recall on both the JRD and SOP tasks. Tabletop learning performance correlated with SOP performance only, suggesting a reliance on egocentric strategies, although increased utilization of the affordances of the tabletop task were related to JRD performance. In Experiment 2, using a modified criterion map learning task, participants who learned using maps provided more precise responses on the JRD compared to route or tabletop learning. Together, these findings provide mixed evidence for both optimization and egocentric predominance after learning from the novel tabletop navigation task.

OpenMaze: An open-source toolbox for creating virtual navigation experiments

Incorporating 3D virtual environments into psychological experiments offers an innovative solution for balancing experimental control and ecological validity. Their flexible application to virtual navigation experiments, however, has been limited because accessible development tools best support only a subset of desirable task design features. We created OpenMaze, an open-source toolbox for the Unity game engine, to overcome this barrier. OpenMaze offers researchers the ability to conduct a wide range of first-person spatial navigation experiment paradigms in fully customized 3D environments. Crucially, because all experiments are defined using human-readable configuration files, our toolbox allows even those with no prior coding experience to build bespoke tasks. OpenMaze is also compatible with a variety of input devices and operating systems, broadening its possible applications. To demonstrate its advantages and limitations, we review and contrast other available software options before providing an overview of our design objectives and walking the reader through the process of building an experiment in OpenMaze.

DeFINE: Delayed feedback-based immersive navigation environment for studying goal-directed human navigation

With the advent of consumer-grade products for presenting an immersive virtual environment (VE), there is a growing interest in utilizing VEs for testing human navigation behavior. However, preparing a VE still requires a high level of technical expertise in computer graphics and virtual reality, posing a significant hurdle to embracing the emerging technology. To address this issue, this paper presents Delayed Feedback-based Immersive Navigation Environment (DeFINE), a framework that allows for easy creation and administration of navigation tasks within customizable VEs via intuitive graphical user interfaces and simple settings files. Importantly, DeFINE has a built-in capability to provide performance feedback to participants during an experiment, a feature that is critically missing in other similar frameworks. To show the usability of DeFINE from both experimentalists' and participants' perspectives, a demonstration was made in which participants navigated to a hidden goal location with feedback that differentially weighted speed and accuracy of their responses. In addition, the participants evaluated DeFINE in terms of its ease of use, required workload, and proneness to induce cybersickness. The demonstration exemplified typical experimental manipulations DeFINE accommodates and what types of data it can collect for characterizing participants' task performance. With its out-of-the-box functionality and potential customizability due to open-source licensing, DeFINE makes VEs more accessible to many researchers.

Older Adults Show Reduced Spatial Precision but Preserved Strategy-Use During Spatial Navigation Involving Body-Based Cues

Older adults typically perform worse on spatial navigation tasks, although whether this is due to degradation of memory or an impairment in using specific strategies has yet to be determined. An issue with some past studies is that older adults are tested on desktop-based virtual reality: a technology many report lacking familiarity with. Even when controlling for familiarity, these paradigms reduce the information-rich, three-dimensional experience of navigating to a simple two-dimensional task that utilizes a mouse and keyboard (or joystick) as means for ambulation. Here, we utilize a wireless head-mounted display and free ambulation to create a fully immersive virtual Morris water maze in which we compare the navigation of older and younger adults. Older and younger adults learned the locations of hidden targets from same and different start points. Across different conditions tested, older adults remembered target locations less precisely compared to younger adults. Importantly, however, they performed comparably from the same viewpoint as a switched viewpoint, suggesting that they could generalize their memory for the location of a hidden target given a new point of view. When we implicitly moved one of the distal cues to determine whether older adults used an allocentric (multiple landmarks) or beaconing (single landmark) strategy to remember the hidden target, both older and younger adults showed comparable degrees of reliance on allocentric and beacon cues. These findings support the hypothesis that while older adults have less precise spatial memories, they maintain the ability to utilize various strategies when navigating.