Hexadirectional coding of visual space in human entorhinal cortex

Interactions between neural representations of the social and spatial environment

Even in our highly interconnected modern world, geographic factors play an important role in human social connections. Similarly, social relationships influence how and where we travel, and how we think about our spatial world. Here, we review the growing body of neuroscience research that is revealing multiple interactions between social and spatial processes in both humans and non-human animals. We review research on the cognitive and neural representation of spatial and social information, and highlight recent findings suggesting that underlying mechanisms might be common to both. We discuss how spatial factors can influence social behaviour, and how social concepts modify representations of space. In so doing, this review elucidates not only how neural representations of social and spatial information interact but also similarities in how the brain represents and operates on analogous information about its social and spatial surroundings.This article is part of the theme issue 'The spatial-social interface: a theoretical and empirical integration'.

Concept and location neurons in the human brain provide the 'what' and 'where' in memory formation

Our brains create new memories by capturing the 'who/what', 'where' and 'when' of everyday experiences. On a neuronal level, mechanisms facilitating a successful transfer into episodic memory are still unclear. We investigated this by measuring single neuron activity in the human medial temporal lobe during encoding of item-location associations. While previous research has found predictive effects in population activity in human MTL structures, we could attribute such effects to two specialized sub-groups of neurons: concept cells in the hippocampus, amygdala and entorhinal cortex (EC), and a second group of parahippocampal location-selective neurons. In both item- and location-selective populations, firing rates were significantly higher during successfully encoded trials. These findings are in line with theories of hippocampal indexing, since selective index neurons may act as pointers to neocortical representations. Overall, activation of distinct populations of neurons could directly support the connection of the 'what' and 'where' of episodic memory.

Spatial updating of gaze position in younger and older adults - A path integration-like process in eye movements

Path integration (PI) is a navigation process that allows an organism to update its current location in reference to a starting point. PI can involve updating self-position continuously with respect to the starting point (continuous updating) or creating a map representation of the route which is then used to compute the homing vector (configural updating). One of the brain areas involved in PI, the entorhinal cortex, is modulated similarly by whole-body and eye movements, suggesting that if PI updates self-position, an analogous process may be used to update gaze position, and may undergo age-related changes. Here, we created an eyetracking version of a PI task in which younger and older participants followed routes with their eyes as guided by visual onsets; at the end of each route, participants were cued to return to the starting point or another enroute location. When only memory for the starting location was required for successful task performance, younger and older adults were generally not influenced by the number of locations, indicative of continuous updating. However, when participants could be cued to any enroute location, thereby requiring memory for the entire route, processing times increased, accuracy decreased, and overt revisits to enroute locations increased with the number of locations in a route, indicative of configural updating. Older participants showed evidence for similar updating strategies as younger participants, but they were less accurate and made more overt revisits to mid-route locations. These findings suggest that spatial updating mechanisms are generalizable across effector systems.

Quantitative modeling of the emergence of macroscopic grid-like representations

When subjects navigate through spatial environments, grid cells exhibit firing fields that are arranged in a triangular grid pattern. Direct recordings of grid cells from the human brain are rare. Hence, functional magnetic resonance imaging (fMRI) studies proposed an indirect measure of entorhinal grid-cell activity, quantified as hexadirectional modulation of fMRI activity as a function of the subject's movement direction. However, it remains unclear how the activity of a population of grid cells may exhibit hexadirectional modulation. Here, we use numerical simulations and analytical calculations to suggest that this hexadirectional modulation is best explained by head-direction tuning aligned to the grid axes, whereas it is not clearly supported by a bias of grid cells toward a particular phase offset. Firing-rate adaptation can result in hexadirectional modulation, but the available cellular data is insufficient to clearly support or refute this option. The magnitude of hexadirectional modulation furthermore depends considerably on the subject's navigation pattern, indicating that future fMRI studies could be designed to test which hypothesis most likely accounts for the fMRI measure of grid cells. Our findings also underline the importance of quantifying the properties of human grid cells to further elucidate how hexadirectional modulations of fMRI activity may emerge.

Memory traces of duration and location in the right intraparietal sulcus

Time and space form an integral part of every human experience, and for the neuronal representation of these perceptual dimensions, previous studies point to the involvement of the right-hemispheric intraparietal sulcus and structures in the medial temporal lobe. Here we used multi-voxel pattern analysis (MVPA) to investigate long-term memory traces for temporal and spatial stimulus features in those areas. Participants were trained on four images associated with short versus long durations and with left versus right locations. Our results demonstrate stable representations of both temporal and spatial information in the right posterior intraparietal sulcus. Building upon previous findings of stable neuronal codes for directly perceived durations and locations, these results show that the reactivation of long-term memory traces for temporal and spatial features can be decoded from neuronal activation patterns in the right parietal cortex.

A theory of hippocampal function: New developments

We develop further here the only quantitative theory of the storage of information in the hippocampal episodic memory system and its recall back to the neocortex. The theory is upgraded to account for a revolution in understanding of spatial representations in the primate, including human, hippocampus, that go beyond the place where the individual is located, to the location being viewed in a scene. This is fundamental to much primate episodic memory and navigation: functions supported in humans by pathways that build 'where' spatial view representations by feature combinations in a ventromedial visual cortical stream, separate from those for 'what' object and face information to the inferior temporal visual cortex, and for reward information from the orbitofrontal cortex. Key new computational developments include the capacity of the CA3 attractor network for storing whole charts of space; how the correlations inherent in self-organizing continuous spatial representations impact the storage capacity; how the CA3 network can combine continuous spatial and discrete object and reward representations; the roles of the rewards that reach the hippocampus in the later consolidation into long-term memory in part via cholinergic pathways from the orbitofrontal cortex; and new ways of analysing neocortical information storage using Potts networks.

Electrophysiological signatures of veridical head direction in humans

Information about heading direction is critical for navigation as it provides the means to orient ourselves in space. However, given that veridical head-direction signals require physical rotation of the head and most human neuroimaging experiments depend upon fixing the head in position, little is known about how the human brain is tuned to such heading signals. Here we adress this by asking 52 healthy participants undergoing simultaneous electroencephalography and motion tracking recordings (split into two experiments) and 10 patients undergoing simultaneous intracranial electroencephalography and motion tracking recordings to complete a series of orientation tasks in which they made physical head rotations to target positions. We then used a series of forward encoding models and linear mixed-effects models to isolate electrophysiological activity that was specifically tuned to heading direction. We identified a robust posterior central signature that predicts changes in veridical head orientation after regressing out confounds including sensory input and muscular activity. Both source localization and intracranial analysis implicated the medial temporal lobe as the origin of this effect. Subsequent analyses disentangled head-direction signatures from signals relating to head rotation and those reflecting location-specific effects. Lastly, when directly comparing head direction and eye-gaze-related tuning, we found that the brain maintains both codes while actively navigating, with stronger tuning to head direction in the medial temporal lobe. Together, these results reveal a taxonomy of population-level head-direction signals within the human brain that is reminiscent of those reported in the single units of rodents.

Distance and grid-like codes support the navigation of abstract social space in the human brain

People form impressions about others during daily social encounters and infer personality traits from others' behaviors. Such trait inference is thought to rely on two universal dimensions: competence and warmth. These two dimensions can be used to construct a 'social cognitive map' organizing massive information obtained from social encounters efficiently. Originating from spatial cognition, the neural codes supporting the representation and navigation of spatial cognitive maps have been widely studied. Recent studies suggest similar neural mechanism subserves the map-like architecture in social cognition as well. Here we investigated how spatial codes operate beyond the physical environment and support the representation and navigation of social cognitive map. We designed a social value space defined by two dimensions of competence and warmth. Behaviorally, participants were able to navigate to a learned location from random starting locations in this abstract social space. At the neural level, we identified the representation of distance in the precuneus, fusiform gyrus, and middle occipital gyrus. We also found partial evidence of grid-like representation patterns in the medial prefrontal cortex and entorhinal cortex. Moreover, the intensity of grid-like response scaled with the performance of navigating in social space and social avoidance trait scores. Our findings suggest a neurocognitive mechanism by which social information can be organized into a structured representation, namely cognitive map and its relevance to social well-being.

Robust variability of grid cell properties within individual grid modules enhances encoding of local space

Although grid cells are one of the most well studied functional classes of neurons in the mammalian brain, the assumption that there is a single grid orientation and spacing per grid module has not been carefully tested. We investigate and analyze a recent large-scale recording of medial entorhinal cortex to characterize the presence and degree of heterogeneity of grid properties within individual modules. We find evidence for small, but robust, variability and hypothesize that this property of the grid code could enhance the ability of encoding local spatial information. Performing analysis on synthetic populations of grid cells, where we have complete control over the amount heterogeneity in grid properties, we demonstrate that variability, of a similar magnitude to the analyzed data, leads to significantly decreased decoding error, even when restricted to activity from a single module. Our results highlight how the heterogeneity of the neural response properties may benefit coding and opens new directions for theoretical and experimental analysis of grid cells.

Evidence for grid-cell-like activity in the time domain

The relation between the processing of space and time in the brain has been an enduring cross-disciplinary question. Grid cells have been recognized as a hallmark of the mammalian navigation system, with recent studies attesting to their involvement in the organization of conceptual knowledge in humans. To determine whether grid-cell-like representations support temporal processing, we asked subjects to mentally simulate changes in age and time-of-day, each constituting "trajectory" in an age-day space, while undergoing fMRI. We found that grid-cell-like representations supported trajecting across this age-day space. Furthermore, brain regions concurrently coding past-to-future orientation positively modulated the magnitude of grid-cell-like representation in the left entorhinal cortex. Finally, our findings suggest that temporal processing may be supported by spatially modulated systems, and that innate regularities of abstract domains may interface and alter grid-cell-like representations, similarly to spatial geometry.

Altered grid-like coding in early blind people

Cognitive maps in the hippocampal-entorhinal system are central for the representation of both spatial and non-spatial relationships. Although this system, especially in humans, heavily relies on vision, the role of visual experience in shaping the development of cognitive maps remains largely unknown. Here, we test sighted and early blind individuals in both imagined navigation in fMRI and real-world navigation. During imagined navigation, the Human Navigation Network, constituted by frontal, medial temporal, and parietal cortices, is reliably activated in both groups, showing resilience to visual deprivation. However, neural geometry analyses highlight crucial differences between groups. A 60° rotational symmetry, characteristic of a hexagonal grid-like coding, emerges in the entorhinal cortex of sighted but not blind people, who instead show a 90° (4-fold) symmetry, indicative of a square grid. Moreover, higher parietal cortex activity during navigation in blind people correlates with the magnitude of 4-fold symmetry. In sum, early blindness can alter the geometry of entorhinal cognitive maps, possibly as a consequence of higher reliance on parietal egocentric coding during navigation.

Spontaneous eye movements reflect the representational geometries of conceptual spaces

Functional neuroimaging studies indicate that the human brain can represent concepts and their relational structure in memory using coding schemes typical of spatial navigation. However, whether we can read out the internal representational geometries of conceptual spaces solely from human behavior remains unclear. Here, we report that the relational structure between concepts in memory might be reflected in spontaneous eye movements during verbal fluency tasks: When we asked participants to randomly generate numbers, their eye movements correlated with distances along the left-to-right one-dimensional geometry of the number space (mental number line), while they scaled with distance along the ring-like two-dimensional geometry of the color space (color wheel) when they randomly generated color names. Moreover, when participants randomly produced animal names, eye movements correlated with low-dimensional similarity in word frequencies. These results suggest that the representational geometries used to internally organize conceptual spaces might be read out from gaze behavior.

Grid-like entorhinal representation of an abstract value space during prospective decision making

How valuable a choice option is often changes over time, making the prediction of value changes an important challenge for decision making. Prior studies identified a cognitive map in the hippocampal-entorhinal system that encodes relationships between states and enables prediction of future states, but does not inherently convey value during prospective decision making. In this fMRI study, participants predicted changing values of choice options in a sequence, forming a trajectory through an abstract two-dimensional value space. During this task, the entorhinal cortex exhibited a grid-like representation with an orientation aligned to the axis through the value space most informative for choices. A network of brain regions, including ventromedial prefrontal cortex, tracked the prospective value difference between options. These findings suggest that the entorhinal grid system supports the prediction of future values by representing a cognitive map, which might be used to generate lower-dimensional value signals to guide prospective decision making.

Grid codes underlie multiple cognitive maps in the human brain

Grid cells fire at multiple positions that organize the vertices of equilateral triangles tiling a 2D space and are well studied in rodents. The last decade witnessed rapid progress in two other research lines on grid codes-empirical studies on distributed human grid-like representations in physical and multiple non-physical spaces, and cognitive computational models addressing the function of grid cells based on principles of efficient and predictive coding. Here, we review the progress in these fields and integrate these lines into a systematic organization. We also discuss the coordinate mechanisms of grid codes in the human entorhinal cortex and medial prefrontal cortex and their role in neurological and psychiatric diseases.

Mental search of concepts is supported by egocentric vector representations and restructured grid maps

The human hippocampal-entorhinal system is known to represent both spatial locations and abstract concepts in memory in the form of allocentric cognitive maps. Using fMRI, we show that the human parietal cortex evokes complementary egocentric representations in conceptual spaces during goal-directed mental search, akin to those observable during physical navigation to determine where a goal is located relative to oneself (e.g., to our left or to our right). Concurrently, the strength of the grid-like signal, a neural signature of allocentric cognitive maps in entorhinal, prefrontal, and parietal cortices, is modulated as a function of goal proximity in conceptual space. These brain mechanisms might support flexible and parallel readout of where target conceptual information is stored in memory, capitalizing on complementary reference frames.

Multisensory input modulates memory-guided spatial navigation in humans

Efficient navigation is supported by a cognitive map of space. The hippocampus plays a key role for this map by linking multimodal sensory information with spatial memory representations. However, in human navigation studies, the full range of sensory information is often unavailable due to the stationarity of experimental setups. We investigated the contribution of multisensory information to memory-guided spatial navigation by presenting a virtual version of the Morris water maze on a screen and in an immersive mobile virtual reality setup. Patients with hippocampal lesions and matched controls navigated to memorized object locations in relation to surrounding landmarks. Our results show that availability of multisensory input improves memory-guided spatial navigation in both groups. It has distinct effects on navigational behaviour, with greater improvement in spatial memory performance in patients. We conclude that congruent multisensory information shifts computations to extrahippocampal areas that support spatial navigation and compensates for spatial navigation deficits.

MEG frequency tagging reveals a grid-like code during attentional movements

Grid-cells firing fields tile the environment with a 6-fold periodicity during both locomotion and visual exploration. Here, we tested, in humans, whether movements of covert attention elicit grid-like coding using frequency tagging. Participants observed visual trajectories presented sequentially at fixed rate, allowing different spatial periodicities (e.g., 4-, 6-, and 8-fold) to have corresponding temporal periodicities (e.g., 1, 1.5, and 2 Hz), thus resulting in distinct spectral responses. We found a higher response for the (grid-like) 6-fold periodicity and localized this effect in medial-temporal sources. In a control experiment featuring the same temporal periodicity but lacking spatial structure, the 6-fold effect did not emerge, suggesting its dependency on spatial movements of attention. We report evidence that grid-like signals in the human medial-temporal lobe can be elicited by covert attentional movements and suggest that attentional coding may provide a suitable mechanism to support the activation of cognitive maps during conceptual navigation.

Recruitment of grid-like responses in human entorhinal and piriform cortices by odor landmark-based navigation

Olfactory navigation is universal across the animal kingdom. Humans, however, have rarely been considered in this context. Here, we combined olfactometry techniques, virtual reality (VR) software, and neuroimaging methods to investigate whether humans can navigate an olfactory landscape by learning the spatial relationships among discrete odor cues and integrating this knowledge into a spatial map. Our data show that over time, participants improved their performance on the odor navigation task by taking more direct paths toward targets and completing more trials within a given time period. This suggests that humans can successfully navigate a complex odorous environment, reinforcing the notion of human olfactory navigation. fMRI data collected during the olfactory navigation task revealed the emergence of grid-like responses in entorhinal and piriform cortices that were attuned to the same grid orientation. This result implies the existence of a specialized olfactory grid network tasked with guiding spatial navigation based on odor landmarks.

A map of spatial navigation for neuroscience

Spatial navigation has received much attention from neuroscientists, leading to the identification of key brain areas and the discovery of numerous spatially selective cells. Despite this progress, our understanding of how the pieces fit together to drive behavior is generally lacking. We argue that this is partly caused by insufficient communication between behavioral and neuroscientific researchers. This has led the latter to under-appreciate the relevance and complexity of spatial behavior, and to focus too narrowly on characterizing neural representations of space-disconnected from the computations these representations are meant to enable. We therefore propose a taxonomy of navigation processes in mammals that can serve as a common framework for structuring and facilitating interdisciplinary research in the field. Using the taxonomy as a guide, we review behavioral and neural studies of spatial navigation. In doing so, we validate the taxonomy and showcase its usefulness in identifying potential issues with common experimental approaches, designing experiments that adequately target particular behaviors, correctly interpreting neural activity, and pointing to new avenues of research.

Empirically validated theoretical analysis of visual-spatial perception under change of nervous system arousal

Introduction

Visual-spatial perception is a process for extracting the spatial relationship between objects in the environment. The changes in visual-spatial perception due to factors such as the activity of the sympathetic nervous system (hyperactivation) or parasympathetic nervous system (hypoactivation) can affect the internal representation of the external visual-spatial world. We formulated a quantitative model of the modulation of visual-perceptual space under action by hyperactivation or hypoactivation-inducing neuromodulating agents. We showed a Hill equation based relationship between neuromodulator agent concentration and alteration of visual-spatial perception utilizing the metric tensor to quantify the visual space.

Methods

We computed the dynamics of the psilocybin (hyperactivation-inducing agent) and chlorpromazine (hypoactivation-inducing agent) in brain tissue. Then, we validated our quantitative model by analyzing the findings of different independent behavioral studies where subjects were assessed for alterations in visual-spatial perception under the action of psilocybin and under chlorpromazine. To validate the neuronal correlates, we simulated the effect of the neuromodulating agent on the computational model of the grid-cell network, and also performed diffusion MRI-based tractography to find the neural tracts between the cortical areas involved: V2 and the entorhinal cortex.

Results

We applied our computational model to an experiment (where perceptual alterations were measured under psilocybin) and found that for n (Hill-coefficient) = 14.8 and k = 1.39, the theoretical prediction followed experimental observations very well (χ2 test robustly satisfied, p > 0.99). We predicted the outcome of another psilocybin-based experiment using these values (n = 14.8 and k = 1.39), whereby our prediction and experimental outcomes were well corroborated. Furthermore, we found that also under hypoactivation (chlorpromazine), the modulation of the visual-spatial perception follows our model. Moreover, we found neural tracts between the area V2 and entorhinal cortex, thus providing a possible brain network responsible for encoding visual-spatial perception. Thence, we simulated the altered grid-cell network activity, which was also found to follow the Hill equation.

Conclusion

We developed a computational model of visuospatial perceptual alterations under altered neural sympathetic/parasympathetic tone. We validated our model using analysis of behavioral studies, neuroimaging assessment, and neurocomputational evaluation. Our quantitative approach may be probed as a potential behavioral screening and monitoring methodology in neuropsychology to analyze perceptual misjudgment and mishaps by highly stressed workers.

Computational cross-species views of the hippocampal formation

The discovery of place cells and head direction cells in the hippocampal formation of freely foraging rodents has led to an emphasis of its role in encoding allocentric spatial relationships. In contrast, studies in head-fixed primates have additionally found representations of spatial views. We review recent experiments in freely moving monkeys that expand upon these findings and show that postural variables such as eye/head movements strongly influence neural activity in the hippocampal formation, suggesting that the function of the hippocampus depends on where the animal looks. We interpret these results in the light of recent studies in humans performing challenging navigation tasks which suggest that depending on the context, eye/head movements serve one of two roles-gathering information about the structure of the environment (active sensing) or externalizing the contents of internal beliefs/deliberation (embodied cognition). These findings prompt future experimental investigations into the information carried by signals flowing between the hippocampal formation and the brain regions controlling postural variables, and constitute a basis for updating computational theories of the hippocampal system to accommodate the influence of eye/head movements.

Hippocampal spatial view cells for memory and navigation, and their underlying connectivity in humans

Hippocampal and parahippocampal gyrus spatial view neurons in primates respond to the spatial location being looked at. The representation is allocentric, in that the responses are to locations "out there" in the world, and are relatively invariant with respect to retinal position, eye position, head direction, and the place where the individual is located. The underlying connectivity in humans is from ventromedial visual cortical regions to the parahippocampal scene area, leading to the theory that spatial view cells are formed by combinations of overlapping feature inputs self-organized based on their closeness in space. Thus, although spatial view cells represent "where" for episodic memory and navigation, they are formed by ventral visual stream feature inputs in the parahippocampal gyrus in what is the parahippocampal scene area. A second "where" driver of spatial view cells are parietal inputs, which it is proposed provide the idiothetic update for spatial view cells, used for memory recall and navigation when the spatial view details are obscured. Inferior temporal object "what" inputs and orbitofrontal cortex reward inputs connect to the human hippocampal system, and in macaques can be associated in the hippocampus with spatial view cell "where" representations to implement episodic memory. Hippocampal spatial view cells also provide a basis for navigation to a series of viewed landmarks, with the orbitofrontal cortex reward inputs to the hippocampus providing the goals for navigation, which can then be implemented by hippocampal connectivity in humans to parietal cortex regions involved in visuomotor actions in space. The presence of foveate vision and the highly developed temporal lobe for object and scene processing in primates including humans provide a basis for hippocampal spatial view cells to be key to understanding episodic memory in the primate and human hippocampus, and the roles of this system in primate including human navigation.

Hippocampal spatio-predictive cognitive maps adaptively guide reward generalization

The brain forms cognitive maps of relational knowledge-an organizing principle thought to underlie our ability to generalize and make inferences. However, how can a relevant map be selected in situations where a stimulus is embedded in multiple relational structures? Here, we find that both spatial and predictive cognitive maps influence generalization in a choice task, where spatial location determines reward magnitude. Mirroring behavior, the hippocampus not only builds a map of spatial relationships but also encodes the experienced transition structure. As the task progresses, participants' choices become more influenced by spatial relationships, reflected in a strengthening of the spatial map and a weakening of the predictive map. This change is driven by orbitofrontal cortex, which represents the degree to which an outcome is consistent with the spatial rather than the predictive map and updates hippocampal representations accordingly. Taken together, this demonstrates how hippocampal cognitive maps are used and updated flexibly for inference.

Successor-like representation guides the prediction of future events in human visual cortex and hippocampus

Human agents build models of their environment, which enable them to anticipate and plan upcoming events. However, little is known about the properties of such predictive models. Recently, it has been proposed that hippocampal representations take the form of a predictive map-like structure, the so-called successor representation (SR). Here, we used human functional magnetic resonance imaging to probe whether activity in the early visual cortex (V1) and hippocampus adhere to the postulated properties of the SR after visual sequence learning. Participants were exposed to an arbitrary spatiotemporal sequence consisting of four items (A-B-C-D). We found that after repeated exposure to the sequence, merely presenting single sequence items (e.g., - B - -) resulted in V1 activation at the successor locations of the full sequence (e.g., C-D), but not at the predecessor locations (e.g., A). This highlights that visual representations are skewed toward future states, in line with the SR. Similar results were also found in the hippocampus. Moreover, the hippocampus developed a coactivation profile that showed sensitivity to the temporal distance in sequence space, with fading representations for sequence events in the more distant past and future. V1, in contrast, showed a coactivation profile that was only sensitive to spatial distance in stimulus space. Taken together, these results provide empirical evidence for the proposition that both visual and hippocampal cortex represent a predictive map of the visual world akin to the SR.

From cognitive maps to spatial schemas

A schema refers to a structured body of prior knowledge that captures common patterns across related experiences. Schemas have been studied separately in the realms of episodic memory and spatial navigation across different species and have been grounded in theories of memory consolidation, but there has been little attempt to integrate our understanding across domains, particularly in humans. We propose that experiences during navigation with many similarly structured environments give rise to the formation of spatial schemas (for example, the expected layout of modern cities) that share properties with but are distinct from cognitive maps (for example, the memory of a modern city) and event schemas (such as expected events in a modern city) at both cognitive and neural levels. We describe earlier theoretical frameworks and empirical findings relevant to spatial schemas, along with more targeted investigations of spatial schemas in human and non-human animals. Consideration of architecture and urban analytics, including the influence of scale and regionalization, on different properties of spatial schemas may provide a powerful approach to advance our understanding of spatial schemas.

Entorhinal grid-like codes and time-locked network dynamics track others navigating through space

Navigating through crowded, dynamically changing environments requires the ability to keep track of other individuals. Grid cells in the entorhinal cortex are a central component of self-related navigation but whether they also track others' movement is unclear. Here, we propose that entorhinal grid-like codes make an essential contribution to socio-spatial navigation. Sixty human participants underwent functional magnetic resonance imaging (fMRI) while observing and re-tracing different paths of a demonstrator that navigated a virtual reality environment. Results revealed that grid-like codes in the entorhinal cortex tracked the other individual navigating through space. The activity of grid-like codes was time-locked to increases in co-activation and entorhinal-cortical connectivity that included the striatum, the hippocampus, parahippocampal and right posterior parietal cortices. Surprisingly, the grid-related effects during observation were stronger the worse participants performed when subsequently re-tracing the demonstrator's paths. Our findings suggests that network dynamics time-locked to entorhinal grid-cell-related activity might serve to distribute information about the location of others throughout the brain.

Excitatory-inhibitory recurrent dynamics produce robust visual grids and stable attractors

Spatially modulated grid cells have been recently found in the rat secondary visual cortex (V2) during active navigation. However, the computational mechanism and functional significance of V2 grid cells remain unknown. To address the knowledge gap, we train a biologically inspired excitatory-inhibitory recurrent neural network to perform a two-dimensional spatial navigation task with multisensory input. We find grid-like responses in both excitatory and inhibitory RNN units, which are robust with respect to spatial cues, dimensionality of visual input, and activation function. Population responses reveal a low-dimensional, torus-like manifold and attractor. We find a link between functional grid clusters with similar receptive fields and structured excitatory-to-excitatory connections. Additionally, multistable torus-like attractors emerged with increasing sparsity in inter- and intra-subnetwork connectivity. Finally, irregular grid patterns are found in recurrent neural network (RNN) units during a visual sequence recognition task. Together, our results suggest common computational mechanisms of V2 grid cells for spatial and non-spatial tasks.

Integrated world modeling theory expanded: Implications for the future of consciousness

Integrated world modeling theory (IWMT) is a synthetic theory of consciousness that uses the free energy principle and active inference (FEP-AI) framework to combine insights from integrated information theory (IIT) and global neuronal workspace theory (GNWT). Here, I first review philosophical principles and neural systems contributing to IWMT's integrative perspective. I then go on to describe predictive processing models of brains and their connections to machine learning architectures, with particular emphasis on autoencoders (perceptual and active inference), turbo-codes (establishment of shared latent spaces for multi-modal integration and inferential synergy), and graph neural networks (spatial and somatic modeling and control). Future directions for IIT and GNWT are considered by exploring ways in which modules and workspaces may be evaluated as both complexes of integrated information and arenas for iterated Bayesian model selection. Based on these considerations, I suggest novel ways in which integrated information might be estimated using concepts from probabilistic graphical models, flow networks, and game theory. Mechanistic and computational principles are also considered with respect to the ongoing debate between IIT and GNWT regarding the physical substrates of different kinds of conscious and unconscious phenomena. I further explore how these ideas might relate to the "Bayesian blur problem," or how it is that a seemingly discrete experience can be generated from probabilistic modeling, with some consideration of analogies from quantum mechanics as potentially revealing different varieties of inferential dynamics. I go on to describe potential means of addressing critiques of causal structure theories based on network unfolding, and the seeming absurdity of conscious expander graphs (without cybernetic symbol grounding). Finally, I discuss future directions for work centered on attentional selection and the evolutionary origins of consciousness as facilitated "unlimited associative learning." While not quite solving the Hard problem, this article expands on IWMT as a unifying model of consciousness and the potential future evolution of minds.

Connecting to the long axis

New study reveals how various regions of the human cortex connect to the hippocampus along its longer anterior-posterior axis, shedding light on the way this structure is functionally organized.

The hearing hippocampus

The hippocampus has a well-established role in spatial and episodic memory but a broader function has been proposed including aspects of perception and relational processing. Neural bases of sound analysis have been described in the pathway to auditory cortex, but wider networks supporting auditory cognition are still being established. We review what is known about the role of the hippocampus in processing auditory information, and how the hippocampus itself is shaped by sound. In examining imaging, recording, and lesion studies in species from rodents to humans, we uncover a hierarchy of hippocampal responses to sound including during passive exposure, active listening, and the learning of associations between sounds and other stimuli. We describe how the hippocampus' connectivity and computational architecture allow it to track and manipulate auditory information - whether in the form of speech, music, or environmental, emotional, or phantom sounds. Functional and structural correlates of auditory experience are also identified. The extent of auditory-hippocampal interactions is consistent with the view that the hippocampus makes broad contributions to perception and cognition, beyond spatial and episodic memory. More deeply understanding these interactions may unlock applications including entraining hippocampal rhythms to support cognition, and intervening in links between hearing loss and dementia.

Rapid encoding of task regularities in the human hippocampus guides sensorimotor timing

The brain encodes the statistical regularities of the environment in a task-specific yet flexible and generalizable format. Here, we seek to understand this process by bridging two parallel lines of research, one centered on sensorimotor timing, and the other on cognitive mapping in the hippocampal system. By combining functional magnetic resonance imaging (fMRI) with a fast-paced time-to-contact (TTC) estimation task, we found that the hippocampus signaled behavioral feedback received in each trial as well as performance improvements across trials along with reward-processing regions. Critically, it signaled performance improvements independent from the tested intervals, and its activity accounted for the trial-wise regression-to-the-mean biases in TTC estimation. This is in line with the idea that the hippocampus supports the rapid encoding of temporal context even on short time scales in a behavior-dependent manner. Our results emphasize the central role of the hippocampus in statistical learning and position it at the core of a brain-wide network updating sensorimotor representations in real time for flexible behavior.

How to build a cognitive map

Learning and interpreting the structure of the environment is an innate feature of biological systems, and is integral to guiding flexible behaviors for evolutionary viability. The concept of a cognitive map has emerged as one of the leading metaphors for these capacities, and unraveling the learning and neural representation of such a map has become a central focus of neuroscience. In recent years, many models have been developed to explain cellular responses in the hippocampus and other brain areas. Because it can be difficult to see how these models differ, how they relate and what each model can contribute, this Review aims to organize these models into a clear ontology. This ontology reveals parallels between existing empirical results, and implies new approaches to understand hippocampal-cortical interactions and beyond.

Human Variation in Error-Based and Reinforcement Motor Learning Is Associated With Entorhinal Volume

Error-based and reward-based processes are critical for motor learning and are thought to be mediated via distinct neural pathways. However, recent behavioral work in humans suggests that both learning processes can be bolstered by the use of cognitive strategies, which may mediate individual differences in motor learning ability. It has been speculated that medial temporal lobe regions, which have been shown to support motor sequence learning, also support the use of cognitive strategies in error-based and reinforcement motor learning. However, direct evidence in support of this idea remains sparse. Here we first show that better overall learning during error-based visuomotor adaptation is associated with better overall learning during the reward-based shaping of reaching movements. Given the cognitive contribution to learning in both of these tasks, these results support the notion that strategic processes, associated with better performance, drive intersubject variation in both error-based and reinforcement motor learning. Furthermore, we show that entorhinal cortex volume is larger in better learning individuals-characterized across both motor learning tasks-compared with their poorer learning counterparts. These results suggest that individual differences in learning performance during error and reinforcement learning are related to neuroanatomical differences in entorhinal cortex.

Are Grid-Like Representations a Component of All Perception and Cognition?

Grid cells or grid-like responses have been reported in the rodent, bat and human brains during various spatial and non-spatial tasks. However, the functions of grid-like representations beyond the classical hippocampal formation remain elusive. Based on accumulating evidence from recent rodent recordings and human fMRI data, we make speculative accounts regarding the mechanisms and functional significance of the sensory cortical grid cells and further make theory-driven predictions. We argue and reason the rationale why grid responses may be universal in the brain for a wide range of perceptual and cognitive tasks that involve locomotion and mental navigation. Computational modeling may provide an alternative and complementary means to investigate the grid code or grid-like map. We hope that the new discussion will lead to experimentally testable hypotheses and drive future experimental data collection.

Selective neural coding of object, feature, and geometry spatial cues in humans

Orienting in space requires the processing of visual spatial cues. The dominant hypothesis about the brain structures mediating the coding of spatial cues stipulates the existence of a hippocampal-dependent system for the representation of geometry and a striatal-dependent system for the representation of landmarks. However, this dual-system hypothesis is based on paradigms that presented spatial cues conveying either conflicting or ambiguous spatial information and that used the term landmark to refer to both discrete three-dimensional objects and wall features. Here, we test the hypothesis of complex activation patterns in the hippocampus and the striatum during visual coding. We also postulate that object-based and feature-based navigation are not equivalent instances of landmark-based navigation. We examined how the neural networks associated with geometry-, object-, and feature-based spatial navigation compared with a control condition in a two-choice behavioral paradigm using fMRI. We showed that the hippocampus was involved in all three types of cue-based navigation, whereas the striatum was more strongly recruited in the presence of geometric cues than object or feature cues. We also found that unique, specific neural signatures were associated with each spatial cue. Object-based navigation elicited a widespread pattern of activity in temporal and occipital regions relative to feature-based navigation. These findings extend the current view of a dual, juxtaposed hippocampal-striatal system for visual spatial coding in humans. They also provide novel insights into the neural networks mediating object versus feature spatial coding, suggesting a need to distinguish these two types of landmarks in the context of human navigation.

Sense of self impacts spatial navigation and hexadirectional coding in human entorhinal cortex

Grid cells in entorhinal cortex (EC) encode an individual's location in space and rely on environmental cues and self-motion cues derived from the individual's body. Body-derived signals are also primary signals for the sense of self and based on integrated sensorimotor signals (proprioceptive, tactile, visual, motor) that have been shown to enhance self-centered processing. However, it is currently unknown whether such sensorimotor signals that modulate self-centered processing impact grid cells and spatial navigation. Integrating the online manipulation of bodily signals, to modulate self-centered processing, with a spatial navigation task and an fMRI measure to detect grid cell-like representation (GCLR) in humans, we report improved performance in spatial navigation and decreased GCLR in EC. This decrease in entorhinal GCLR was associated with an increase in retrosplenial cortex activity, which was correlated with participants' navigation performance. These data link self-centered processes during spatial navigation to entorhinal and retrosplenial activity and highlight the role of different bodily factors at play when navigating in VR.

Microcircuits for spatial coding in the medial entorhinal cortex

The hippocampal formation is critically involved in learning and memory and contains a large proportion of neurons encoding aspects of the organism's spatial surroundings. In the medial entorhinal cortex (MEC), this includes grid cells with their distinctive hexagonal firing fields as well as a host of other functionally defined cell types including head direction cells, speed cells, border cells, and object-vector cells. Such spatial coding emerges from the processing of external inputs by local microcircuits. However, it remains unclear exactly how local microcircuits and their dynamics within the MEC contribute to spatial discharge patterns. In this review we focus on recent investigations of intrinsic MEC connectivity, which have started to describe and quantify both excitatory and inhibitory wiring in the superficial layers of the MEC. Although the picture is far from complete, it appears that these layers contain robust recurrent connectivity that could sustain the attractor dynamics posited to underlie grid pattern formation. These findings pave the way to a deeper understanding of the mechanisms underlying spatial navigation and memory.

Neurons as will and representation

Memory recollections and voluntary actions are often perceived as spontaneously generated irrespective of external stimuli. Although products of our neurons, they are only rarely accessible in humans at the neuronal level. Here I review insights gleaned from unique neurosurgical opportunities to record and stimulate single-neuron activity in people who can declare their thoughts, memories and wishes. I discuss evidence that the subjective experience of human recollection and that of voluntary action arise from the activity of two internal neuronal generators, the former from medial temporal lobe reactivation and the latter from frontoparietal preactivation. I characterize properties of these generators and their interaction, enabling flexible recruitment of memory-based choices for action as well as recruitment of action-based plans for the representation of conceptual knowledge in memories. Both internal generators operate on surprisingly explicit but different neuronal codes, which appear to arise with distinct single-neuron activity, often observed before participants' reports of conscious awareness. I discuss prediction of behaviour based on these codes, and the potential for their modulation. The prospects of editing human memories and volitions by enhancement, inception or deletion of specific, selected content raise therapeutic possibilities and ethical concerns.

Spatialization of Time in the Entorhinal-Hippocampal System

The functional role of the entorhinal-hippocampal system has been a long withstanding mystery. One key theory that has become most popular is that the entorhinal-hippocampal system represents space to facilitate navigation in one's surroundings. In this Perspective article, I introduce a novel idea that undermines the inherent uniqueness of spatial information in favor of time driving entorhinal-hippocampal activity. Specifically, by spatializing events that occur in succession (i.e., across time), the entorhinal-hippocampal system is critical for all types of cognitive representations. I back up this argument with empirical evidence that hints at a role for the entorhinal-hippocampal system in non-spatial representation, and computational models of the logarithmic compression of time in the brain.

A neural surveyor to map touch on the body

Perhaps the most recognizable “sensory map” in neuroscience is the somatosensory homunculus. Although the homunculus suggests a direct link between cortical territory and body part, the relationship is actually ambiguous without a decoder that knows this mapping. How the somatosensory system derives a spatial code from an activation in the homunculus is a longstanding mystery we aimed to solve. We propose that touch location is disambiguated using multilateration, a computation used by surveying and global positioning systems to localize objects. We develop a Bayesian formulation of multilateration, which we implement in a neural network to identify its computational signature. We then detect this signature in psychophysical experiments. Our results suggest that multilateration provides the homunculus-to-body mapping necessary for localizing touch.

Perhaps the most recognizable sensory map in all of neuroscience is the somatosensory homunculus. Although it seems straightforward, this simple representation belies the complex link between an activation in a somatotopic map and the associated touch location on the body. Any isolated activation is spatially ambiguous without a neural decoder that can read its position within the entire map, but how this is computed by neural networks is unknown. We propose that the somatosensory system implements multilateration, a common computation used by surveying and global positioning systems to localize objects. Specifically, to decode touch location on the body, multilateration estimates the relative distance between the afferent input and the boundaries of a body part (e.g., the joints of a limb). We show that a simple feedforward neural network, which captures several fundamental receptive field properties of cortical somatosensory neurons, can implement a Bayes-optimal multilateral computation. Simulations demonstrated that this decoder produced a pattern of localization variability between two boundaries that was unique to multilateration. Finally, we identify this computational signature of multilateration in actual psychophysical experiments, suggesting that it is a candidate computational mechanism underlying tactile localization.

Working memory and active sampling of the environment: Medial temporal contributions

Working memory (WM) refers to the ability to maintain and actively process information-either derived from perception or long-term memory (LTM)-for intelligent thought and action. This chapter focuses on the contributions of the temporal lobe, particularly medial temporal lobe (MTL) to WM. First, neuropsychological evidence for the involvement of MTL in WM maintenance is reviewed, arguing for a crucial role in the case of retaining complex relational bindings between memorized features. Next, MTL contributions at the level of neural mechanisms are covered-with a focus on WM encoding and maintenance, including interactions with ventral temporal cortex. Among WM use processes, we focus on active sampling of environmental information, a key input source to capacity-limited WM. MTL contributions to the bidirectional relationship between active sampling and memory are highlighted-WM control of active sampling and sampling as a way of selecting input to WM. Memory-based sampling studies relying on scene and object inspection, visual-based exploration behavior (e.g., vicarious behavior), and memory-guided visual search are reviewed. The conclusion is that MTL serves an important function in the selection of information from perception and transfer from LTM to capacity-limited WM.

Neurodegenerative damage reduces firing coherence in a continuous attractor model of grid cells

Grid cells in the dorsolateral band of the medial entorhinal cortex (dMEC) display strikingly regular periodic firing patterns on a lattice of positions in two-dimensional (2D) space. This helps animals to encode relative spatial location without reference to external cues. The dMEC is damaged in the early stages of Alzheimer's disease, which affects navigation ability of a disease victim, reducing the synaptic density of neurons in the network. Within an established two-dimensional continuous attractor neural network model of grid cell activity, we introduce neural sheet damage parametrized by radius and by the strength of the synaptic output for neurons in the damaged region. The mean proportionality of the grid field flow rate in the dMEC to the velocity of the model animal is maintained, but there is a broadened distribution of flow rates in the damaged case. This flow-rate-to-velocity proportionality is essential to establish coherent grid firing fields for individual grid cells for a roaming animal. When we examine the coherence of the grid cell firing field by studying Bragg peaks of the Fourier transformed lattice firing field intensity in both damaged and undamaged regions, we find that, for a wide range of damage radius and reduced synaptic strength, for undamaged model grid cells there is an incoherent firing field structure with only a single central peak. In the radius-damage plane this is adjacent to narrow bands of striped lattices (two additional Bragg peaks), which abut an orthorhombic pattern (four additional Bragg peaks), that abuts the undamaged hexagonal region (six additional Bragg peaks). Within the damaged region, grid cells show no Bragg peaks outside the central one, which shows reduced intensity with increasing damage, and outside the damaged region the central Bragg peak strength is largely unaffected. There is a reentrant region of normal grid firing fields for very large damage area. We anticipate that the modified grid cell behavior can be observed in noninvasive functional magnetic resonance imaging (fMRI) of the dMEC.

Magnetic resonance-based eye tracking using deep neural networks

Viewing behavior provides a window into many central aspects of human cognition and health, and it is an important variable of interest or confound in many functional magnetic resonance imaging (fMRI) studies. To make eye tracking freely and widely available for MRI research, we developed DeepMReye, a convolutional neural network (CNN) that decodes gaze position from the magnetic resonance signal of the eyeballs. It performs cameraless eye tracking at subimaging temporal resolution in held-out participants with little training data and across a broad range of scanning protocols. Critically, it works even in existing datasets and when the eyes are closed. Decoded eye movements explain network-wide brain activity also in regions not associated with oculomotor function. This work emphasizes the importance of eye tracking for the interpretation of fMRI results and provides an open source software solution that is widely applicable in research and clinical settings.

Theta oscillations coordinate grid-like representations between ventromedial prefrontal and entorhinal cortex

Grid cells and theta oscillations are fundamental constituents of the brain’s navigation system and have been described in the entorhinal cortex (EC). Recent fMRI studies reveal that the ventromedial prefrontal cortex (vmPFC) contains grid-like representations. However, the neural mechanisms underlying human vmPFC grid-like representations and their interactions with EC grid activity have remained unknown. We conducted intracranial electroencephalography (iEEG) recordings from epilepsy patients during a virtual spatial navigation task. Oscillatory theta power in the vmPFC exhibited a sixfold rotational symmetry that was coordinated with grid-like representations in the EC. We found that synchronous theta oscillations occurred between these regions that predicted navigational performance. Analysis of information transfer revealed a unidirectional signal from vmPFC to EC during memory retrieval. Together, this study provides insights into the previously unknown neural signature and functional role of grid-like representations outside the EC and their synchronization with the entorhinal grid during human spatial navigation.

Vision for action: thalamic and cortical inputs to the macaque superior parietal lobule

The dorsal visual stream, the cortical circuit that in the primate brain is mainly dedicated to the visual control of actions, is split into two routes, a lateral and a medial one, both involved in coding different aspects of sensorimotor control of actions. The lateral route, named "lateral grasping network", is mainly involved in the control of the distal part of prehension, namely grasping and manipulation. The medial route, named "reach-to-grasp network", is involved in the control of the full deployment of prehension act, from the direction of arm movement to the shaping of the hand according to the object to be grasped. In macaque monkeys, the reach-to-grasp network (the target of this review) includes areas of the superior parietal lobule (SPL) that hosts visual and somatosensory neurons well suited to control goal-directed limb movements toward stationary as well as moving objects. After a brief summary of the neuronal functional properties of these areas, we will analyze their cortical and thalamic inputs thanks to retrograde neuronal tracers separately injected into the SPL areas V6, V6A, PEc, and PE. These areas receive visual and somatosensory information distributed in a caudorostral, visuosomatic trend, and some of them are directly connected with the dorsal premotor cortex. This review is particularly focused on the origin and type of visual information reaching the SPL, and on the functional role this information can play in guiding limb interaction with objects in structured and dynamic environments.

Resolving visual motion through perceptual gaps

Perceptual gaps can be caused by objects in the foreground temporarily occluding objects in the background or by eyeblinks, which briefly but frequently interrupt visual information. Resolving visual motion across perceptual gaps is particularly challenging, as object position changes during the gap. We examine how visual motion is maintained and updated through externally driven (occlusion) and internally driven (eyeblinks) perceptual gaps. Focusing on both phenomenology and potential mechanisms such as suppression, extrapolation, and integration, we present a framework for how perceptual gaps are resolved over space and time. We finish by highlighting critical questions and directions for future work.

Inferences on a multidimensional social hierarchy use a grid-like code

Generalizing experiences to guide decision-making in novel situations is a hallmark of flexible behavior. Cognitive maps of an environment or task can theoretically afford such flexibility, but direct evidence has proven elusive. In this study, we found that discretely sampled abstract relationships between entities in an unseen two-dimensional social hierarchy are reconstructed into a unitary two-dimensional cognitive map in the hippocampus and entorhinal cortex. We further show that humans use a grid-like code in entorhinal cortex and medial prefrontal cortex for inferred direct trajectories between entities in the reconstructed abstract space during discrete decisions. These grid-like representations in the entorhinal cortex are associated with decision value computations in the medial prefrontal cortex and temporoparietal junction. Collectively, these findings show that grid-like representations are used by the human brain to infer novel solutions, even in abstract and discrete problems, and suggest a general mechanism underpinning flexible decision-making and generalization.

Subjective value, not a gridlike code, describes neural activity in ventromedial prefrontal cortex during value-based decision-making

Across many studies, ventromedial prefrontal cortex (vmPFC) activity has been found to correlate with subjective value during value-based decision-making. Recently, however, vmPFC has also been shown to reflect a hexagonal gridlike code during navigation through physical and conceptual space, and such gridlike codes have been proposed to enable value-based choices between novel options. Here, we first show that, in theory, a hexagonal gridlike code can in some cases mimic vmPFC activity previously attributed to subjective value, raising the possibility that the subjective value correlates previously observed in vmPFC may have actually been a misconstrued gridlike signal. We then compare the two accounts empirically, using fMRI data from a large number of subjects performing an intertemporal choice task. We find clear and unambiguous evidence that subjective value is a better description of vmPFC activity in this task than a hexagonal gridlike code. In fact, we find no significant evidence at all for a hexagonal gridlike code in vmPFC activity during intertemporal choice. This result limits the generality of gridlike modulation as description of vmPFC activity. We suggest that vmPFC may flexibly switch representational schemes so as to encode the most relevant information for the current task.