Nonlocal spatiotemporal representation in the hippocampus of freely flying bats

Transformers and cortical waves: encoders for pulling in context across time

The capabilities of transformer networks such as ChatGPT and other large language models (LLMs) have captured the world's attention. The crucial computational mechanism underlying their performance relies on transforming a complete input sequence - for example, all the words in a sentence - into a long 'encoding vector' that allows transformers to learn long-range temporal dependencies in naturalistic sequences. Specifically, 'self-attention' applied to this encoding vector enhances temporal context in transformers by computing associations between pairs of words in the input sequence. We suggest that waves of neural activity traveling across single cortical areas, or multiple regions on the whole-brain scale, could implement a similar encoding principle. By encapsulating recent input history into a single spatial pattern at each moment in time, cortical waves may enable a temporal context to be extracted from sequences of sensory inputs, the same computational principle as that used in transformers.

Integration and competition between space and time in the hippocampus

Episodic memory is organized in both spatial and temporal contexts. The hippocampus is crucial for episodic memory and has been demonstrated to encode spatial and temporal information. However, how the representations of space and time interact in the hippocampal memory system is still unclear. Here, we recorded the activity of hippocampal CA1 neurons in mice in a variety of one-dimensional navigation tasks while systematically varying the speed of the animals. For all tasks, we found neurons simultaneously represented space and elapsed time. There was a negative correlation between the preferred space and lap duration, e.g., the preferred spatial position shifted more toward the origin when the lap duration became longer. A similar relationship between the preferred time and traveled distance was also observed. The results strongly suggest a competitive and integrated representation of space-time by single hippocampal neurons, which may provide the neural basis for spatiotemporal contexts.

Neural representation of human experimenters in the bat hippocampus

Here we conducted wireless electrophysiological recording of hippocampal neurons from Egyptian fruit bats in the presence of human experimenters. In flying bats, many neurons modulated their activity depending on the identity of the human at the landing target. In stationary bats, many neurons carried significant spatial information about the position and identity of humans traversing the environment. Our results reveal that hippocampal activity is robustly modulated by the presence, movement and identity of human experimenters.

Acute Neuropixels Recordings in the Marmoset Monkey

High-density linear probes, such as Neuropixels, provide an unprecedented opportunity to understand how neural populations within specific laminar compartments contribute to behavior. Marmoset monkeys, unlike macaque monkeys, have a lissencephalic (smooth) cortex that enables recording perpendicular to the cortical surface, thus making them an ideal animal model for studying laminar computations. Here we present a method for acute Neuropixels recordings in the common marmoset (Callithrix jacchus). The approach replaces the native dura with an artificial silicon-based dura that grants visual access to the cortical surface, which is helpful in avoiding blood vessels, ensures perpendicular penetrations, and could be used in conjunction with optical imaging or optogenetic techniques. The chamber housing the artificial dura is simple to maintain with minimal risk of infection and could be combined with semichronic microdrives and wireless recording hardware. This technique enables repeated acute penetrations over a period of several months. With occasional removal of tissue growth on the pial surface, recordings can be performed for a year or more. The approach is fully compatible with Neuropixels probes, enabling the recording of hundreds of single neurons distributed throughout the cortical column.

Phase-Amplitude Coupling between Theta Rhythm and High-Frequency Oscillations in the Hippocampus of Pigeons during Navigation

Navigation is a complex task in which the hippocampus (Hp), which plays an important role, may be involved in interactions between different frequency bands. However, little is known whether this cross-frequency interaction exists in the Hp of birds during navigation. Therefore, we examined the electrophysiological characteristics of hippocampal cross-frequency interactions of domestic pigeons (Columba livia domestica) during navigation. Two goal-directed navigation tasks with different locomotor modes were designed, and the local field potentials (LFPs) were recorded for analysis. We found that the amplitudes of high-frequency oscillations in Hp were dynamically modulated by the phase of co-occurring theta-band oscillations both during ground-based maze and outdoor flight navigation. The high-frequency amplitude sub-frequency bands modulated by the hippocampal theta phase were different at different tasks, and this process was independent of the navigation path and goal. These results suggest that phase-amplitude coupling (PAC) in the avian Hp may be more associated with the ongoing cognitive demands of navigational processes. Our findings contribute to the understanding of potential mechanisms of hippocampal PAC on multi-frequency informational interactions in avian navigation and provide valuable insights into cross-species evolution.

Distinguishing examples while building concepts in hippocampal and artificial networks

The hippocampal subfield CA3 is thought to function as an auto-associative network that stores experiences as memories. Information from these experiences arrives directly from the entorhinal cortex as well as indirectly through the dentate gyrus, which performs sparsification and decorrelation. The computational purpose for these dual input pathways has not been firmly established. We model CA3 as a Hopfield-like network that stores both dense, correlated encodings and sparse, decorrelated encodings. As more memories are stored, the former merge along shared features while the latter remain distinct. We verify our model's prediction in rat CA3 place cells, which exhibit more distinct tuning during theta phases with sparser activity. Finally, we find that neural networks trained in multitask learning benefit from a loss term that promotes both correlated and decorrelated representations. Thus, the complementary encodings we have found in CA3 can provide broad computational advantages for solving complex tasks.

Acute Neuropixels recordings in the marmoset monkey

High-density linear probes, like Neuropixels, provide an unprecedented opportunity to understand how neural populations within specific laminar compartments contribute to behavior. Marmoset monkeys, unlike macaque monkeys, have a lissencephalic (smooth) cortex that enables recording perpendicular to the cortical surface, thus making them an ideal animal model for studying laminar computations. Here we present a method for acute Neuropixels recordings in the common marmoset (Callithrix jacchus). The approach replaces the native dura with an artificial silicon-based dura that grants visual access to the cortical surface, which is helpful in avoiding blood vessels, ensures perpendicular penetrations, and could be used in conjunction with optical imaging or optogenetic techniques. The chamber housing the artificial dura is simple to maintain with minimal risk of infection and could be combined with semi-chronic microdrives and wireless recording hardware. This technique enables repeated acute penetrations over a period of several months. With occasional removal of tissue growth on the pial surface, recordings can be performed for a year or more. The approach is fully compatible with Neuropixels probes, enabling the recording of hundreds of single neurons distributed throughout the cortical column.

Mixed representations of sound and action in the auditory midbrain

Linking sensory input and its consequences is a fundamental brain operation. Accordingly, neural activity of neo-cortical and limbic systems often reflects dynamic combinations of sensory and behaviorally relevant variables, and these "mixed representations" are suggested to be important for perception, learning, and plasticity. However, the extent to which such integrative computations might occur in brain regions upstream of the forebrain is less clear. Here, we conduct cellular-resolution 2-photon Ca2+ imaging in the superficial "shell" layers of the inferior colliculus (IC), as head-fixed mice of either sex perform a reward-based psychometric auditory task. We find that the activity of individual shell IC neurons jointly reflects auditory cues and mice's actions, such that trajectories of neural population activity diverge depending on mice's behavioral choice. Consequently, simple classifier models trained on shell IC neuron activity can predict trial-by-trial outcomes, even when training data are restricted to neural activity occurring prior to mice's instrumental actions. Thus in behaving animals, auditory midbrain neurons transmit a population code that reflects a joint representation of sound and action.

Hippocampal representation during collective spatial behaviour in bats

Social animals live and move through spaces shaped by the presence, motion and sensory cues of multiple other individuals1-6. Neural activity in the hippocampus is known to reflect spatial behaviour7-9 yet its study is lacking in such dynamic group settings, which are ubiquitous in natural environments. Here we studied hippocampal activity in groups of bats engaged in collective spatial behaviour. We find that, under spontaneous conditions, a robust spatial structure emerges at the group level whereby behaviour is anchored to specific locations, movement patterns and individual social preferences. Using wireless electrophysiological recordings from both stationary and flying bats, we find that many hippocampal neurons are tuned to key features of group dynamics. These include the presence or absence of a conspecific, but not typically of an object, at landing sites, shared spatial locations, individual identities and sensory signals that are broadcasted in the group setting. Finally, using wireless calcium imaging, we find that social responses are anatomically distributed and robustly represented at the population level. Combined, our findings reveal that hippocampal activity contains a rich representation of naturally emerging spatial behaviours in animal groups that could in turn support the complex feat of collective behaviour.

Simultaneous representation of multiple time horizons by entorhinal grid cells and CA1 place cells

Grid cells and place cells represent the spatiotemporal continuum of an animal's past, present, and future locations. However, their spatiotemporal relationship is unclear. Here, we co-record grid and place cells in freely foraging rats. We show that average time shifts in grid cells tend to be prospective and are proportional to their spatial scale, providing a nearly instantaneous readout of a spectrum of progressively increasing time horizons ranging hundreds of milliseconds. Average time shifts of place cells are generally larger compared to grid cells and also increase with place field sizes. Moreover, time horizons display nonlinear modulation by the animal's trajectories in relation to the local boundaries and locomotion cues. Finally, long and short time horizons occur at different parts of the theta cycle, which may facilitate their readout. Together, these findings suggest that population activity of grid and place cells may represent local trajectories essential for goal-directed navigation and planning.

An entorhinal-like region in food-caching birds

The mammalian entorhinal cortex routes inputs from diverse sources into the hippocampus. This information is mixed and expressed in the activity of many specialized entorhinal cell types, which are considered indispensable for hippocampal function. However, functionally similar hippocampi exist even in non-mammals that lack an obvious entorhinal cortex or, generally, any layered cortex. To address this dilemma, we mapped extrinsic hippocampal connections in chickadees, whose hippocampi are used for remembering numerous food caches. We found a well-delineated structure in these birds that is topologically similar to the entorhinal cortex and interfaces between the hippocampus and other pallial regions. Recordings of this structure revealed entorhinal-like activity, including border and multi-field grid-like cells. These cells were localized to the subregion predicted by anatomical mapping to match the dorsomedial entorhinal cortex. Our findings uncover an anatomical and physiological equivalence of vastly different brains, suggesting a fundamental nature of entorhinal-like computations for hippocampal function.

Brain-inspired multimodal hybrid neural network for robot place recognition

Place recognition is an essential spatial intelligence capability for robots to understand and navigate the world. However, recognizing places in natural environments remains a challenging task for robots because of resource limitations and changing environments. In contrast, humans and animals can robustly and efficiently recognize hundreds of thousands of places in different conditions. Here, we report a brain-inspired general place recognition system, dubbed NeuroGPR, that enables robots to recognize places by mimicking the neural mechanism of multimodal sensing, encoding, and computing through a continuum of space and time. Our system consists of a multimodal hybrid neural network (MHNN) that encodes and integrates multimodal cues from both conventional and neuromorphic sensors. Specifically, to encode different sensory cues, we built various neural networks of spatial view cells, place cells, head direction cells, and time cells. To integrate these cues, we designed a multiscale liquid state machine that can process and fuse multimodal information effectively and asynchronously using diverse neuronal dynamics and bioinspired inhibitory circuits. We deployed the MHNN on Tianjic, a hybrid neuromorphic chip, and integrated it into a quadruped robot. Our results show that NeuroGPR achieves better performance compared with conventional and existing biologically inspired approaches, exhibiting robustness to diverse environmental uncertainty, including perceptual aliasing, motion blur, light, or weather changes. Running NeuroGPR as an overall multi-neural network workload on Tianjic showcases its advantages with 10.5 times lower latency and 43.6% lower power consumption than the commonly used mobile robot processor Jetson Xavier NX.

Learning to predict future locations with internally generated theta sequences

Representing past, present and future locations is key for spatial navigation. Indeed, within each cycle of the theta oscillation, the population of hippocampal place cells appears to represent trajectories starting behind the current position of the animal and sweeping ahead of it. In particular, we reported recently that the position represented by CA1 place cells at a given theta phase corresponds to the location where animals were or will be located at a fixed time interval into the past or future assuming the animal ran at its typical, not the current, speed through that part of the environment. This coding scheme leads to longer theta trajectories, larger place fields and shallower phase precession in areas where animals typically run faster. Here we present a mechanistic computational model that accounts for these experimental observations. The model consists of a continuous attractor network with short-term synaptic facilitation and depression that internally generates theta sequences that advance at a fixed pace. Spatial locations are then mapped onto the active units via modified Hebbian plasticity. As a result, neighboring units become associated with spatial locations further apart where animals run faster, reproducing our earlier experimental results. The model also accounts for the higher density of place fields generally observed where animals slow down, such as around rewards. Furthermore, our modeling results reveal that an artifact of the decoding analysis might be partly responsible for the observation that theta trajectories start behind the animal's current position. Overall, our results shed light on how the hippocampal code might arise from the interplay between behavior, sensory input and predefined network dynamics.

Miniscope-LFOV: A large-field-of-view, single-cell-resolution, miniature microscope for wired and wire-free imaging of neural dynamics in freely behaving animals

Imaging large-population, single-cell fluorescent dynamics in freely behaving animals larger than mice remains a key endeavor of neuroscience. We present a large-field-of-view open-source miniature microscope (MiniLFOV) designed for large-scale (3.6 mm × 2.7 mm), cellular resolution neural imaging in freely behaving rats. It has an electrically adjustable working distance of up to 3.5 mm ± 100 μm, incorporates an absolute head orientation sensor, and weighs only 13.9 g. The MiniLFOV is capable of both deep brain and cortical imaging and has been validated in freely behaving rats by simultaneously imaging >1000 GCaMP7s-expressing neurons in the hippocampal CA1 layer and in head-fixed mice by simultaneously imaging ~2000 neurons in the dorsal cortex through a cranial window. The MiniLFOV also supports optional wire-free operation using a novel, wire-free data acquisition expansion board. We expect that this new open-source implementation of the UCLA Miniscope platform will enable researchers to address novel hypotheses concerning brain function in freely behaving animals.

Contextual and pure time coding for self and other in the hippocampus

Navigation and episodic memory depend critically on representing temporal sequences. Hippocampal 'time cells' form temporal sequences, but it is unknown whether they represent context-dependent experience or time per se. Here we report on time cells in bat hippocampal area CA1, which, surprisingly, formed two distinct populations. One population of time cells generated different temporal sequences when the bat hung at different locations, thus conjunctively encoding spatial context and time-'contextual time cells'. A second population exhibited similar preferred times across different spatial contexts, thus purely encoding elapsed time. When examining neural responses after the landing moment of another bat, in a social imitation task, we found time cells that encoded temporal sequences aligned to the other's landing. We propose that these diverse time codes may support the perception of interval timing, episodic memory and temporal coordination between self and others.

Entorhinal grid-like codes and time-locked network dynamics track others navigating through space

Navigating through crowded, dynamically changing environments requires the ability to keep track of other individuals. Grid cells in the entorhinal cortex are a central component of self-related navigation but whether they also track others' movement is unclear. Here, we propose that entorhinal grid-like codes make an essential contribution to socio-spatial navigation. Sixty human participants underwent functional magnetic resonance imaging (fMRI) while observing and re-tracing different paths of a demonstrator that navigated a virtual reality environment. Results revealed that grid-like codes in the entorhinal cortex tracked the other individual navigating through space. The activity of grid-like codes was time-locked to increases in co-activation and entorhinal-cortical connectivity that included the striatum, the hippocampus, parahippocampal and right posterior parietal cortices. Surprisingly, the grid-related effects during observation were stronger the worse participants performed when subsequently re-tracing the demonstrator's paths. Our findings suggests that network dynamics time-locked to entorhinal grid-cell-related activity might serve to distribute information about the location of others throughout the brain.

An entorhinal-like region in food-caching birds

The mammalian entorhinal cortex routes inputs from diverse sources into the hippocampus. This information is mixed and expressed in the activity of many specialized entorhinal cell types, which are considered indispensable for hippocampal function. However, functionally similar hippocampi exist even in non-mammals that lack an obvious entorhinal cortex, or generally any layered cortex. To address this dilemma, we mapped extrinsic hippocampal connections in chickadees, whose hippocampi are used for remembering numerous food caches. We found a well-delineated structure in these birds that is topologically similar to the entorhinal cortex and interfaces between the hippocampus and other pallial regions. Recordings of this structure revealed entorhinal-like activity, including border and multi-field grid-like cells. These cells were localized to the subregion predicted by anatomical mapping to match the dorsomedial entorhinal cortex. Our findings uncover an anatomical and physiological equivalence of vastly different brains, suggesting a fundamental nature of entorhinal-like computations for hippocampal function.

Spatio-temporal categorization for first-person-view videos using a convolutional variational autoencoder and Gaussian processes

In this study, HcVGH, a method that learns spatio-temporal categories by segmenting first-person-view (FPV) videos captured by mobile robots, is proposed. Humans perceive continuous high-dimensional information by dividing and categorizing it into significant segments. This unsupervised segmentation capability is considered important for mobile robots to learn spatial knowledge. The proposed HcVGH combines a convolutional variational autoencoder (cVAE) with HVGH, a past method, which follows the hierarchical Dirichlet process-variational autoencoder-Gaussian process-hidden semi-Markov model comprising deep generative and statistical models. In the experiment, FPV videos of an agent were used in a simulated maze environment. FPV videos contain spatial information, and spatial knowledge can be learned by segmenting them. Using the FPV-video dataset, the segmentation performance of the proposed model was compared with previous models: HVGH and hierarchical recurrent state space model. The average segmentation F-measure achieved by HcVGH was 0.77; therefore, HcVGH outperformed the baseline methods. Furthermore, the experimental results showed that the parameters that represent the movability of the maze environment can be learned.

Neural Substrates of Homing Pigeon Spatial Navigation: Results From Electrophysiology Studies

Over many centuries, the homing pigeon has been selectively bred for returning home from a distant location. As a result of this strong selective pressure, homing pigeons have developed an excellent spatial navigation system. This system passes through the hippocampal formation (HF), which shares many striking similarities to the mammalian hippocampus; there are a host of shared neuropeptides, interconnections, and its role in the storage and manipulation of spatial maps. There are some notable differences as well: there are unique connectivity patterns and spatial encoding strategies. This review summarizes the comparisons between the avian and mammalian hippocampal systems, and the responses of single neurons in several general categories: (1) location and place cells responding in specific areas, (2) path and goal cells responding between goal locations, (3) context-dependent cells that respond before or during a task, and (4) pattern, grid, and boundary cells that increase firing at stable intervals. Head-direction cells, responding to a specific compass direction, are found in mammals and other birds but not to date in pigeons. By studying an animal that evolved under significant adaptive pressure to quickly develop a complex and efficient spatial memory system, we may better understand the comparative neurology of neurospatial systems, and plot new and potentially fruitful avenues of comparative research in the future.

A stable hippocampal code in freely flying bats

Neural activity in the hippocampus is known to reflect how animals move through an environment1,2. Although navigational behaviour may show considerable stability3-6, the tuning stability of individual hippocampal neurons remains unclear7-12. Here we used wireless calcium imaging to longitudinally monitor the activity of dorsal CA1 hippocampal neurons in freely flying bats performing highly reproducible flights in a familiar environment. We find that both the participation and the spatial selectivity of most neurons remain stable over days and weeks. We also find that apparent changes in tuning can be largely attributed to variations in the flight behaviour of the bats. Finally, we show that bats navigating in the same environment under different room lighting conditions (lights on versus lights off) exhibit substantial changes in flight behaviour that can give the illusion of neuronal instability. However, when similar flight paths are compared across conditions, the stability of the hippocampal code persists. Taken together, we show that the underlying hippocampal code is highly stable over days and across contexts if behaviour is taken into account.

Moving bar of light evokes vectorial spatial selectivity in the immobile rat hippocampus

Visual cortical neurons encode the position and motion direction of specific stimuli retrospectively, without any locomotion or task demand¹. The hippocampus, which is a part of the visual system, is hypothesized to require self-motion or a cognitive task to generate allocentric spatial selectivity that is scalar, abstract^2,3 and prospective^4-7. Here we measured rodent hippocampal selectivity to a moving bar of light in a body-fixed rat to bridge these seeming disparities. About 70% of dorsal CA1 neurons showed stable activity modulation as a function of the angular position of the bar, independent of behaviour and rewards. One-third of tuned cells also encoded the direction of revolution. In other experiments, neurons encoded the distance of the bar, with preference for approaching motion. Collectively, these demonstrate visually evoked vectorial selectivity (VEVS). Unlike place cells, VEVS was retrospective. Changes in the visual stimulus or its predictability did not cause remapping but only caused gradual changes. Most VEVS-tuned neurons behaved like place cells during spatial exploration and the two selectivities were correlated. Thus, VEVS could form the basic building block of hippocampal activity. When combined with self-motion, reward or multisensory stimuli⁸, it can generate the complexity of prospective representations including allocentric space⁹, time^10,11 and episodes¹².