Differential Emergence and Stability of Sensory and Temporal Representations in Context-Specific Hippocampal Sequences

The influence of emotion on temporal context models

Temporal context models (TCMs) have been influential in understanding episodic memory and its neural underpinnings. Recently, TCMs have been extended to explain emotional memory effects, one of the most clinically important findings in the field of memory research. This review covers recent advances in hypotheses for the neural representation of spatiotemporal context through the lens of TCMs, including their ability to explain the influence of emotion on episodic and temporal memory. In recent years, simplifying assumptions of "classical" TCMs - with exponential trace decay and the mechanism by which temporal context is recovered - have become increasingly clear. The review also outlines how recent advances could be incorporated into a future TCM, beyond classical assumptions, to integrate emotional modulation.

Learning to express reward prediction error-like dopaminergic activity requires plastic representations of time

The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference learning (TD) learning, whereby certain units signal reward prediction errors (RPE). The TD algorithm has been traditionally mapped onto the dopaminergic system, as firing properties of dopamine neurons can resemble RPEs. However, certain predictions of TD learning are inconsistent with experimental results, and previous implementations of the algorithm have made unscalable assumptions regarding stimulus-specific fixed temporal bases. We propose an alternate framework to describe dopamine signaling in the brain, FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, dopamine release is similar, but not identical to RPE, leading to predictions that contrast to those of TD. While FLEX itself is a general theoretical framework, we describe a specific, biophysically plausible implementation, the results of which are consistent with a preponderance of both existing and reanalyzed experimental data.

Multiphoton imaging of hippocampal neural circuits: techniques and biological insights into region-, cell-type-, and pathway-specific functions

Significance

The function of the hippocampus in behavior and cognition has long been studied primarily through electrophysiological recordings from freely moving rodents. However, the application of optical recording methods, particularly multiphoton fluorescence microscopy, in the last decade or two has dramatically advanced our understanding of hippocampal function. This article provides a comprehensive overview of techniques and biological findings obtained from multiphoton imaging of hippocampal neural circuits.

Aim

This review aims to summarize and discuss the recent technical advances in multiphoton imaging of hippocampal neural circuits and the accumulated biological knowledge gained through this technology.

Approach

First, we provide a brief overview of various techniques of multiphoton imaging of the hippocampus and discuss its advantages, drawbacks, and associated key innovations and practices. Then, we review a large body of findings obtained through multiphoton imaging by region (CA1 and dentate gyrus), cell type (pyramidal neurons, inhibitory interneurons, and glial cells), and cellular compartment (dendrite and axon).

Results

Multiphoton imaging of the hippocampus is primarily performed under head-fixed conditions and can reveal detailed mechanisms of circuit operation owing to its high spatial resolution and specificity. As the hippocampus lies deep below the cortex, its imaging requires elaborate methods. These include imaging cannula implantation, microendoscopy, and the use of long-wavelength light sources. Although many studies have focused on the dorsal CA1 pyramidal cells, studies of other local and inter-areal circuitry elements have also helped provide a more comprehensive picture of the information processing performed by the hippocampal circuits. Imaging of circuit function in mouse models of Alzheimer's disease and other brain disorders such as autism spectrum disorder has also contributed greatly to our understanding of their pathophysiology.

Conclusions

Multiphoton imaging has revealed much regarding region-, cell-type-, and pathway-specific mechanisms in hippocampal function and dysfunction in health and disease. Future technological advances will allow further illustration of the operating principle of the hippocampal circuits via the large-scale, high-resolution, multimodal, and minimally invasive imaging.

Volatile working memory representations crystallize with practice

Working memory, the process through which information is transiently maintained and manipulated over a brief period, is essential for most cognitive functions1-4. However, the mechanisms underlying the generation and evolution of working-memory neuronal representations at the population level over long timescales remain unclear. Here, to identify these mechanisms, we trained head-fixed mice to perform an olfactory delayed-association task in which the mice made decisions depending on the sequential identity of two odours separated by a 5 s delay. Optogenetic inhibition of secondary motor neurons during the late-delay and choice epochs strongly impaired the task performance of the mice. Mesoscopic calcium imaging of large neuronal populations of the secondary motor cortex (M2), retrosplenial cortex (RSA) and primary motor cortex (M1) showed that many late-delay-epoch-selective neurons emerged in M2 as the mice learned the task. Working-memory late-delay decoding accuracy substantially improved in the M2, but not in the M1 or RSA, as the mice became experts. During the early expert phase, working-memory representations during the late-delay epoch drifted across days, while the stimulus and choice representations stabilized. In contrast to single-plane layer 2/3 (L2/3) imaging, simultaneous volumetric calcium imaging of up to 73,307 M2 neurons, which included superficial L5 neurons, also revealed stabilization of late-delay working-memory representations with continued practice. Thus, delay- and choice-related activities that are essential for working-memory performance drift during learning and stabilize only after several days of expert performance.

Working memory features are embedded in hippocampal place fields

Hippocampal principal neurons display both spatial tuning properties and memory features. Whether this distinction corresponds to separate neuron types or a context-dependent continuum has been debated. We report here that the task-context ("splitter") feature is highly variable along both trial and spatial position axes. Neurons acquire or lose splitter features across trials even when place field features remain unaltered. Multiple place fields of the same neuron can individually encode both past or future run trajectories, implying that splitter fields are under the control of assembly activity. Place fields can be differentiated into subfields by the behavioral choice of the animal, and splitting within subfields evolves across trials. Interneurons also differentiate choices by integrating inputs from pyramidal cells. Finally, bilateral optogenetic inactivation of the medial entorhinal cortex reversibly decreases the fraction of splitter fields. Our findings suggest that place or splitter features are different manifestations of the same hippocampal computation.

Sequential activity of CA1 hippocampal cells constitutes a temporal memory map for associative learning in mice

Sequential neural dynamics encoded by time cells play a crucial role in hippocampal function. However, the role of hippocampal sequential neural dynamics in associative learning is an open question. We used two-photon Ca2+ imaging of dorsal CA1 (dCA1) neurons in the stratum pyramidale (SP) in head-fixed mice performing a go-no go associative learning task to investigate how odor valence is temporally encoded in this area of the brain. We found that SP cells responded differentially to the rewarded or unrewarded odor. The stimuli were decoded accurately from the activity of the neuronal ensemble, and accuracy increased substantially as the animal learned to differentiate the stimuli. Decoding the stimulus from individual SP cells responding differentially revealed that decision-making took place at discrete times after stimulus presentation. Lick prediction decoded from the ensemble activity of cells in dCA1 correlated linearly with lick behavior. Our findings indicate that sequential activity of SP cells in dCA1 constitutes a temporal memory map used for decision-making in associative learning. VIDEO ABSTRACT.

Hippocampal sequences span experience relative to rewards

Hippocampal place cells fire in sequences that span spatial environments and non-spatial modalities, suggesting that hippocampal activity can anchor to the most behaviorally salient aspects of experience. As reward is a highly salient event, we hypothesized that sequences of hippocampal activity can anchor to rewards. To test this, we performed two-photon imaging of hippocampal CA1 neurons as mice navigated virtual environments with changing hidden reward locations. When the reward moved, the firing fields of a subpopulation of cells moved to the same relative position with respect to reward, constructing a sequence of reward-relative cells that spanned the entire task structure. The density of these reward-relative sequences increased with task experience as additional neurons were recruited to the reward-relative population. Conversely, a largely separate subpopulation maintained a spatially-based place code. These findings thus reveal separate hippocampal ensembles can flexibly encode multiple behaviorally salient reference frames, reflecting the structure of the experience.

Neural Sequences and the Encoding of Time

Converging experimental and computational evidence indicate that on the scale of seconds the brain encodes time through changing patterns of neural activity. Experimentally, two general forms of neural dynamic regimes that can encode time have been observed: neural population clocks and ramping activity. Neural population clocks provide a high-dimensional code to generate complex spatiotemporal output patterns, in which each neuron exhibits a nonlinear temporal profile. A prototypical example of neural population clocks are neural sequences, which have been observed across species, brain areas, and behavioral paradigms. Additionally, neural sequences emerge in artificial neural networks trained to solve time-dependent tasks. Here, we examine the role of neural sequences in the encoding of time, and how they may emerge in a biologically plausible manner. We conclude that neural sequences may represent a canonical computational regime to perform temporal computations.

Reward Expectation Reduces Representational Drift in the Hippocampus

Spatial memory in the hippocampus involves dynamic neural patterns that change over days, termed representational drift. While drift may aid memory updating, excessive drift could impede retrieval. Memory retrieval is influenced by reward expectation during encoding, so we hypothesized that diminished reward expectation would exacerbate representational drift. We found that high reward expectation limited drift, with CA1 representations on one day gradually re-emerging over successive trials the following day. Conversely, the absence of reward expectation resulted in increased drift, as the gradual re-emergence of the previous day's representation did not occur. At the single cell level, lowering reward expectation caused an immediate increase in the proportion of place-fields with low trial-to-trial reliability. These place fields were less likely to be reinstated the following day, underlying increased drift in this condition. In conclusion, heightened reward expectation improves memory encoding and retrieval by maintaining reliable place fields that are gradually reinstated across days, thereby minimizing representational drift.

Automating licking bias correction in a two-choice delayed match-to-sample task to accelerate learning

Animals often display choice bias, or a preference for one option over the others, which can significantly impede learning new tasks. Delayed match-to-sample (DMS) tasks with two-alternative choices of lickports on the left and right have been widely used to study sensory processing, working memory, and associative memory in head-fixed animals. However, extensive training time, primarily due to the animals' biased licking responses, limits their practical utility. Here, we present the implementation of an automated side bias correction system in an olfactory DMS task, where the lickport positions and the ratio of left- and right-rewarded trials are dynamically adjusted to counterbalance mouse's biased licking responses during training. The correction algorithm moves the preferred lickport farther away from the mouse's mouth and the non-preferred lickport closer, while also increasing the proportion of non-preferred side trials when biased licking occurs. We found that adjusting lickport distances and the proportions of left- versus right-rewarded trials effectively reduces the mouse's side bias. Further analyses reveal that these adjustments also correlate with subsequent improvements in behavioral performance. Our findings suggest that the automated side bias correction system is a valuable tool for enhancing the applicability of behavioral tasks involving two-alternative lickport choices.

Temporal encoding in deep reinforcement learning agents

Neuroscientists have observed both cells in the brain that fire at specific points in time, known as "time cells", and cells whose activity steadily increases or decreases over time, known as "ramping cells". It is speculated that time and ramping cells support temporal computations in the brain and carry mnemonic information. However, due to the limitations in animal experiments, it is difficult to determine how these cells really contribute to behavior. Here, we show that time cells and ramping cells naturally emerge in the recurrent neural networks of deep reinforcement learning models performing simulated interval timing and working memory tasks, which have learned to estimate expected rewards in the future. We show that these cells do indeed carry information about time and items stored in working memory, but they contribute to behavior in large part by providing a dynamic representation on which policy can be computed. Moreover, the information that they do carry depends on both the task demands and the variables provided to the models. Our results suggest that time cells and ramping cells could contribute to temporal and mnemonic calculations, but the way in which they do so may be complex and unintuitive to human observers.

A neural code for time and space in the human brain

Time and space are primary dimensions of human experience. Separate lines of investigation have identified neural correlates of time and space, yet little is known about how these representations converge during self-guided experience. Here, 10 subjects with intracranially implanted microelectrodes play a timed, virtual navigation game featuring object search and retrieval tasks separated by fixed delays. Time cells and place cells activate in parallel during timed navigation intervals, whereas a separate time cell sequence spans inter-task delays. The prevalence, firing rates, and behavioral coding strengths of time cells and place cells are indistinguishable-yet time cells selectively remap between search and retrieval tasks, while place cell responses remain stable. Thus, the brain can represent time and space as overlapping but dissociable dimensions. Time cells and place cells may constitute a biological basis for the cognitive map of spatiotemporal context onto which memories are written.

Temporally specific patterns of neural activity in interconnected corticolimbic structures during reward anticipation

Functional neuroimaging studies indicate that interconnected parts of the subcallosal anterior cingulate cortex (ACC), striatum, and amygdala play a fundamental role in affect in health and disease. Yet, although the patterns of neural activity engaged in the striatum and amygdala during affective processing are well established, especially during reward anticipation, less is known about subcallosal ACC. Here, we recorded neural activity in non-human primate subcallosal ACC and compared this with interconnected parts of the basolateral amygdala and rostromedial striatum while macaque monkeys performed reward-based tasks. Applying multiple analysis approaches, we found that neurons in subcallosal ACC and rostromedial striatum preferentially signal anticipated reward using short bursts of activity that form temporally specific patterns. By contrast, the basolateral amygdala uses a mixture of both temporally specific and more sustained patterns of activity to signal anticipated reward. Thus, dynamic patterns of neural activity across populations of neurons are engaged in affect, especially in subcallosal ACC.

Time for Memories

The ability to store information about the past to dynamically predict and prepare for the future is among the most fundamental tasks the brain performs. To date, the problems of understanding how the brain stores and organizes information about the past (memory) and how the brain represents and processes temporal information for adaptive behavior have generally been studied as distinct cognitive functions. This Symposium explores the inherent link between memory and temporal cognition, as well as the potential shared neural mechanisms between them. We suggest that working memory and implicit timing are interconnected and may share overlapping neural mechanisms. Additionally, we explore how temporal structure is encoded in associative and episodic memory and, conversely, the influences of episodic memory on subsequent temporal anticipation and the perception of time. We suggest that neural sequences provide a general computational motif that contributes to timing and working memory, as well as the spatiotemporal coding and recall of episodes.

Decision-Making Time Cells in Hippocampal Dorsal CA1

Sequential neural dynamics encoded by "time cells" play a crucial role in hippocampal function. However, the role of hippocampal sequential neural dynamics in associative learning is an open question. In this manuscript, we used two-photon Ca2+ imaging of dorsal CA1 pyramidal neurons in head-fixed mice performing a go-no-go associative learning task. We found that pyramidal cells responded differentially to the rewarded or unrewarded stimuli. The stimuli were decoded accurately from the activity of the neuronal ensemble, and accuracy increased substantially as the animal learned to differentiate the stimuli. Decoding the stimulus from individual pyramidal cells that responded differentially revealed that decision-making took place at discrete times after stimulus presentation. Lick prediction decoded from the ensemble activity of cells in dCA1 correlated linearly with lick behavior indicating that sequential activity of pyramidal cells in dCA1 constitutes a temporal memory map used for decision-making in associative learning.

Learning to Express Reward Prediction Error-like Dopaminergic Activity Requires Plastic Representations of Time

The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference (TD) reinforcement learning. The TD framework predicts that some neuronal elements should represent the reward prediction error (RPE), which means they signal the difference between the expected future rewards and the actual rewards. The prominence of the TD theory arises from the observation that firing properties of dopaminergic neurons in the ventral tegmental area appear similar to those of RPE model-neurons in TD learning. Previous implementations of TD learning assume a fixed temporal basis for each stimulus that might eventually predict a reward. Here we show that such a fixed temporal basis is implausible and that certain predictions of TD learning are inconsistent with experiments. We propose instead an alternative theoretical framework, coined FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, feature specific representations of time are learned, allowing for neural representations of stimuli to adjust their timing and relation to rewards in an online manner. In FLEX dopamine acts as an instructive signal which helps build temporal models of the environment. FLEX is a general theoretical framework that has many possible biophysical implementations. In order to show that FLEX is a feasible approach, we present a specific biophysically plausible model which implements the principles of FLEX. We show that this implementation can account for various reinforcement learning paradigms, and that its results and predictions are consistent with a preponderance of both existing and reanalyzed experimental data.

Accelerated social representational drift in the nucleus accumbens in a model of autism

Impaired social interaction is one of the core deficits of autism spectrum disorder (ASD) and may result from social interactions being less rewarding. How the nucleus accumbens (NAc), as a key hub of reward circuitry, encodes social interaction and whether these representations are altered in ASD remain poorly understood. We identified NAc ensembles encoding social interactions by calcium imaging using miniaturized microscopy. NAc population activity, specifically D1 receptor-expressing medium spiny neurons (D1-MSNs) activity, predicted social interaction epochs. Despite a high turnover of NAc neurons modulated by social interaction, we found a stable population code for social interaction in NAc which was dramatically degraded in Cntnap2-/- mouse model of ASD. Surprisingly, non-specific optogenetic inhibition of NAc core neurons increased social interaction time and significantly improved sociability in Cntnap2-/- mice. Inhibition of D1- or D2-MSNs showed reciprocal effects, with D1 inhibition decreasing social interaction and D2 inhibition increasing interaction. Therefore, social interactions are preferentially, specifically and dynamically encoded by NAc neurons and social representations are degraded in this autism model.

A positively tuned voltage indicator for extended electrical recordings in the brain

Genetically encoded voltage indicators (GEVIs) enable optical recording of electrical signals in the brain, providing subthreshold sensitivity and temporal resolution not possible with calcium indicators. However, one- and two-photon voltage imaging over prolonged periods with the same GEVI has not yet been demonstrated. Here, we report engineering of ASAP family GEVIs to enhance photostability by inversion of the fluorescence-voltage relationship. Two of the resulting GEVIs, ASAP4b and ASAP4e, respond to 100-mV depolarizations with ≥180% fluorescence increases, compared with the 50% fluorescence decrease of the parental ASAP3. With standard microscopy equipment, ASAP4e enables single-trial detection of spikes in mice over the course of minutes. Unlike GEVIs previously used for one-photon voltage recordings, ASAP4b and ASAP4e also perform well under two-photon illumination. By imaging voltage and calcium simultaneously, we show that ASAP4b and ASAP4e can identify place cells and detect voltage spikes with better temporal resolution than commonly used calcium indicators. Thus, ASAP4b and ASAP4e extend the capabilities of voltage imaging to standard one- and two-photon microscopes while improving the duration of voltage recordings.

Multiplexing working memory and time in the trajectories of neural networks

Working memory (WM) and timing are generally considered distinct cognitive functions, but similar neural signatures have been implicated in both. To explore the hypothesis that WM and timing may rely on shared neural mechanisms, we used psychophysical tasks that contained either task-irrelevant timing or WM components. In both cases, the task-irrelevant component influenced performance. We then developed recurrent neural network (RNN) simulations that revealed that cue-specific neural sequences, which multiplexed WM and time, emerged as the dominant regime that captured the behavioural findings. During training, RNN dynamics transitioned from low-dimensional ramps to high-dimensional neural sequences, and depending on task requirements, steady-state or ramping activity was also observed. Analysis of RNN structure revealed that neural sequences relied primarily on inhibitory connections, and could survive the deletion of all excitatory-to-excitatory connections. Our results indicate that in some instances WM is encoded in time-varying neural activity because of the importance of predicting when WM will be used.

Hippocampal and Medial Prefrontal Cortex Fractal Spiking Patterns Encode Episodes and Rules

A central question in neuroscience is how the brain represents and processes information to guide behavior. The principles that organize brain computations are not fully known, and could include scale-free, or fractal patterns of neuronal activity. Scale-free brain activity may be a natural consequence of the relatively small subsets of neuronal populations that respond to task features, i.e., sparse coding. The size of the active subsets constrains the possible sequences of inter-spike intervals (ISI), and selecting from this limited set may produce firing patterns across wide-ranging timescales that form fractal spiking patterns. To investigate the extent to which fractal spiking patterns corresponded with task features, we analyzed ISIs in simultaneously recorded populations of CA1 and medial prefrontal cortical (mPFC) neurons in rats performing a spatial memory task that required both structures. CA1 and mPFC ISI sequences formed fractal patterns that predicted memory performance. CA1 pattern duration, but not length or content, varied with learning speed and memory performance whereas mPFC patterns did not. The most common CA1 and mPFC patterns corresponded with each region's cognitive function: CA1 patterns encoded behavioral episodes which linked the start, choice, and goal of paths through the maze whereas mPFC patterns encoded behavioral "rules" which guided goal selection. mPFC patterns predicted changing CA1 spike patterns only as animals learned new rules. Together, the results suggest that CA1 and mPFC population activity may predict choice outcomes by using fractal ISI patterns to compute task features.

Neural dynamics underlying associative learning in the dorsal and ventral hippocampus

Animals associate cues with outcomes and update these associations as new information is presented. This requires the hippocampus, yet how hippocampal neurons track changes in cue-outcome associations remains unclear. Using two-photon calcium imaging, we tracked the same dCA1 and vCA1 neurons across days to determine how responses evolve across phases of odor-outcome learning. Initially, odors elicited robust responses in dCA1, whereas, in vCA1, odor responses primarily emerged after learning and embedded information about the paired outcome. Population activity in both regions rapidly reorganized with learning and then stabilized, storing learned odor representations for days, even after extinction or pairing with a different outcome. Additionally, we found stable, robust signals across CA1 when mice anticipated outcomes under behavioral control but not when mice anticipated an inescapable aversive outcome. These results show how the hippocampus encodes, stores and updates learned associations and illuminates the unique contributions of dorsal and ventral hippocampus.

Voltage imaging reveals that hippocampal interneurons tune memory-encoding pyramidal sequences

Hippocampal spiking sequences encode and link behavioral information across time. How inhibition sculpts these sequences remains unknown. We performed longitudinal voltage imaging of CA1 parvalbumin- and somatostatin-expressing interneurons in mice during an odor-cued working memory task, before and after training. During this task, pyramidal odor-specific sequences encode the cue throughout a delay period. In contrast, most interneurons encoded odor delivery, but not odor identity, nor delay time. Population inhibition was stable across days, with constant field turnover, though some cells retained odor-responses for days. At odor onset, a brief, synchronous burst of parvalbumin cells was followed by widespread membrane hyperpolarization and then rebound theta-paced spiking, synchronized across cells. Two-photon calcium imaging revealed that most pyramidal cells were suppressed throughout the odor. Positive pyramidal odor-responses coincided with interneuronal rebound spiking; otherwise, they had weak odor-selectivity. Therefore, inhibition increases the signal-to-noise ratio of cue representations, which is crucial for entraining downstream targets.

Hippocampal conjunctive and complementary CA1 populations relate sensory events to movement

Hippocampal CA1 neurons respond to sensory stimuli during enforced immobility, movement, and their transitions in a new conveyor belt task. Head-fixed mice were exposed to light flashes or air streams while at rest, spontaneously moving, or running a fixed distance. Two-photon calcium imaging of CA1 neurons revealed that 62% of 3341 imaged cells were active during one or more of 20 sensorimotor events. Of these active cells, 17% were active for any given sensorimotor event, with a higher proportion during locomotion. The study found two types of cells: Conjunctive cells that were active across multiple events, and complementary cells that were active only during individual events, encoding novel sensorimotor events or their delayed repetitions. The configuration of these cells across changing sensorimotor events may signify the role of hippocampus in functional networks integrating sensory information with ongoing movement making it suitable for movement guidance.

A CA1 circuit motif that signals the start of information integration

Integrating information from the recent past is critical for guiding predictions and shaping behavior. The process of integrating information, such as tracking distance traveled or time elapsed, begins with establishing a starting point. Yet, the mechanisms by which neural circuits utilize relevant cues to initiate integration remain unknown. Our study sheds light on this question by identifying a subpopulation of CA1 pyramidal neurons called PyrDown. These neurons shut down their activity at the beginning of distance or time integration and then gradually ramp up their firing as the animal approaches the reward. PyrDown neurons provide a mechanism for representing integrated information through ramping activity, complementing the well-known place/time cells that respond to specific distances or time points. Our findings also reveal that parvalbumin inhibitory interneurons mediate the shutdown of PyrDown neurons, uncovering a circuit motif that enables the initiation of subsequent information integration to improve future predictions.

Foundations of a temporal RL

Recent advances in neuroscience and psychology show that the brain has access to timelines of both the past and the future. Spiking across populations of neurons in many regions of the mammalian brain maintains a robust temporal memory, a neural timeline of the recent past. Behavioral results demonstrate that people can estimate an extended temporal model of the future, suggesting that the neural timeline of the past could extend through the present into the future. This paper presents a mathematical framework for learning and expressing relationships between events in continuous time. We assume that the brain has access to a temporal memory in the form of the real Laplace transform of the recent past. Hebbian associations with a diversity of synaptic time scales are formed between the past and the present that record the temporal relationships between events. Knowing the temporal relationships between the past and the present allows one to predict relationships between the present and the future, thus constructing an extended temporal prediction for the future. Both memory for the past and the predicted future are represented as the real Laplace transform, expressed as the firing rate over populations of neurons indexed by different rate constants s. The diversity of synaptic timescales allows for a temporal record over the much larger time scale of trial history. In this framework, temporal credit assignment can be assessed via a Laplace temporal difference. The Laplace temporal difference compares the future that actually follows a stimulus to the future predicted just before the stimulus was observed. This computational framework makes a number of specific neurophysiological predictions and, taken together, could provide the basis for a future iteration of RL that incorporates temporal memory as a fundamental building block.

Contextual and pure time coding for self and other in the hippocampus

Navigation and episodic memory depend critically on representing temporal sequences. Hippocampal 'time cells' form temporal sequences, but it is unknown whether they represent context-dependent experience or time per se. Here we report on time cells in bat hippocampal area CA1, which, surprisingly, formed two distinct populations. One population of time cells generated different temporal sequences when the bat hung at different locations, thus conjunctively encoding spatial context and time-'contextual time cells'. A second population exhibited similar preferred times across different spatial contexts, thus purely encoding elapsed time. When examining neural responses after the landing moment of another bat, in a social imitation task, we found time cells that encoded temporal sequences aligned to the other's landing. We propose that these diverse time codes may support the perception of interval timing, episodic memory and temporal coordination between self and others.

Calcium activity is a degraded estimate of spikes

Recording action potentials extracellularly during behavior has led to fundamental discoveries regarding neural function-hippocampal neurons respond to locations in space,1 motor cortex neurons encode movement direction,2 and dopamine neurons signal reward prediction errors3-observations undergirding current theories of cognition,4 movement,5 and learning.6 Recently it has become possible to measure calcium flux, an internal cellular signal related to spiking. The ability to image calcium flux in anatomically7,8 or genetically9 identified neurons can extend our knowledge of neural circuit function by allowing activity to be monitored in specific cell types or projections, or in the same neurons across many days. However, while initial studies were grounded in prior unit recording work, it has become fashionable to assume that calcium is identical to spiking, even though the spike-to-fluorescence transformation is nonlinear, noisy, and unpredictable under real-world conditions.10 It remains an open question whether calcium provides a high-fidelity representation of single-unit activity in awake, behaving subjects. Here, we have addressed this question by recording both signals in the lateral orbitofrontal cortex (OFC) of rats during olfactory discrimination learning. Activity in the OFC during olfactory learning has been well-studied in humans,11,12,13,14 nonhuman primates,15,16 and rats,17,18,19,20,21 where it has been shown to signal information about both the sensory properties of odor cues and the rewards they predict. Our single-unit results replicated prior findings, whereas the calcium signal provided only a degraded estimate of the information available in the single-unit spiking, reflecting primarily reward value.

Reconciling neuronal representations of schema, abstract task structure, and categorization under cognitive maps in the entorhinal-hippocampal-frontal circuits

Learning leads to a neuronal representation of acquired knowledge. This idea of knowledge representation was traditionally developed as a "cognitive map" of spatial memory represented in the hippocampus. The framework of cognitive mapping has been extended in the past decade to include not only spatial memory, but also non-spatial factual and temporal memory. Following this conceptual advancement, a line of recent neurophysiological research discovered such knowledge representations not only in the hippocampus, but also in the entorhinal cortex and frontal cortex. Although the distinct terms "cognitive map," "schema," "abstract task structure" or "categorization" were used in these studies, it is likely that these terms can be reconciled as a common mechanism of learned knowledge representations. Future experimental work will be required to differentiate the parametric nature of knowledge representations across brain areas.

Task-selective place cells show behaviorally driven dynamics during learning and stability during memory recall

Decades of work propose that hippocampal activity supports internal representation of learned experiences and contexts, allowing individuals to form long-term memories and quickly adapt behavior to changing environments. However, recent studies insinuate hippocampal representations can drift over time, raising the question: how could the hippocampus hold stable memories when activity of its neuronal maps fluctuates? We hypothesized that task-dependent hippocampal maps set by learning rules and structured attention stabilize as a function of behavioral performance. To test this, we imaged hippocampal CA1 pyramidal neurons during learning and memory recall phases of a new task where mice use odor cues to navigate between two reward zones. Across learning, both orthogonal and overlapping task-dependent place maps form rapidly, discriminating trial context with strong correlation to behavioral performance. Once formed, task-selective place maps show increased long-term stability during memory recall phases. We conclude that memory demand and attention stabilize hippocampal activity to maintain contextually rich spatial representations.

Neural population clocks: Encoding time in dynamic patterns of neural activity

The ability to predict and prepare for near- and far-future events is among the most fundamental computations the brain performs. Because of the importance of time for prediction and sensorimotor processing, the brain has evolved multiple mechanisms to tell and encode time across scales ranging from microseconds to days and beyond. Converging experimental and computational data indicate that, on the scale of seconds, timing relies on diverse neural mechanisms distributed across different brain areas. Among the different encoding mechanisms on the scale of seconds, we distinguish between neural population clocks and ramping activity as distinct strategies to encode time. One instance of neural population clocks, neural sequences, represents in some ways an optimal and flexible dynamic regime for the encoding of time. Specifically, neural sequences comprise a high-dimensional representation that can be used by downstream areas to flexibly generate arbitrarily simple and complex output patterns using biologically plausible learning rules. We propose that high-level integration areas may use high-dimensional dynamics such as neural sequences to encode time, providing downstream areas information to build low-dimensional ramp-like activity that can drive movements and temporal expectation. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The neural bases for timing of durations

Durations are defined by a beginning and an end, and a major distinction is drawn between durations that start in the present and end in the future ('prospective timing') and durations that start in the past and end either in the past or the present ('retrospective timing'). Different psychological processes are thought to be engaged in each of these cases. The former is thought to engage a clock-like mechanism that accurately tracks the continuing passage of time, whereas the latter is thought to engage a reconstructive process that utilizes both temporal and non-temporal information from the memory of past events. We propose that, from a biological perspective, these two forms of duration 'estimation' are supported by computational processes that are both reliant on population state dynamics but are nevertheless distinct. Prospective timing is effectively carried out in a single step where the ongoing dynamics of population activity directly serve as the computation of duration, whereas retrospective timing is carried out in two steps: the initial generation of population state dynamics through the process of event segmentation and the subsequent computation of duration utilizing the memory of those dynamics.

Complementary representations of time in the prefrontal cortex and hippocampus

Episodic memory binds the spatial and temporal relationships between the elements of experience. The hippocampus encodes space through place cells that fire at specific spatial locations. Similarly, time cells fire sequentially at specific time points within a temporally organized experience. Recent studies in rodents, monkeys, and humans have identified time cells with discrete firing fields and cells with monotonically changing activity in supporting the temporal organization of events across multiple timescales. Using in vivo electrophysiological tetrode recordings, we simultaneously recorded neurons from the prefrontal cortex and dorsal CA1 of the hippocampus while rats performed a delayed match to sample task. During the treadmill mnemonic delay, hippocampal time cells exhibited sparser firing fields with decreasing resolution over time, consistent with previous results. In comparison, temporally modulated cells in the prefrontal cortex showed more monotonically changing firing rates, ramping up or decaying with the passage of time, and exhibited greater temporal precision for Bayesian decoding of time at long time lags. These time cells show exquisite temporal resolution both in their firing fields and in the fine timing of spikes relative to the phase of theta oscillations. Here, we report evidence of theta phase precession in both the prefrontal cortex and hippocampus during the temporal delay, however, hippocampal cells exhibited steeper phase precession slopes and more punctate time fields. To disentangle whether time cell activity reflects elapsed time or distance traveled, we varied the treadmill running speed on each trial. While many neurons contained multiplexed representations of time and distance, both regions were more strongly influenced by time than distance. Overall, these results demonstrate the flexible integration of spatiotemporal dimensions and reveal complementary representations of time in the prefrontal cortex and hippocampus in supporting memory-guided behavior.

A behavioral paradigm for measuring perceptual distances in mice

Perceptual similarities between a specific stimulus and other stimuli of the same modality provide valuable information about the structure and geometry of sensory spaces. While typically assessed in human behavioral experiments, perceptual similarities-or distances-are rarely measured in other species. However, understanding the neural computations responsible for sensory representations requires the monitoring and often manipulation of neural activity, which is more readily achieved in non-human experimental models. Here, we develop a behavioral paradigm that enables the quantification of perceptual similarity between sensory stimuli using mouse olfaction as a model system.

Consistent population activity on the scale of minutes in the mouse hippocampus

Neurons in the hippocampus fire in consistent sequence over the timescale of seconds during the delay period of some memory experiments. For longer timescales, the firing of hippocampal neurons also changes slowly over minutes within experimental sessions. It was thought that these slow dynamics are caused by stochastic drift or a continuous change in the representation of the episode, rather than consistent sequences unfolding over minutes. This paper studies the consistency of contextual drift in three chronic calcium imaging recordings from the hippocampus CA1 region in mice. Computational measures of consistency show reliable sequences within experimental trials at the scale of seconds as one would expect from time cells or place cells during the trial, as well as across experimental trials on the scale of minutes within a recording session. Consistent sequences in the hippocampus are observed over a wide range of time scales, from seconds to minutes. The hippocampal activity could reflect a scale-invariant spatiotemporal context as suggested by theories of memory from cognitive psychology.

Input rate encoding and gain control in dendrites of neocortical pyramidal neurons

Elucidating how neurons encode network activity is essential to understanding how the brain processes information. Neocortical pyramidal cells receive excitatory input onto spines distributed along dendritic branches. Local dendritic branch nonlinearities can boost the response to spatially clustered and synchronous input, but how this translates into the integration of patterns of ongoing activity remains unclear. To examine dendritic integration under naturalistic stimulus regimes, we use two-photon glutamate uncaging to repeatedly activate multiple dendritic spines at random intervals. In the proximal dendrites of two populations of layer 5 pyramidal neurons in the mouse motor cortex, spatially restricted synchrony is not a prerequisite for dendritic boosting. Branches encode afferent inputs with distinct rate sensitivities depending upon cell and branch type. Thus, inputs distributed along a dendritic branch can recruit supralinear boosting and the window of this nonlinearity may provide a mechanism by which dendrites can preferentially amplify slow-frequency network oscillations.

Rhythmic coordination and ensemble dynamics in the hippocampal-prefrontal network during odor-place associative memory and decision making

Memory-guided decision making involves long-range coordination across sensory and cognitive brain networks, with key roles for the hippocampus and prefrontal cortex (PFC). In order to investigate the mechanisms of such coordination, we monitored activity in hippocampus (CA1), PFC, and olfactory bulb (OB) in rats performing an odor-place associative memory guided decision task on a T-maze. During odor sampling, the beta (20-30 Hz) and respiratory (7-8 Hz) rhythms (RR) were prominent across the three regions, with beta and RR coherence between all pairs of regions enhanced during the odor-cued decision making period. Beta phase modulation of phase-locked CA1 and PFC neurons during this period was linked to accurate decisions, with a key role of CA1 interneurons in temporal coordination. Single neurons and ensembles in both CA1 and PFC encoded and predicted animals' upcoming choices, with different cell ensembles engaged during decision-making and decision execution on the maze. Our findings indicate that rhythmic coordination within the hippocampal-prefrontal-olfactory bulb network supports utilization of odor cues for memory-guided decision making.

Peering into the Brain through the Retrosplenial Cortex to Assess Cognitive Function of the Injured Brain

The retrosplenial cortex (RSC) is a posterior cortical area that has been drawing increasing interest in recent years, with a growing number of studies studying its contribution to cognitive and sensory functions. From an anatomical perspective, it has been established that the RSC is extensively and often reciprocally connected with the hippocampus, neocortex, and many midbrain regions. Functionally, the RSC is an important hub of the default-mode network. This endowment, with vast anatomical and functional connections, positions the RSC to play an important role in episodic memory, spatial and contextual learning, sensory-cognitive activities, and multi-modal sensory information processing and integration. Additionally, RSC dysfunction has been reported in cases of cognitive decline, particularly in Alzheimer's disease and stroke. We review the literature to examine whether the RSC can act as a cortical marker of persistent cognitive dysfunction after traumatic brain injury (TBI). Because the RSC is easily accessible at the brain's surface using in vivo techniques, we argue that studying RSC network activity post-TBI can shed light into the mechanisms of less-accessible brain regions, such as the hippocampus. There is a fundamental gap in the TBI field about the microscale alterations occurring post-trauma, and by studying the RSC's neuronal activity at the cellular level we will be able to design better therapeutic tools. Understanding how neuronal activity and interactions produce normal and abnormal activity in the injured brain is crucial to understanding cognitive dysfunction. By using this approach, we expect to gain valuable insights to better understand brain disorders like TBI.

Olfactory landmarks and path integration converge to form a cognitive spatial map

The convergence of internal path integration and external sensory landmarks generates a cognitive spatial map in the hippocampus. We studied how localized odor cues are recognized as landmarks by recording the activity of neurons in CA1 during a virtual navigation task. We found that odor cues enriched place cell representations, dramatically improving navigation. Presentation of the same odor at different locations generated distinct place cell representations. An odor cue at a proximal location enhanced the local place cell density and also led to the formation of place cells beyond the cue. This resulted in the recognition of a second, more distal odor cue as a distinct landmark, suggesting an iterative mechanism for extending spatial representations into unknown territory. Our results establish that odors can serve as landmarks, motivating a model in which path integration and odor landmarks interact sequentially and iteratively to generate cognitive spatial maps over long distances.

Learning differentially shapes prefrontal and hippocampal activity during classical conditioning

The ability to use sensory cues to inform goal-directed actions is a critical component of behavior. To study how sounds guide anticipatory licking during classical conditioning, we employed high-density electrophysiological recordings from the hippocampal CA1 area and the prefrontal cortex (PFC) in mice. CA1 and PFC neurons undergo distinct learning-dependent changes at the single-cell level and maintain representations of cue identity at the population level. In addition, reactivation of task-related neuronal assemblies during hippocampal awake Sharp-Wave Ripples (aSWRs) changed within individual sessions in CA1 and over the course of multiple sessions in PFC. Despite both areas being highly engaged and synchronized during the task, we found no evidence for coordinated single cell or assembly activity during conditioning trials or aSWR. Taken together, our findings support the notion that persistent firing and reactivation of task-related neural activity patterns in CA1 and PFC support learning during classical conditioning.

The grid code for ordered experience

Entorhinal cortical grid cells fire in a periodic pattern that tiles space, which is suggestive of a spatial coordinate system. However, irregularities in the grid pattern as well as responses of grid cells in contexts other than spatial navigation have presented a challenge to existing models of entorhinal function. In this Perspective, we propose that hippocampal input provides a key informative drive to the grid network in both spatial and non-spatial circumstances, particularly around salient events. We build on previous models in which neural activity propagates through the entorhinal-hippocampal network in time. This temporal contiguity in network activity points to temporal order as a necessary characteristic of representations generated by the hippocampal formation. We advocate that interactions in the entorhinal-hippocampal loop build a topological representation that is rooted in the temporal order of experience. In this way, the structure of grid cell firing supports a learned topology rather than a rigid coordinate frame that is bound to measurements of the physical world.

Behavior determines the hippocampal spatial mapping of a multisensory environment

Animals behave in multisensory environments guided by various modalities of spatial information. Mammalian navigation engages a cognitive map of space in the hippocampus. Yet it is unknown whether and how this map incorporates multiple modalities of spatial information. Here, we establish two behavioral tasks in which mice navigate the same multisensory virtual environment by either pursuing a visual landmark or tracking an odor gradient. These tasks engage different proportions of visuo-spatial and olfacto-spatial mapping CA1 neurons and different population-level representations of each sensory-spatial coordinate. Switching between tasks results in global remapping. In a third task, mice pursue a target of varying sensory modality, and this engages modality-invariant neurons mapping the abstract behaviorally relevant coordinate irrespective of its physical modality. These findings demonstrate that the hippocampus does not necessarily map space as one coherent physical variable but as a combination of sensory and abstract reference frames determined by the subject's behavioral goal.

Navigating for reward

An organism's survival can depend on its ability to recall and navigate to spatial locations associated with rewards, such as food or a home. Accumulating research has revealed that computations of reward and its prediction occur on multiple levels across a complex set of interacting brain regions, including those that support memory and navigation. However, how the brain coordinates the encoding, recall and use of reward information to guide navigation remains incompletely understood. In this Review, we propose that the brain's classical navigation centres - the hippocampus and the entorhinal cortex - are ideally suited to coordinate this larger network by representing both physical and mental space as a series of states. These states may be linked to reward via neuromodulatory inputs to the hippocampus-entorhinal cortex system. Hippocampal outputs can then broadcast sequences of states to the rest of the brain to store reward associations or to facilitate decision-making, potentially engaging additional value signals downstream. This proposal is supported by recent advances in both experimental and theoretical neuroscience. By discussing the neural systems traditionally tied to navigation and reward at their intersection, we aim to offer an integrated framework for understanding navigation to reward as a fundamental feature of many cognitive processes.

Dynamic and reversible remapping of network representations in an unchanging environment

Neurons in the medial entorhinal cortex alter their firing properties in response to environmental changes. This flexibility in neural coding is hypothesized to support navigation and memory by dividing sensory experience into unique episodes. However, it is unknown how the entorhinal circuit as a whole transitions between different representations when sensory information is not delineated into discrete contexts. Here we describe rapid and reversible transitions between multiple spatial maps of an unchanging task and environment. These remapping events were synchronized across hundreds of neurons, differentially affected navigational cell types, and correlated with changes in running speed. Despite widespread changes in spatial coding, remapping comprised a translation along a single dimension in population-level activity space, enabling simple decoding strategies. These findings provoke reconsideration of how the medial entorhinal cortex dynamically represents space and suggest a remarkable capacity of cortical circuits to rapidly and substantially reorganize their neural representations.

Distinct Neural Representations of Content and Ordinal Structure in Auditory Sequence Memory

Two forms of information, frequency (content) and ordinal position (structure), have to be stored when retaining a sequence of auditory tones in working memory (WM). However, the neural representations and coding characteristics of content and structure, particularly during WM maintenance, remain elusive. Here, in two EEG studies in human participants (both sexes), by transiently perturbing the "activity-silent" WM retention state and decoding the reactivated WM information, we demonstrate that content and structure are stored in a dissociative manner with distinct characteristics throughout WM process. First, each tone in the sequence is associated with two codes in parallel, characterizing its frequency and ordinal position, respectively. Second, during retention, a structural retrocue successfully reactivates structure but not content, whereas a following white noise triggers content but not structure. Third, structure representation remains stable, whereas content code undergoes a dynamic transformation through memory progress. Finally, the noise-triggered content reactivations during retention correlate with subsequent WM behavior. Overall, our results support distinct content and structure representations in auditory WM and provide an efficient approach to access the silently stored WM information in the human brain. The dissociation of content and structure could facilitate efficient memory formation via generalizing stable structure to new auditory contents.SIGNIFICANCE STATEMENT In memory experiences, contents do not exist independently but are linked with each other via ordinal structure. For instance, recalling a piece of favorite music relies on correct ordering (sequence structure) of musical tones (content). How are the structure and content for an auditory temporally structured experience maintained in working memory? Here, by using impulse-response approach and time-resolved representational dissimilarity analysis on human EEG recordings in an auditory working memory task, we reveal that content and structure are stored in a dissociated way, which would facilitate efficient and rapid memory formation through generalizing stable structure knowledge to new auditory inputs.

Geometry of abstract learned knowledge in the hippocampus

Hippocampal neurons encode physical variables^1-7 such as space¹ or auditory frequency⁶ in cognitive maps⁸. In addition, functional magnetic resonance imaging studies in humans have shown that the hippocampus can also encode more abstract, learned variables^9-11. However, their integration into existing neural representations of physical variables^12,13 is unknown. Here, using two-photon calcium imaging, we show that individual neurons in the dorsal hippocampus jointly encode accumulated evidence with spatial position in mice performing a decision-making task in virtual reality^14-16. Nonlinear dimensionality reduction¹³ showed that population activity was well-described by approximately four to six latent variables, which suggests that neural activity is constrained to a low-dimensional manifold. Within this low-dimensional space, both physical and abstract variables were jointly mapped in an orderly manner, creating a geometric representation that we show is similar across mice. The existence of conjoined cognitive maps suggests that the hippocampus performs a general computation-the creation of task-specific low-dimensional manifolds that contain a geometric representation of learned knowledge.

Is Activity Silent Working Memory Simply Episodic Memory?

Working memory (WM) maintains task-relevant information in a state ready for processing. While traditional theories assume that sustained neuronal activity is responsible for WM, the Activity Silent WM (ASWM) account proposes that maintenance can also be supported by short-term synaptic weight changes. Here, we argue that the evidence for ASWM can be explained more parsimoniously by the involvement of episodic memory (EM) in WM tasks. Like ASWM, EM relies on rapid synaptic modification that is also activity silent; however, while ASWM posits transient synaptic modifications, EM traces persist over longer time periods. We discuss how, despite this difference, well-established EM mechanisms can account for the key findings attributed to ASWM, and describe predictions of this account.

Hippocampal spatial memory representations in mice are heterogeneously stable

The population of hippocampal neurons actively coding space continually changes across days as mice repeatedly perform tasks. Many hippocampal place cells become inactive while other previously silent neurons become active, challenging the idea that stable behaviors and memory representations are supported by stable patterns of neural activity. Active cell replacement may disambiguate unique episodes that contain overlapping memory cues, and could contribute to reorganization of memory representations. How active cell replacement affects the evolution of representations of different behaviors within a single task is unknown. We trained mice to perform a delayed nonmatching to place task over multiple weeks, and performed calcium imaging in area CA1 of the dorsal hippocampus using head-mounted miniature microscopes. Cells active on the central stem of the maze "split" their calcium activity according to the animal's upcoming turn direction (left or right), the current task phase (study or test), or both task dimensions, even while spatial cues remained unchanged. We found that, among reliably active cells, different splitter neuron populations were replaced at unequal rates, resulting in an increasing number of cells modulated by turn direction and a decreasing number of cells with combined modulation by both turn direction and task phase. Despite continual reorganization, the ensemble code stably segregated these task dimensions. These results show that hippocampal memories can heterogeneously reorganize even while behavior is unchanging.