Hippocampal activity patterns carry information about objects in temporal context

Event memory uniquely predicts memory for large-scale space

When a person explores a new environment, they begin to construct a spatial representation of it. Doing so is important for navigating and remaining oriented. How does one's ability to learn a new environment relate to one's ability to remember experiences in that environment? Here, 208 adults experienced a first-person videotaped route, and then completed a spatial map construction task. They also took tests of general cognitive abilities (working memory, laboratory episodic memory, processing speed, general knowledge) and of memory for familiar, everyday activities (event memory). Regression analyses revealed that event memory (memory for everyday events and their temporal structure), laboratory episodic memory (memory for words and pictures) and gender were unique predictors of spatial memory. These results implicate the processing of temporal structure and organization as an important cognitive ability in large-scale spatial-memory-from-route experience. Accounting for the temporal structure of people's experience while learning the layout of novel spaces may improve interventions for addressing navigation problems.

Precise temporal memories are supported by the lateral entorhinal cortex in humans

There is accumulating evidence that the entorhinal-hippocampal network is important for temporal memory. However, relatively little is known about the precise neurobiological mechanisms underlying memory for time. In particular, whether the lateral entorhinal cortex (LEC) is involved in temporal processing remains an open question. During high-resolution functional magnetic resonance imaging (fMRI) scanning, participants watched a ~28-min episode of a television show. During the test, they viewed still-frames and indicated on a continuous timeline the precise time each still-frame was viewed during the study. This procedure allowed us to measure error in seconds for each trial. We analyzed fMRI data from retrieval and found that high temporal precision was associated with increased blood-oxygen-level-dependent fMRI activity in the anterolateral entorhinal (a homolog of the LEC in rodents) and perirhinal cortices, but not in the posteromedial entorhinal and parahippocampal cortices. This suggests a previously unknown role for the LEC in processing of high-precision, minute-scale temporal memories.

Hierarchical cognitive control and the frontal lobes

Cognitive control refers to our ability to choose courses of thought and action that achieve our goals over habitual but contextually inappropriate ones. Hierarchical control problems are those in which multiple goals or contextual contingency must be managed at once and related to one another. In the open-ended complexity of the real world, hierarchical control arguably characterizes most of the problems faced by our control systems. And, it is these cases of hierarchical control where patients with damage to executive systems are most apt to fail, even those that perform well on simplified laboratory tasks. In this chapter, we consider the functional organization of frontal brain systems that support hierarchical cognitive control. We focus on two particular cases of hierarchical control. First, we discuss a line of work testing how managing multiple contingencies en route to a response relates to processing along the rostrocaudal axis of frontal cortex. Second, we consider cases of sequential tasks that require monitoring and behaving according to a series of tasks performed in time. In this latter case, we focus on the particular role of rostrolateral prefrontal cortex. We conclude with considerations of future directions of basic and clinically relevant research in this domain.

Verbal Paired Associates and the Hippocampus: The Role of Scenes

It is widely agreed that patients with bilateral hippocampal damage are impaired at binding pairs of words together. Consequently, the verbal paired associates (VPA) task has become emblematic of hippocampal function. This VPA deficit is not well understood and is particularly difficult for hippocampal theories with a visuospatial bias to explain (e.g., cognitive map and scene construction theories). Resolving the tension among hippocampal theories concerning the VPA could be important for leveraging a fuller understanding of hippocampal function. Notably, VPA tasks typically use high imagery concrete words and so conflate imagery and binding. To determine why VPA engages the hippocampus, we devised an fMRI encoding task involving closely matched pairs of scene words, pairs of object words, and pairs of very low imagery abstract words. We found that the anterior hippocampus was engaged during processing of both scene and object word pairs in comparison to abstract word pairs, despite binding occurring in all conditions. This was also the case when just subsequently remembered stimuli were considered. Moreover, for object word pairs, fMRI activity patterns in anterior hippocampus were more similar to those for scene imagery than object imagery. This was especially evident in participants who were high imagery users and not in mid and low imagery users. Overall, our results show that hippocampal engagement during VPA, even when object word pairs are involved, seems to be evoked by scene imagery rather than binding. This may help to resolve the issue that visuospatial hippocampal theories have in accounting for verbal memory.

Differentiation of Human Medial Prefrontal Cortex Activity Underlies Long-Term Resistance to Forgetting in Memory

It is well known that distributing study events over time leads to better memory over long time scales, compared with massing study events together. One explanation for such long-term resistance to forgetting is that distributed study leads to neural differentiation in memory, which supports retrieval of past experiences by disambiguating highly similar memory representations. Neuroanatomical models of episodic memory retrieval propose that the hippocampus and medial prefrontal cortex (MPFC) work together to enable retrieval of behaviorally appropriate memories. However, it is not known how representations in these regions jointly support resistance to forgetting long after initial learning. Using fMRI, we measured differentiation in retrieved memory representations following an extended delay in male and female human participants. After 1 week, word-object associations were better remembered if studied across 2 d (overnight), allowing associations to be learned in distinct temporal contexts, compared with learning within a single day (same day). MPFC retrieval patterns showed differentiation for overnight relative to same day memories, whereas hippocampal patterns reflected associative retrieval success. Overnight memory differentiation in MPFC was higher for associative than item memories and higher than differentiation assessed over a brain-wide set of retrieval-active voxels. The memory-related difference in MPFC pattern differentiation correlated with memory success for overnight learning and with hippocampal-MPFC functional connectivity. These results show that learning information across days leads to differentiated MPFC memory representations, reducing forgetting after 1 week, and suggest this arises from persistent interactions between MPFC and hippocampus.SIGNIFICANCE STATEMENT Neural activity in both the hippocampus and medial prefrontal cortex (MPFC) has been linked to memory-related representations, but prior work has not examined how these representations support episodic memory retrieval over extended time scales that are characteristic of everyday retrieval. We show that differentiation in MPFC activity 1 week after encoding is higher for retrieved information learned across 2 d compared with within a single day. In hippocampus, differentiation was greater for detailed memory retrieval but was not influenced by whether information had been learned over 1 or 2 d. Differentiation in MPFC predicted behavioral robustness to forgetting and was correlated with hippocampal-MPFC connectivity. The results suggest that context-based differentiation supports robust long-term memory via persistent MPFC-hippocampal interactions.

Decoding hierarchical control of sequential behavior in oscillatory EEG activity

Despite strong theoretical reasons for assuming that abstract representations organize complex action sequences in terms of subplans (chunks) and sequential positions, we lack methods to directly track such content-independent, hierarchical representations in humans. We applied time-resolved, multivariate decoding analysis to the pattern of rhythmic EEG activity that was registered while participants planned and executed individual elements from pre-learned, structured sequences. Across three experiments, the theta and alpha-band activity coded basic elements and abstract control representations, in particular, the ordinal position of basic elements, but also the identity and position of chunks. Further, a robust representation of higher level, chunk identity information was only found in individuals with above-median working memory capacity, potentially providing a neural-level explanation for working-memory differences in sequential performance. Our results suggest that by decoding oscillatory activity we can track how the cognitive system traverses through the states of a hierarchical control structure.

Hippocampal contributions to serial-order memory

Our memories form a record not only of our experiences, but also of their temporal structure. Although memory for the temporal structure of experience likely relies on multiple neural systems, numerous studies have implicated the hippocampus in the encoding and retrieval of temporal information. This review evaluates the literature on hippocampal contributions to human serial-order memory from the perspective of three cognitive theories: associative chaining theory, positional-coding theory and retrieved-context theory. Evaluating neural findings through the lens of cognitive theories enables us to draw more incisive conclusions about the relations between brain and behavior.

The Hippocampal Film Editor: Sensitivity and Specificity to Event Boundaries in Continuous Experience

The function of the human hippocampus is normally investigated by experimental manipulation of discrete events. Less is known about what triggers hippocampal activity during more naturalistic, continuous experience. We hypothesized that the hippocampus would be sensitive to the occurrence of event boundaries, that is, moments in time identified by observers as a transition between events. To address this, we analyzed functional MRI data from two groups: one (n = 253, 131 female) who viewed an 8.5 min film and another (n = 15, 6 female) who viewed a 120 min film. We observed a strong hippocampal response at boundaries defined by independent observers, which was modulated by boundary salience (the number of observers that identified each boundary). In the longer film, there were sufficient boundaries to show that this modulation remained after covarying out a large number of perceptual factors. This hypothesis-driven approach was complemented by a data-driven approach, in which we identified hippocampal events as moments in time with the strongest hippocampal activity. The correspondence between these hippocampal events and event boundaries was highly significant, revealing that the hippocampal response is not only sensitive, but also specific to event boundaries. We conclude that event boundaries play a key role in shaping hippocampal activity during encoding of naturalistic events.SIGNIFICANCE STATEMENT Recent years have seen the field of human neuroscience research transitioning from experiments with simple stimuli to the study of more complex and naturalistic experience. Nonetheless, our understanding of the function of many brain regions, such as the hippocampus, is based primarily on the study of brief, discrete events. As a result, we know little of what triggers hippocampal activity in real-life settings when we are exposed to a continuous stream of information. When does the hippocampus "decide" to respond during the encoding of naturalistic experience? We reveal here that hippocampal activity measured by fMRI during film watching is both sensitive and specific to event boundaries, identifying a potential mechanism whereby event boundaries shape experience by modulation of hippocampal activity.

Abstract Representation of Prospective Reward in the Hippocampus

Motivation enhances memory by increasing hippocampal engagement during encoding. However, whether such increased hippocampal activation reflects encoding of the value of highly rewarding events per se is less understood. Here, using a monetary incentive encoding task with a novel manipulation, we tested in humans whether the hippocampus represents abstract reward value, independent of perceptual content. During functional MRI scanning, men and women studied object pairs, each preceded by a monetary reward cue indicating the amount of money they would receive if they successfully remembered the object pair at test. Reward cues varied on both the level of reward (penny, dime, and dollar) and visual form (picture or word) across trials to dissociate hippocampal responses to reward value from those reflecting the perceptual properties of the cues. Behaviorally, participants remembered pairs associated with the high reward (dollar) more often than pairs associated with lower rewards. Neural pattern-similarity analysis revealed that hippocampal and parahippocampal cortex activation patterns discriminated between cues of different value regardless of their visual form, and that hippocampal discrimination of value was most pronounced in participants who showed the greatest behavioral sensitivity to reward. Strikingly, hippocampal patterns were most distinct for reward cues that differed in value but had similar visual appearance, consistent with theoretical proposals of hippocampal-pattern differentiation of competing representations. Our data illustrate how contextual representations within the hippocampus go beyond space and time to include information about the motivational salience of events, with hippocampal reward coding tracking the motivational impact on later memory.SIGNIFICANCE STATEMENT Motivation, such as the promise of future rewards, enhances hippocampal engagement during encoding and promotes successful retention of events associated with valuable rewards. However, whether the hippocampus explicitly encodes reward value, dissociable from sensory information, is unclear. Here, we show that the hippocampus forms abstract representation of valuable rewards, encoding conceptual rather than perceptual information about the motivational context of individual events. Reward representation within the hippocampus is associated with preferential retention of high-value events in memory. Furthermore, we show that hippocampal-pattern differentiation serves to emphasize differences between visually similar events with distinct motivational salience. Collectively, these findings indicate that hippocampal contextual representations enable individuals to distinguish the motivational value of events, leading to prioritized encoding of significant memories.

Space and Time: The Hippocampus as a Sequence Generator

Neural computations are often compared to instrument-measured distance or duration, and such relationships are interpreted by a human observer. However, neural circuits do not depend on human-made instruments but perform computations relative to an internally defined rate-of-change. While neuronal correlations with external measures, such as distance or duration, can be observed in spike rates or other measures of neuronal activity, what matters for the brain is how such activity patterns are utilized by downstream neural observers. We suggest that hippocampal operations can be described by the sequential activity of neuronal assemblies and their internally defined rate of change without resorting to the concept of space or time.

Functional brain networks for learning predictive statistics

Making predictions about future events relies on interpreting streams of information that may initially appear incomprehensible. This skill relies on extracting regular patterns in space and time by mere exposure to the environment (i.e., without explicit feedback). Yet, we know little about the functional brain networks that mediate this type of statistical learning. Here, we test whether changes in the processing and connectivity of functional brain networks due to training relate to our ability to learn temporal regularities. By combining behavioral training and functional brain connectivity analysis, we demonstrate that individuals adapt to the environment's statistics as they change over time from simple repetition to probabilistic combinations. Further, we show that individual learning of temporal structures relates to decision strategy. Our fMRI results demonstrate that learning-dependent changes in fMRI activation within and functional connectivity between brain networks relate to individual variability in strategy. In particular, extracting the exact sequence statistics (i.e., matching) relates to changes in brain networks known to be involved in memory and stimulus-response associations, while selecting the most probable outcomes in a given context (i.e., maximizing) relates to changes in frontal and striatal networks. Thus, our findings provide evidence that dissociable brain networks mediate individual ability in learning behaviorally-relevant statistics.

The Hippocampus Generalizes across Memories that Share Item and Context Information

Episodic memory is known to rely on the hippocampus, but how the hippocampus organizes different episodes to permit their subsequent retrieval remains controversial. One major area of debate hinges on a discrepancy between two hypothesized roles of the hippocampus: differentiating between similar events to reduce interference and assigning similar representations to events that share overlapping items and contextual information. Here, we used multivariate analyses of activity patterns measured with fMRI to characterize how the hippocampus distinguishes between memories based on similarity at the level of items and/or context. Hippocampal activity patterns discriminated between events that shared either item or context information but generalized across events that shared similar item-context associations. The current findings provide evidence that, whereas the hippocampus can reduce mnemonic interference by separating events that generalize along a single attribute dimension, overlapping hippocampal codes may support memory for events with overlapping item-context relations. This lends new insights into the way the hippocampus may balance multiple mnemonic operations in adaptively guiding behavior.

Individual differences in valence bias: fMRI evidence of the initial negativity hypothesis

Facial expressions offer an ecologically valid model for examining individual differences in affective decision-making. They convey an emotional signal from a social agent and provide important predictive information about one's environment (presence of potential rewards or threats). Although some expressions provide clear predictive information (angry, happy), others (surprised) are ambiguous in that they predict both positive and negative outcomes. Thus, surprised faces can delineate an individual's valence bias, or the tendency to interpret ambiguity as positive or negative. Our initial negativity hypothesis suggests that the initial response to ambiguity is negative, and that positivity relies on emotion regulation. We tested this hypothesis by comparing brain activity during explicit emotion regulation (reappraisal) and while freely viewing facial expressions, and measuring the relationship between brain activity and valence bias. Brain regions recruited during reappraisal showed greater activity for surprise in individuals with an increasingly positive valence bias. Additionally, we linked amygdala activity with an initial negativity, revealing a pattern similarity in individuals with negative bias between viewing surprised faces and maintaining negativity. Finally, these individuals failed to show normal habituation to clear negativity. These results support the initial negativity hypothesis, and are consistent with emotion research in both children and adult populations.

Stability of medial entorhinal cortex representations over time

Distinct functional cell types in the medial entorhinal cortex (mEC) have been shown to represent different aspects of experiences. To further characterize mEC cell populations, we examined whether spatial representations of neurons in mEC superficial layers depended on the scale of the environment and changed over extended time periods. Accordingly, mEC cells were recorded while rats repeatedly foraged in a small or a large environment in sessions that were separated by time intervals from minutes to hours. Comparing between large and small environments, we found that the overall precision of grid and non-grid cell spatial maps was higher in smaller environments. When examining the stability of spatial firing patterns over time, differences and similarities were observed across cell types. Within-session stability was higher for grid cells than for non-grid cell populations. Despite differences in baseline stability between cell types, stability levels remained consistent over time between sessions, up to 1 hr. Even for sessions separated by 6 hrs, activity patterns of grid cells and of most non-grid cells lacked any systematic decrease in spatial similarity over time. However, a subset of ~15% of mEC non-grid cells recorded preferentially from layer III exhibited dramatic, time dependent changes in firing patterns across 6 hrs, reminiscent of previous characterizations of the hippocampal CA2 subregion. Collectively, our data suggest that mEC grid cell input to hippocampus in conjunction with many time invariant non-grid cells may aid in stabilizing hippocampal spatial maps, while a subset of time varying non-grid cells could provide complementary temporal information.

Theta oscillations promote temporal sequence learning

Many theoretical models suggest that neural oscillations play a role in learning or retrieval of temporal sequences, but the extent to which oscillations support sequence representation remains unclear. To address this question, we used scalp electroencephalography (EEG) to examine oscillatory activity over learning of different object sequences. Participants made semantic decisions on each object as they were presented in a continuous stream. For three "Consistent" sequences, the order of the objects was always fixed. Activity during Consistent sequences was compared to "Random" sequences that consisted of the same objects presented in a different order on each repetition. Over the course of learning, participants made faster semantic decisions to objects in Consistent, as compared to objects in Random sequences. Thus, participants were able to use sequence knowledge to predict upcoming items in Consistent sequences. EEG analyses revealed decreased oscillatory power in the theta (4-7 Hz) band at frontal sites following decisions about objects in Consistent sequences, as compared with objects in Random sequences. The theta power difference between Consistent and Random only emerged in the second half of the task, as participants were more effectively able to predict items in Consistent sequences. Moreover, we found increases in parieto-occipital alpha (10-13 Hz) and beta (14-28 Hz) power during the pre-response period for objects in Consistent sequences, relative to objects in Random sequences. Linear mixed effects modeling revealed that single trial theta oscillations were related to reaction time for future objects in a sequence, whereas beta and alpha oscillations were only predictive of reaction time on the current trial. These results indicate that theta and alpha/beta activity preferentially relate to future and current events, respectively. More generally our findings highlight the importance of band-specific neural oscillations in the learning of temporal order information.

Integrating time from experience in the lateral entorhinal cortex

The encoding of time and its binding to events are crucial for episodic memory, but how these processes are carried out in hippocampal-entorhinal circuits is unclear. Here we show in freely foraging rats that temporal information is robustly encoded across time scales from seconds to hours within the overall population state of the lateral entorhinal cortex. Similarly pronounced encoding of time was not present in the medial entorhinal cortex or in hippocampal areas CA3-CA1. When animals' experiences were constrained by behavioural tasks to become similar across repeated trials, the encoding of temporal flow across trials was reduced, whereas the encoding of time relative to the start of trials was improved. The findings suggest that populations of lateral entorhinal cortex neurons represent time inherently through the encoding of experience. This representation of episodic time may be integrated with spatial inputs from the medial entorhinal cortex in the hippocampus, allowing the hippocampus to store a unified representation of what, where and when.

What does the functional organization of cortico-hippocampal networks tell us about the functional organization of memory?

Historically, research on the cognitive processes that support human memory proceeded, to a large extent, independently of research on the neural basis of memory. Accumulating evidence from neuroimaging, however, has enabled the field to develop a broader and more integrative perspective. Here, we briefly outline how advances in cognitive neuroscience can potentially shed light on concepts and controversies in human memory research. We argue that research on the functional properties of cortico-hippocampal networks informs us about how memories might be organized in the brain, which, in turn, helps to reconcile seemingly disparate perspectives in cognitive psychology. Finally, we discuss several open questions and directions for future research.

What's in a context? Cautions, limitations, and potential paths forward

The purpose of memory is to guide current and future behavior based on previous experiences. Part of this process involves either discriminating between or generalizing across similar experiences that contain overlapping conditions (such as space, time, or internal state), which we often conceptualize as "contexts". In this review, we highlight major challenges facing the field as we attempt a neuroscience-based approach to the study of context and its impact on learning and memory. Here, we review some of the methodologies and approaches used to investigate context in both animals and humans, including the neurobiological mechanisms involved. Finally, we propose three tenets for operationalizing context in the experimental setting: 1) contexts must be stable over time along an experiential dimension; 2) contexts must be at least moderately complex in nature and their representations must be modifiable or adaptable, and 3) contexts must have some behavioral relevance (be it overt or incidental) so that its role can be measured.

Learning Naturalistic Temporal Structure in the Posterior Medial Network

The posterior medial network is at the apex of a temporal integration hierarchy in the brain, integrating information over many seconds of viewing intact, but not scrambled, movies. This has been interpreted as an effect of temporal structure. Such structure in movies depends on preexisting event schemas, but temporal structure can also arise de novo from learning. Here, we examined the relative role of schema-consistent temporal structure and arbitrary but consistent temporal structure on the human posterior medial network. We tested whether, with repeated viewing, the network becomes engaged by scrambled movies with temporal structure. Replicating prior studies, activity in posterior medial regions was immediately locked to stimulus structure upon exposure to intact, but not scrambled, movies. However, for temporally structured scrambled movies, functional coupling within the network increased across stimulus repetitions, rising to the level of intact movies. Thus, temporal structure is a key determinant of network dynamics and function in the posterior medial network.

Associative Prediction of Visual Shape in the Hippocampus

Perception can be cast as a process of inference, in which bottom-up signals are combined with top-down predictions in sensory systems. In line with this, neural activity in sensory cortex is strongly modulated by prior expectations. Such top-down predictions often arise from cross-modal associations, such as when a sound (e.g., bell or bark) leads to an expectation of the visual appearance of the corresponding object (e.g., bicycle or dog). We hypothesized that the hippocampus, which rapidly learns arbitrary relationships between stimuli over space and time, may be involved in forming such associative predictions. We exposed male and female human participants to auditory cues predicting visual shapes, while measuring high-resolution fMRI signals in visual cortex and the hippocampus. Using multivariate reconstruction methods, we discovered a dissociation between these regions: representations in visual cortex were dominated by whichever shape was presented, whereas representations in the hippocampus reflected only which shape was predicted by the cue. The strength of hippocampal predictions correlated across participants with the amount of expectation-related facilitation in visual cortex. These findings help bridge the gap between memory and sensory systems in the human brain.SIGNIFICANCE STATEMENT The way we perceive the world is to a great extent determined by our prior knowledge. Despite this intimate link between perception and memory, these two aspects of cognition have mostly been studied in isolation. Here we investigate their interaction by asking how memory systems that encode and retrieve associations can inform perception. We find that upon hearing a familiar auditory cue, the hippocampus represents visual information that had previously co-occurred with the cue, even when this expectation differs from what is currently visible. Furthermore, the strength of this hippocampal expectation correlates with facilitation of perceptual processing in visual cortex. These findings help bridge the gap between memory and sensory systems in the human brain.

Nonmonotonic recruitment of ventromedial prefrontal cortex during remote memory recall

Systems-level consolidation refers to the time-dependent reorganisation of memory traces in the neocortex, a process in which the ventromedial prefrontal cortex (vmPFC) has been implicated. Capturing the precise temporal evolution of this crucial process in humans has long proved elusive. Here, we used multivariate methods and a longitudinal functional magnetic resonance imaging (fMRI) design to detect, with high granularity, the extent to which autobiographical memories of different ages were represented in vmPFC and how this changed over time. We observed an unexpected time course of vmPFC recruitment during retrieval, rising and falling around an initial peak of 8–12 months, before reengaging for older 2- and 5-year-old memories. This pattern was replicated in 2 independent sets of memories. Moreover, it was further replicated in a follow-up study 8 months later with the same participants and memories, for which the individual memory representations had undergone their hypothesised strengthening or weakening over time. We conclude that the temporal engagement of vmPFC in memory retrieval seems to be nonmonotonic, revealing a complex relationship between systems-level consolidation and prefrontal cortex recruitment that is unaccounted for by current theories.

Our past experiences are captured in autobiographical memories that allow us to recollect events from our lives long after they originally occurred. A part of the brain’s frontal lobe, called the ventromedial prefrontal cortex (vmPFC), is known to be important for supporting autobiographical memories, especially as memories become more remote. The precise temporal profile of the vmPFC’s involvement is unclear, yet this information is vital if we are to understand how memories change over time and the mechanisms involved. In this study, we sought to establish the time course of vmPFC engagement in the recollection of autobiographical memories while participants recalled memories of different ages during functional magnetic resonance imaging (fMRI). Using a method that detects the brain activity patterns associated with individual memories, we found that memory-specific neural patterns in vmPFC became more distinct over the first few months after a memory was formed, but then this initial involvement of vmPFC subsided after 1 year. However, more remote memories (2 years and older) appeared to reengage vmPFC once again. This temporal profile is difficult to accommodate within any single existing theory. Consequently, our results provoke a rethink about how memories evolve over time and the role played by the vmPFC.

Human midcingulate cortex encodes distributed representations of task progress

The function of midcingulate cortex (MCC) remains elusive despite decades of investigation and debate. Complicating matters, individual MCC neurons respond to highly diverse task-related events, and MCC activation is reported in most human neuroimaging studies employing a wide variety of task manipulations. Here we investigate this issue by applying a model-based cognitive neuroscience approach involving neural network simulations, functional magnetic resonance imaging, and representational similarity analysis. We demonstrate that human MCC encodes distributed, dynamically evolving representations of extended, goal-directed action sequences. These representations are uniquely sensitive to the stage and identity of each sequence, indicating that MCC sustains contextual information necessary for discriminating between task states. These results suggest that standard univariate approaches for analyzing MCC function overlook the major portion of task-related information encoded by this brain area and point to promising new avenues for investigation.

Multivoxel pattern similarity suggests the integration of temporal duration in hippocampal event sequence representations

Recent rodent work suggests the hippocampus may provide a temporal representation of event sequences, in which the order of events and the interval durations between them are encoded. There is, however, limited human evidence for the latter, in particular whether the hippocampus processes duration information pertaining to the passage of time rather than qualitative or quantitative changes in event content. We scanned participants while they made match-mismatch judgements on each trial between a study sequence of events and a subsequent test sequence. Participants explicitly remembered event order or interval duration information (Experiment 1), or monitored order only, with duration being manipulated implicitly (Experiment 2). Hippocampal study-test pattern similarity was significantly reduced by changes to order or duration in mismatch trials, even when duration was processed implicitly. Our findings suggest the human hippocampus processes short intervals within sequences and support the idea that duration information is integrated into hippocampal mnemonic representations.

Still searching for the engram

For nearly a century, neurobiologists have searched for the engram-the neural representation of a memory. Early studies showed that the engram is widely distributed both within and across brain areas and is supported by interactions among large networks of neurons. Subsequent research has identified engrams that support memory within dedicated functional systems for habit learning and emotional memory, but the engram for declarative memories has been elusive. Nevertheless, recent years have brought progress from molecular biological approaches that identify neurons and networks that are necessary and sufficient to support memory, and from recording approaches and population analyses that characterize the information coded by large neural networks. These new directions offer the promise of revealing the engrams for episodic and semantic memories.

Hippocampus, Retrosplenial and Parahippocampal Cortices Encode Multicompartment 3D Space in a Hierarchical Manner

Humans commonly operate within 3D environments such as multifloor buildings and yet there is a surprising dearth of studies that have examined how these spaces are represented in the brain. Here, we had participants learn the locations of paintings within a virtual multilevel gallery building and then used behavioral tests and fMRI repetition suppression analyses to investigate how this 3D multicompartment space was represented, and whether there was a bias in encoding vertical and horizontal information. We found faster response times for within-room egocentric spatial judgments and behavioral priming effects of visiting the same room, providing evidence for a compartmentalized representation of space. At the neural level, we observed a hierarchical encoding of 3D spatial information, with left anterior hippocampus representing local information within a room, while retrosplenial cortex, parahippocampal cortex, and posterior hippocampus represented room information within the wider building. Of note, both our behavioral and neural findings showed that vertical and horizontal location information was similarly encoded, suggesting an isotropic representation of 3D space even in the context of a multicompartment environment. These findings provide much-needed information about how the human brain supports spatial memory and navigation in buildings with numerous levels and rooms.

White-Matter Pathways for Statistical Learning of Temporal Structures

Extracting the statistics of event streams in natural environments is critical for interpreting current events and predicting future ones. The brain is known to rapidly find structure and meaning in unfamiliar streams of sensory experience, often by mere exposure to the environment (i.e., without explicit feedback). Yet, we know little about the brain pathways that support this type of statistical learning. Here, we test whether changes in white-matter (WM) connectivity due to training relate to our ability to extract temporal regularities. By combining behavioral training and diffusion tensor imaging (DTI), we demonstrate that humans adapt to the environment's statistics as they change over time from simple repetition to probabilistic combinations. In particular, we show that learning relates to the decision strategy that individuals adopt when extracting temporal statistics. We next test for learning-dependent changes in WM connectivity and ask whether they relate to individual variability in decision strategy. Our DTI results provide evidence for dissociable WM pathways that relate to individual strategy: extracting the exact sequence statistics (i.e., matching) relates to connectivity changes between caudate and hippocampus, while selecting the most probable outcomes in a given context (i.e., maximizing) relates to connectivity changes between prefrontal, cingulate and basal ganglia (caudate, putamen) regions. Thus, our findings provide evidence for distinct cortico-striatal circuits that show learning-dependent changes of WM connectivity and support individual ability to learn behaviorally-relevant statistics.

Human Episodic Memory Retrieval Is Accompanied by a Neural Contiguity Effect

Cognitive psychologists have long hypothesized that experiences are encoded in a temporal context that changes gradually over time. When an episodic memory is retrieved, the state of context is recovered-a jump back in time. We recorded from single units in the medial temporal lobe of epilepsy patients performing an item recognition task. The population vector changed gradually over minutes during presentation of the list. When a probe from the list was remembered with high confidence, the population vector reinstated the temporal context of the original presentation of that probe during study, a neural contiguity effect that provides a possible mechanism for behavioral contiguity effects. This pattern was only observed for well remembered probes; old probes that were not well remembered showed an anti-contiguity effect. These results constitute the first direct evidence that recovery of an episodic memory in humans is associated with retrieval of a gradually changing state of temporal context, a neural "jump back in time" that parallels the act of remembering.SIGNIFICANCE STATEMENT Episodic memory is the ability to relive a specific experience from one's life. For decades, researchers have hypothesized that, unlike other forms of memory that can be described as simple associations between stimuli, episodic memory depends on the recovery of a neural representation of spatiotemporal context. During study of a sequence of stimuli, the brain state of epilepsy patients changed slowly over at least a minute. When the participant remembered a particular event from the list, this gradually changing state was recovered. This provides direct confirmation of the prediction from computational models of episodic memory. The resolution of this point means that the study of episodic memory can focus on the mechanisms by which this representation of spatiotemporal context is maintained and sometimes recovered.

Mixed selectivity encoding and action selection in the prefrontal cortex during threat assessment

The medial prefrontal cortex (mPFC) regulates expression of emotional behavior. The mPFC combines multivariate information from its inputs, and depending on the imminence of threat, activates downstream networks that either increase or decrease the expression of anxiety-related motor behavior and autonomic activation. Here, we selectively highlight how subcortical input to the mPFC from two example structures, the amygdala and ventral hippocampus, help shape mixed selectivity encoding and action selection during emotional processing. We outline a model where prefrontal subregions modulate behavior along orthogonal motor dimensions, and exhibit connectivity that selects for expression of one behavioral strategy while inhibiting the other.

Hippocampus and Prefrontal Cortex Predict Distinct Timescales of Activation in the Human Ventral Tegmental Area

The mesolimbic dopamine system contributes to a remarkable variety of behaviors at multiple timescales. Midbrain neurons have fast and slow signaling components, and specific afferent systems, such as the hippocampus (HPC) and prefrontal cortex (PFC), have been demonstrated to drive these components in anesthetized animals. Whether these interactions exist during behavior, however, is unknown. To address this question, we developed a novel analysis of human functional magnetic resonance imaging data that fits models of network excitation and inhibition on ventral tegmental area (VTA) activation. We show that specific afferent systems predict distinct temporal components of midbrain VTA signal. We found that PFC, but not HPC, positively predicted transient, event-evoked VTA activation. In contrast, HPC, but not PFC, positively predicted slow shifts in VTA baseline variability. Thus, unique functional contributions of afferent systems to VTA physiology are detectable at the network level in behaving humans. The findings support models of dopamine function in which dissociable neural circuits support different aspects of motivated behavior via active regulation of tonic and phasic signals.

Spatial scaffold effects in event memory and imagination

Spatial context is a defining feature of episodic memories, which are often characterized as being events occurring in specific spatiotemporal contexts. In this review, I summarize research suggesting a common neural basis for episodic and spatial memory and relate this to the role of spatial context in episodic memory. I review evidence that spatial context serves as a scaffold for episodic memory and imagination, in terms of both behavioral and neural effects demonstrating a dependence of episodic memory on spatial representations. These effects are mediated by a posterior-medial set of neocortical regions, including the parahippocampal cortex, retrosplenial cortex, posterior cingulate cortex, precuneus, and angular gyrus, which interact with the hippocampus to represent spatial context in remembered and imagined events. I highlight questions and areas that require further research, including differentiation of hippocampal function along its long axis and subfields, and how these areas interact with the posterior-medial network. This article is categorized under: Psychology > Memory Neuroscience > Cognition.

Memory allocation mechanisms underlie memory linking across time

Memories are dynamic in nature. A cohesive representation of the world requires memories to be altered over time, linked with other memories and eventually integrated into a larger framework of sematic knowledge. Although there is a considerable literature on how single memories are encoded, retrieved and updated, little is known about the mechanisms that govern memory linking, e.g., linking and integration of various memories across hours or days. In this review, we present evidence that specific memory allocation mechanisms, such as changes in CREB and intrinsic excitability, ensure memory storage in ways that facilitate effective recall and linking at a later time. Beyond CREB and intrinsic excitability, we also review a number of other phenomena with potential roles in memory linking.

The Primacy of Spatial Context in the Neural Representation of Events

Some theories of episodic memory hypothesize that spatial context plays a fundamental role in episodic memory, acting as a scaffold on which episodes are constructed. A prediction based on this hypothesis is that spatial context should play a primary role in the neural representation of an event. To test this hypothesis in humans, male and female participants imagined events, composed of familiar locations, people, and objects, during an fMRI scan. We used multivoxel pattern analysis to determine the neural areas in which events could be discriminated based on each feature. We found that events could be discriminated according to their location in areas throughout the autobiographical memory network, including the parahippocampal cortex and posterior hippocampus, retrosplenial cortex, posterior cingulate cortex, precuneus, and medial prefrontal cortex. Events were also discriminable based on person and object features, but in fewer regions. Comparing classifier performance in regions involved in memory for scenes and events demonstrated that the location of an event was more accurately classified than the person or object involved. These results support theories that suggest that spatial context is a prominent defining feature of episodic memory.SIGNIFICANCE STATEMENT Remembered and imagined events are complex, consisting of many elements, including people, objects, and locations. In this study, we sought to determine how these types of elements differentially contribute to how the brain represents an event. Participants imagined events consisting of familiar locations, people, and objects (e.g., kitchen, mom, umbrella) while their brain activity was recorded with fMRI. We found that the neural patterns of activity in brain regions associated with spatial and episodic memory could distinguish events based on their location, and to some extent, based on the people and objects involved. These results suggest that the spatial context of an event plays an important role in how an event is represented in the brain.

Constructing Experience: Event Models from Perception to Action

Mental representations of everyday experience are rich, structured, and multimodal. In this article we consider the adaptive pressures that led to human construction of such representations, arguing that structured event representations enable cognitive systems to more effectively predict the trajectory of naturalistic everyday activity. We propose an account of how cortical systems and the hippocampus (HPC) interact to construct, maintain, and update event representations. This analysis throws light on recent research on story comprehension, event segmentation, episodic memory, and action planning. It also suggests how the growing science base can be deployed to diagnose impairments in event perception and memory, and to improve memory for everyday events.

Event Boundaries Trigger Rapid Memory Reinstatement of the Prior Events to Promote Their Representation in Long-Term Memory

Although everyday experiences unfold continuously over time, shifts in context, or event boundaries, can influence how those events come to be represented in memory [1-4]. Specifically, mnemonic binding across sequential representations is more challenging at context shifts, such that successful temporal associations are more likely to be formed within than across contexts [1, 2, 5-9]. However, in order to preserve a subjective sense of continuity, it is important that the memory system bridge temporally adjacent events, even if they occur in seemingly distinct contexts. Here, we used pattern similarity analysis to scalp electroencephalographic (EEG) recordings during a sequential learning task [2, 3] in humans and showed that the detection of event boundaries triggered a rapid memory reinstatement of the just-encoded sequence episode. Memory reactivation was detected rapidly (∼200-800 ms from the onset of the event boundary) and was specific to context shifts that were preceded by an event sequence with episodic content. Memory reinstatement was not observed during the sequential encoding of events within an episode, indicating that memory reactivation was induced specifically upon context shifts. Finally, the degree of neural similarity between neural responses elicited during sequence encoding and at event boundaries correlated positively with participants' ability to later link across sequences of events, suggesting a critical role in binding temporally adjacent events in long-term memory. Current results shed light onto the neural mechanisms that promote episodic encoding not only for information within the event, but also, importantly, in the ability to link across events to create a memory representation of continuous experience.

Lost in Time and Space: States of High Arousal Disrupt Implicit Acquisition of Spatial and Sequential Context Information

Biased cognition during high arousal states is a relevant phenomenon in a variety of topics: from the development of post-traumatic stress disorders or stress-triggered addictive behaviors to forensic considerations regarding crimes of passion. Recent evidence indicates that arousal modulates the engagement of a hippocampus-based "cognitive" system in favor of a striatum-based "habit" system in learning and memory, promoting a switch from flexible, contextualized to more rigid, reflexive responses. Existing findings appear inconsistent, therefore it is unclear whether and which type of context processing is disrupted by enhanced arousal. In this behavioral study, we investigated such arousal-triggered cognitive-state shifts in human subjects. We validated an arousal induction procedure (three experimental conditions: violent scene, erotic scene, neutral control scene) using pupillometry (Preliminary Experiment, n = 13) and randomly administered this method to healthy young adults to examine whether high arousal states affect performance in two core domains of contextual processing, the acquisition of spatial (spatial discrimination paradigm; Experiment 1, n = 66) and sequence information (learned irrelevance paradigm; Experiment 2, n = 84). In both paradigms, spatial location and sequences were encoded incidentally and both displacements when retrieving spatial position as well as the predictability of the target by a cue in sequence learning changed stepwise. Results showed that both implicit spatial and sequence learning were disrupted during high arousal states, regardless of valence. Compared to the control group, participants in the arousal conditions showed impaired discrimination of spatial positions and abolished learning of associative sequences. Furthermore, Bayesian analyses revealed evidence against the null models. In line with recent models of stress effects on cognition, both experiments provide evidence for decreased engagement of flexible, cognitive systems supporting encoding of context information in active cognition during acute arousal, promoting reduced sensitivity for contextual details. We argue that arousal fosters cognitive adaptation towards less demanding, more present-oriented information processing, which prioritizes a current behavioral response set at the cost of contextual cues. This transient state of behavioral perseverance might reduce reliance on context information in unpredictable environments and thus represent an adaptive response in certain situations.

Dendrites, deep learning, and sequences in the hippocampus

The hippocampus places us both in time and space. It does so over remarkably large spans: milliseconds to years, and centimeters to kilometers. This works for sensory representations, for memory, and for behavioral context. How does it fit in such wide ranges of time and space scales, and keep order among the many dimensions of stimulus context? A key organizing principle for a wide sweep of scales and stimulus dimensions is that of order in time, or sequences. Sequences of neuronal activity are ubiquitous in sensory processing, in motor control, in planning actions, and in memory. Against this strong evidence for the phenomenon, there are currently more models than definite experiments about how the brain generates ordered activity. The flip side of sequence generation is discrimination. Discrimination of sequences has been extensively studied at the behavioral, systems, and modeling level, but again physiological mechanisms are fewer. It is against this backdrop that I discuss two recent developments in neural sequence computation, that at face value share little beyond the label "neural." These are dendritic sequence discrimination, and deep learning. One derives from channel physiology and molecular signaling, the other from applied neural network theory - apparently extreme ends of the spectrum of neural circuit detail. I suggest that each of these topics has deep lessons about the possible mechanisms, scales, and capabilities of hippocampal sequence computation.

Viewpoints: how the hippocampus contributes to memory, navigation and cognition

The hippocampus serves a critical function in memory, navigation, and cognition. Nature Neuroscience asked John Lisman to lead a group of researchers in a dialog on shared and distinct viewpoints on the hippocampus.

There has been a long history of studying the hippocampus, but recent work has made it possible to study the cellular and network basis of defined operations—operations that include cognitive processes that have been otherwise difficult to study (see Box 1 for useful terminology). These operations deal with the context-dependent representation of complex memories, the role of mental exploration based on imagined rather than real movements, and the use of recalled information for navigation and decision-making. The progress that has been made in understanding the hippocampus has motivated the study of other brain regions that provide hippocampal input or receive hippocampal output; the hippocampus is thus serving as a nucleating point for the larger goal of understanding the neural codes that allow inter-regional communication and more generally, understanding how memory-guided behavior is achieved by large scale integration of brain regions. In generating a discussion among experts in the study of the cognitive processes of the hippocampus, the editors and I have posed questions that probe important principles of hippocampal function. We hope that the resulting discussion will make clear to readers the progress that has been made, while also identifying issues where consensus has not yet been achieved and that should be pursued in future research. – John Lisman

The Ebb and Flow of Experience Determines the Temporal Structure of Memory

Everyday life consists of a continuous stream of information, yet somehow we remember the past as distinct episodic events. Prominent models posit that event segmentation is driven by erroneous predictions about how current experiences are unfolding. Yet this perspective fails to explain how memories become integrated or separated in the absence of prior knowledge. Here, we propose that contextual stability dictates the temporal organization of events in episodic memory. To support this view, we summarize new findings showing that neural measures of event organization index how ongoing changes in external contextual cues and internal representations of time influence different forms of episodic memory.

Time Regained: How the Human Brain Constructs Memory for Time

Life's episodes unfold against a context that changes with time. Recent neuroimaging studies have revealed significant findings about how specific areas of the human brain may support the representation of temporal information in memory. A consistent theme in these studies is that the hippocampus appears to play a central role in representing temporal context, as operationalized in neuroimaging studies of arbitrary lists of items, sequences of items, or meaningful, lifelike events. Additionally, activity in a posterior medial cortical network may reflect the representation of generalized temporal information for meaningful events. The hippocampus, posterior medial network, and other regions-particularly in prefrontal cortex-appear to play complementary roles in memory for temporal context.

Temporal and spatial context in the mind and brain

Theories of episodic memory have long hypothesized that recollection of a specific instance from one's life is mediated by recovery of a neural state of spatiotemporal context. This paper reviews recent theoretical advances in formal models of spatiotemporal context and a growing body of neurophysiological evidence from human imaging studies and animal work that neural populations in the hippocampus and other brain regions support a representation of spatiotemporal context.

Memory integration constructs maps of space, time, and concepts

Recent evidence demonstrates that new events are learned in the context of their relationships to existing memories. Within the hippocampus and medial prefrontal cortex, related memories are represented by integrated codes that connect events experienced at different times and places. Integrated codes form the basis of spatial, temporal, and conceptual maps of experience. These maps represent information that goes beyond direct experience and support generalization behaviors that require knowledge be used in new ways. The degree to which an individual memory is integrated into a coherent map is determined by its spatial, temporal, and conceptual proximity to existing knowledge. Integration is observed over a wide range of scales, suggesting that memories contain information about both broad and fine-grained contexts.

The Persistence of Memory: How the Brain Encodes Time in Memory

Time and memory are inextricably linked, but it is far from clear how event durations and temporal sequences are encoded in memory. In this review, we focus on resource allocation models of working memory which suggest that memory resources can be flexibly distributed amongst several items such that the precision of working memory decreases with the number of items to be encoded. This type of model is consistent with human performance in working memory tasks based on visual, auditory as well as temporal stimulus patterns. At the neural-network level, we focus on excitatory-inhibitory oscillatary processes that are able to encode both interval timing and working memory in a coupled excitatory-inhibitory network. This modification of the striatal beat-frequency model of interval timing shows how memories for multiple time intervals are represented by neural oscillations and can also be used to explain the mechanisms of resource allocation in working memory.

Short-term memory for spatial, sequential and duration information

•

Analog report methods provide novel insights on STM for space and time.

•

Space and time may be used to bind features in STM.

•

The hippocampus is involved in object-location binding in STM.

Space and time appear to play key roles in the way that information is organized in short-term memory (STM). Some argue that they are crucial contexts within which other stored features are embedded, allowing binding of information that belongs together within STM. Here we review recent behavioral, neurophysiological and imaging studies that have sought to investigate the nature of spatial, sequential and duration representations in STM, and how these might break down in disease. Findings from these studies point to an important role of the hippocampus and other medial temporal lobe structures in aspects of STM, challenging conventional accounts of involvement of these regions in only long-term memory.