Hippocampal activity patterns carry information about objects in temporal context

The influence of emotion on temporal context models

Temporal context models (TCMs) have been influential in understanding episodic memory and its neural underpinnings. Recently, TCMs have been extended to explain emotional memory effects, one of the most clinically important findings in the field of memory research. This review covers recent advances in hypotheses for the neural representation of spatiotemporal context through the lens of TCMs, including their ability to explain the influence of emotion on episodic and temporal memory. In recent years, simplifying assumptions of "classical" TCMs - with exponential trace decay and the mechanism by which temporal context is recovered - have become increasingly clear. The review also outlines how recent advances could be incorporated into a future TCM, beyond classical assumptions, to integrate emotional modulation.

Goal-directed attention transforms both working and long-term memory representations in the human parietal cortex

The abundance of distractors in the world poses a major challenge to our brain's limited processing capacity, but little is known about how selective attention modulates stimulus representations in the brain to reduce interference and support durable target memory. Here, we collected functional magnetic resonance imaging (fMRI) data in a selective attention task in which target and distractor pictures of different visual categories were simultaneously presented. Participants were asked to selectively process the target according to the effective cue, either before the encoding period (i.e., perceptual attention) or the maintenance period (i.e., reflective attention). On the next day, participants were asked to perform a memory recognition task in the scanner in which the targets, distractors, and novel items were presented in a pseudorandom order. Behavioral results showed that perceptual attention was better at enhancing target memory and reducing distractor memory than reflective attention, although the overall memory capacity (memory for both target and distractor) was comparable. Using multiple-voxel pattern analysis of the neural data, we found more robust target representation and weaker distractor representation in working memory for perceptual attention than for reflective attention. Interestingly, perceptual attention partially shifted the regions involved in maintaining the target representation from the visual cortex to the parietal cortex. Furthermore, the targets and distractors simultaneously presented in the perceptual attention condition showed reduced pattern similarity in the parietal cortex during retrieval compared to items not presented together. This neural pattern repulsion positively correlated with individuals' recognition of both targets and distractors. These results emphasize the critical role of selective attention in transforming memory representations to reduce interference and improve long-term memory performance.

Emotional state dynamics impacts temporal memory

Emotional fluctuations are ubiquitous in everyday life, but precisely how they sculpt the temporal organisation of memories remains unclear. Here, we designed a novel task - the Emotion Boundary Task - wherein participants viewed sequences of negative and neutral images surrounded by a colour border. We manipulated perceptual context (border colour), emotional-picture valence, as well as the direction of emotional-valence shifts (i.e., shifts from neutral-to-negative and negative-to-neutral events) to create events with a shared perceptual and/or emotional context. We measured memory for temporal order and temporal distances for images processed within and across events. Negative images processed within events were remembered as closer in time compared to neutral ones. In contrast, temporal distances were remembered as longer for images spanning neutral-to-negative shifts - suggesting temporal dilation in memory with the onset of a negative event following a previously-neutral state. The extent of negative-picture induced temporal dilation in memory correlated with dispositional negativity across individuals. Lastly, temporal order memory was enhanced for recently-presented negative (versus neutral) images. These findings suggest that emotional-state dynamics matters when considering emotion-temporal memory interactions: While persistent negative events may compress subjectively remembered time, dynamic shifts from neutral-to-negative events produce temporal dilation in memory, with implications for adaptive emotional functioning.

Encoding-related Brain Activity Predicts Subsequent Trial-level Control of Proactive Interference in Working Memory

Proactive interference (PI) appears when familiar information interferes with newly acquired information and is a major cause of forgetting in working memory. It has been proposed that encoding of item-context associations might help mitigate familiarity-based PI. Here, we investigate whether encoding-related brain activation could predict subsequent level of PI at retrieval using trial-specific parametric modulation. Participants were scanned with event-related fMRI while performing a 2-back working memory task with embedded 3-back lures designed to induce PI. We found that the ability to control interference in working memory was modulated by level of activation in the left inferior frontal gyrus, left hippocampus, and bilateral caudate nucleus during encoding. These results provide insight to the processes underlying control of PI in working memory and suggest that encoding of temporal context details support subsequent interference control.

Episodic boundaries affect neural features of representational drift in humans

A core feature of episodic memory is representational drift, the gradual change in aggregate oscillatory features that supports temporal association of memory items. However, models of drift overlook the role of episodic boundaries, which indicate a shift from prior to current context states. Our study focuses on the impact of task boundaries on representational drift in the parietal and temporal lobes in 99 subjects during a free recall task. Using intracranial EEG recordings, we show boundary representations reset gamma band drift in the medial parietal lobe, selectively enhancing the recall of early list (primacy) items. Conversely, the lateral temporal cortex shows increased drift for recalled items but lacked sensitivity to task boundaries. Our results suggest regional sensitivity to varied contextual features: the lateral temporal cortex uses drift to differentiate items, while the medial parietal lobe uses drift-resets to associate items with the current context. We propose drift represents relational information tailored to a region's sensitivity to unique contextual elements. Our findings offer a mechanism to integrate models of temporal association by drift with event segmentation by episodic boundaries.

Event-related theta and gamma band oscillatory dynamics during visuo-spatial sequence memory in younger and older adults

Visuo-spatial working memory (VSWM) for sequences is thought to be crucial for daily behaviors. Decades of research indicate that oscillations in the gamma and theta bands play important functional roles in the support of visuo-spatial working memory, but the vast majority of that research emphasizes measures of neural activity during memory retention. The primary aims of the present study were (1) to determine whether oscillatory dynamics in the Theta and Gamma ranges would reflect item-level sequence encoding during a computerized spatial span task, (2) to determine whether item-level sequence recall is also related to these neural oscillations, and (3) to determine the nature of potential changes to these processes in healthy cognitive aging. Results indicate that VSWM sequence encoding is related to later (∼700 ms) gamma band oscillatory dynamics and may be preserved in healthy older adults; high gamma power over midline frontal and posterior sites increased monotonically as items were added to the spatial sequence in both age groups. Item-level oscillatory dynamics during the recall of VSWM sequences were related only to theta-gamma phase amplitude coupling (PAC), which increased monotonically with serial position in both age groups. Results suggest that, despite a general decrease in frontal theta power during VSWM sequence recall in older adults, gamma band dynamics during encoding and theta-gamma PAC during retrieval play unique roles in VSWM and that the processes they reflect may be spared in healthy aging.

Toward an integrative account of internal and external determinants of event segmentation

Our daily experiences unfold continuously, but we remember them as a series of discrete events through a process called event segmentation. Prominent theories of event segmentation suggest that event boundaries in memory are triggered by significant shifts in the external environment, such as a change in one's physical surroundings. In this review, we argue for a fundamental extension of this research field to also encompass internal state changes as playing a key role in structuring event memory. Accordingly, we propose an expanded taxonomy of event boundary-triggering processes, and review behavioral and neuroscience research on internal state changes in three core domains: affective states, goal states, and motivational states. Finally, we evaluate how well current theoretical frameworks can accommodate the unique and interactive contributions of internal states to event memory. We conclude that a theoretical perspective on event memory that integrates both external environment and internal state changes allows for a more complete understanding of how the brain structures experiences, with important implications for future research in cognitive and clinical neuroscience.

Beyond the ears: A review exploring the interconnected brain behind the hierarchical memory of music

Music is a ubiquitous element of daily life. Understanding how music memory is represented and expressed in the brain is key to understanding how music can influence human daily cognitive tasks. Current music-memory literature is built on data from very heterogeneous tasks for measuring memory, and the neural correlates appear to differ depending on different forms of memory function targeted. Such heterogeneity leaves many exceptions and conflicts in the data underexplained (e.g., hippocampal involvement in music memory is debated). This review provides an overview of existing neuroimaging results from music-memory related studies and concludes that although music is a special class of event in our lives, the memory systems behind it do in fact share neural mechanisms with memories from other modalities. We suggest that dividing music memory into different levels of a hierarchy (structural level and semantic level) helps understand overlap and divergence in neural networks involved. This is grounded in the fact that memorizing a piece of music recruits brain clusters that separately support functions including-but not limited to-syntax storage and retrieval, temporal processing, prediction versus reality comparison, stimulus feature integration, personal memory associations, and emotion perception. The cross-talk between frontal-parietal music structural processing centers and the subcortical emotion and context encoding areas explains why music is not only so easily memorable but can also serve as strong contextual information for encoding and retrieving nonmusic information in our lives.

Emotional state dynamics impacts temporal memory

Emotional fluctuations are ubiquitous in everyday life, but precisely how they sculpt the temporal organization of memories remains unclear. Here, we designed a novel task-the Emotion Boundary Task-wherein participants viewed sequences of negative and neutral images surrounded by a color border. We manipulated perceptual context (border color), emotional valence, as well as the direction of emotional-valence shifts (i.e., shifts from neutral-to-negative and negative-to-neutral events) to create encoding events comprised of image sequences with a shared perceptual and/or emotional context. We measured memory for temporal order and subjectively remembered temporal distances for images processed within and across events. Negative images processed within events were remembered as closer in time compared to neutral ones. In contrast, temporal distance was remembered as longer for images spanning neutral-to-negative shifts-suggesting temporal dilation in memory with the onset of a negative event following a previously-neutral state. The extent of this negative-picture induced temporal dilation in memory correlated with dispositional negativity across individuals. Lastly, temporal order memory was enhanced for recently presented negative (compared to neutral) images. These findings suggest that emotional-state dynamics matters when considering emotion-temporal memory interactions: While persistent negative events may compress subjectively remembered time, dynamic shifts from neutral to negative events produce temporal dilation in memory, which may be relevant for adaptive emotional functioning.

Event Integration and Temporal Differentiation: How Hierarchical Knowledge Emerges in Hippocampal Subfields through Learning

Everyday life is composed of events organized by changes in contexts, with each event containing an unfolding sequence of occurrences. A major challenge facing our memory systems is how to integrate sequential occurrences within events while also maintaining their details and avoiding over-integration across different contexts. We asked if and how distinct hippocampal subfields come to hierarchically and, in parallel, represent both event context and subevent occurrences with learning. Female and male human participants viewed sequential events defined as sequences of objects superimposed on shared color frames while undergoing high-resolution fMRI. Importantly, these events were repeated to induce learning. Event segmentation, as indexed by increased reaction times at event boundaries, was observed in all repetitions. Temporal memory decisions were quicker for items from the same event compared to across different events, indicating that events shaped memory. With learning, hippocampal CA3 multivoxel activation patterns clustered to reflect the event context, with more clustering correlated with behavioral facilitation during event transitions. In contrast, in the dentate gyrus (DG), temporally proximal items that belonged to the same event became associated with more differentiated neural patterns. A computational model explained these results by dynamic inhibition in the DG. Additional similarity measures support the notion that CA3 clustered representations reflect shared voxel populations, while DG's distinct item representations reflect different voxel populations. These findings suggest an interplay between temporal differentiation in the DG and attractor dynamics in CA3. They advance our understanding of how knowledge is structured through integration and separation across time and context.

Neural mechanisms of sequential dependence in time perception: the impact of prior task and memory processing

Our perception and decision-making are susceptible to prior context. Such sequential dependence has been extensively studied in the visual domain, but less is known about its impact on time perception. Moreover, there are ongoing debates about whether these sequential biases occur at the perceptual stage or during subsequent post-perceptual processing. Using functional magnetic resonance imaging, we investigated neural mechanisms underlying temporal sequential dependence and the role of action in time judgments across trials. Participants performed a timing task where they had to remember the duration of green coherent motion and were cued to either actively reproduce its duration or simply view it passively. We found that sequential biases in time perception were only evident when the preceding task involved active duration reproduction. Merely encoding a prior duration without reproduction failed to induce such biases. Neurally, we observed activation in networks associated with timing, such as striato-thalamo-cortical circuits, and performance monitoring networks, particularly when a "Response" trial was anticipated. Importantly, the hippocampus showed sensitivity to these sequential biases, and its activation negatively correlated with the individual's sequential bias following active reproduction trials. These findings highlight the significant role of memory networks in shaping time-related sequential biases at the post-perceptual stages.

Retrieval of temporal structure at recall can occur automatically

Temporal-structure, namely, the order in which events unfold over time, is one of the fundamental principles of episodic memory organization. A seminal empirical demonstration of the prominence of temporal structure in memory organization is the Temporal Contiguity Effect (TCE), whereby the proximity between two items at encoding predicts the likelihood of those two items being retrieved consecutively during recall. Recent studies have found that TCE occurs under a wide variety of conditions in which strategic control processes at encoding are reduced or even eliminated. This suggests that the encoding of temporal structure occurs automatically. Extending these findings, in the current study we asked whether the retrieval of temporal structure, as reflected by indices of the TCE, is influenced by strategic control processes at retrieval. To manipulate participants' ability to rely on strategic control processes, we compared standard recall performance (Full Attention condition) to a condition in which attention was divided between recall and a concurrent task (Divided Attention condition), which has been shown to disrupt such control processes. Across two experiments-one with standard encoding conditions and one with continual distraction during encoding-we found no differences in any index of the TCE between the two conditions. These results are all the more striking considering that in both experiments, dividing attention negatively affected overall recall performance compared to the Full Attention condition. Thus, while recall performance is reduced when disrupting strategic processes, the ability to use temporal structure to drive recall is not affected.

Learning of object-in-context sequences in freely-moving macaques

Flexible learning is a hallmark of primate cognition, which arises through interactions with changing environments. Studies of the neural basis for this flexibility are typically limited by laboratory settings that use minimal environmental cues and restrict interactions with the environment, including active sensing and exploration. To address this, we constructed a 3-D enclosure containing touchscreens on its walls, for studying cognition in freely moving macaques. To test flexible learning, two monkeys completed trials consisting of a regular sequence of object selections across four touchscreens. On each screen, the monkeys had to select by touching the sole correct object item ('target') among a set of four items, irrespective of their positions on the screen. Each item was the target on exactly one screen of the sequence, making correct performance conditioned on the spatiotemporal sequence rule across screens. Both monkeys successfully learned multiple 4-item sets (N=14 and 22 sets), totaling over 50 and 80 unique, conditional item-context memoranda, with no indication of capacity limits. The enclosure allowed freedom of movements leading up to and following the touchscreen interactions. To determine whether movement economy changed with learning, we reconstructed 3D position and movement dynamics using markerless tracking software and gyroscopic inertial measurements. Whereas general body positions remained consistent across repeated sequences, fine head movements varied as monkeys learned, within and across sequence sets, demonstrating learning set or "learning to learn". These results demonstrate monkeys' rapid, capacious, and flexible learning within an integrated, multisensory 3-D space. Furthermore, this approach enables the measurement of continuous behavior while ensuring precise experimental control and behavioral repetition of sequences over time. Overall, this approach harmonizes the design features that are needed for electrophysiological studies with tasks that showcase fully situated, flexible cognition.

Visual Sequence Encoding is Enhanced by Predictable Music Pairing via Modulating Medial Temporal Lobe and Its Connectivity with Frontostriatal Loops

Listening to music during cognitive activities, such as reading and studying, is very common in human daily life. Therefore, it is important to understand how music interacts with concurrent cognitive functions, particularly memory. Current literature has presented mixed results for whether music can benefit learning in other modalities. Evidence is needed for what neural mechanisms music can tap into to enhance concurrent memory processing. This fMRI study aimed to begin filling this gap by investigating how music of varying predictability levels influences parallel visual sequence encoding performance. Behavioral results suggest that overall, predictable music enhances visual sequential encoding, and this effect increases with the structural regularity and familiarity of music. fMRI results indicate that during visual sequence encoding, music activates traditional music-processing and motor-related areas, but decreases parahippocampal and striatal engagement. This deactivation may indicate a more efficient encoding of visual information when music is present. By comparing music conditions of different structural predictability and familiarity, we probed how this occurs. We demonstrate improved encoding with increased syntactical regularity, which was associated with decreased activity in default mode network and increased activity in inferior temporal gyrus. Furthermore, the temporal schema provided by music familiarity may influence encoding through altered functional connectivity between the prefrontal cortex, medial temporal lobe and striatum. Overall, we propose that pairing music with learning might facilitate memory by reducing neural demands for visual encoding and simultaneously strengthening the connectivity between the medial temporal lobe and frontostriatal loops important for sequencing information.

Significance Statement

There is considerable interest in what mechanisms can be tapped to improve human memory. Music provides a potential modulator, but few studies have investigated music effects on encoding episodic memory. This study used a novel design to examine how music can influence concurrent visual item sequence encoding. We provided neural data to better understand mechanisms behind potential benefits of music for learning. Our results demonstrated predictable music may help guide parallel learning of sequences in another modality. We found that music might facilitate processing in neural systems associated with visual declarative long-term and working memory, and familiar music might modulate reward circuits and provide a temporal schema which facilitates better encoding of the temporal structure of new non-music information.

Re-expression of CA1 and entorhinal activity patterns preserves temporal context memory at long timescales

Converging, cross-species evidence indicates that memory for time is supported by hippocampal area CA1 and entorhinal cortex. However, limited evidence characterizes how these regions preserve temporal memories over long timescales (e.g., months). At long timescales, memoranda may be encountered in multiple temporal contexts, potentially creating interference. Here, using 7T fMRI, we measured CA1 and entorhinal activity patterns as human participants viewed thousands of natural scene images distributed, and repeated, across many months. We show that memory for an image's original temporal context was predicted by the degree to which CA1/entorhinal activity patterns from the first encounter with an image were re-expressed during re-encounters occurring minutes to months later. Critically, temporal memory signals were dissociable from predictors of recognition confidence, which were carried by distinct medial temporal lobe expressions. These findings suggest that CA1 and entorhinal cortex preserve temporal memories across long timescales by coding for and reinstating temporal context information.

Neural temporal context reinstatement of event structure during memory recall

The transformation of experiences into meaningful events and memories is intertwined with the notion of time. Temporal perception can influence, and be influenced by, segmenting continuous experience into meaningful events. Episodic memories formed from these events become associated with temporal information as well. However, it is less clear how temporal perception contributes to structuring events and organizing memory: whether it plays a more active or passive role, and whether this temporal information is encoded initially during perception or influenced by retrieval processes. To address these questions, we examined how event segmentation influences temporal representations during initial perception and memory retrieval, without testing temporal information explicitly. Using a neural measure of temporal context extracted from scalp electroencephalography in human participants (N = 170), we found reduced temporal context similarity between studied items separated by an event boundary when compared to items from the same event. Furthermore, while participants freely recalled list items, neural activity reflected reinstatement of temporal context representations from the study phase, including temporal disruption. A computational model of episodic memory, the context maintenance and retrieval (CMR) model, predicted these results, and made novel predictions regarding the influence of temporal disruption on recall order. These findings implicate the impact of event structure on memory organization via temporal representations, underscoring the role of temporal information in event segmentation and episodic memory. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Goal-oriented representations in the human hippocampus during planning and navigation

Recent work in cognitive and systems neuroscience has suggested that the hippocampus might support planning, imagination, and navigation by forming cognitive maps that capture the abstract structure of physical spaces, tasks, and situations. Navigation involves disambiguating similar contexts, and the planning and execution of a sequence of decisions to reach a goal. Here, we examine hippocampal activity patterns in humans during a goal-directed navigation task to investigate how contextual and goal information are incorporated in the construction and execution of navigational plans. During planning, hippocampal pattern similarity is enhanced across routes that share a context and a goal. During navigation, we observe prospective activation in the hippocampus that reflects the retrieval of pattern information related to a key-decision point. These results suggest that, rather than simply representing overlapping associations or state transitions, hippocampal activity patterns are shaped by context and goals.

A Learned Map for Places and Concepts in the Human Medial Temporal Lobe

Distinct lines of research in both humans and animals point to a specific role of the hippocampus in both spatial and episodic memory function. The discovery of concept cells in the hippocampus and surrounding medial temporal lobe (MTL) regions suggests that the MTL maps physical and semantic spaces with a similar neural architecture. Here, we studied the emergence of such maps using MTL microwire recordings from 20 patients (9 female, 11 male) navigating a virtual environment featuring salient landmarks with established semantic meaning. We present several key findings. The array of local field potentials in the MTL contains sufficient information for above-chance decoding of subjects' instantaneous location in the environment. Closer examination revealed that as subjects gain experience with the environment the field potentials come to represent both the subjects' locations in virtual space and in high-dimensional semantic space. Similarly, we observe a learning effect on temporal sequence coding. Over time, field potentials come to represent future locations, even after controlling for spatial proximity. This predictive coding of future states, more so than the strength of spatial representations per se, is linked to variability in subjects' navigation performance. Our results thus support the conceptualization of the MTL as a memory space, representing both spatial- and nonspatial information to plan future actions and predict their outcomes.SIGNIFICANCE STATEMENT Using rare microwire recordings, we studied the representation of spatial, semantic, and temporal information in the human MTL. Our findings demonstrate that subjects acquire a cognitive map that simultaneously represents the spatial and semantic relations between landmarks. We further show that the same learned representation is used to predict future states, implicating MTL cell assemblies as the building blocks of prospective memory functions.

Hippocampal integration and separation processes with different temporal and spatial dynamics during learning for associative memory

The hippocampus is known to be critically involved in associative memory formation. However, the role of the hippocampus during the learning of associative memory is still controversial; while the hippocampus is considered to play a critical role in the integration of related stimuli, numerous studies also suggest a role of the hippocampus in the separation of different memory traces for rapid learning. Here, we employed an associative learning paradigm consisting of repeated learning cycles. By tracking the changes in the hippocampal representations of associated stimuli on a cycle-by-cycle basis as learning progressed, we show that both integration and separation processes occur in the hippocampus with different temporal dynamics. We found that the degree of shared representations for associated stimuli decreased significantly during the early phase of learning, whereas it increased during the later phase of learning. Remarkably, these dynamic temporal changes were observed only for stimulus pairs remembered 1 day or 4 weeks after learning, but not for forgotten pairs. Further, the integration process during learning was prominent in the anterior hippocampus, while the separation process was obvious in the posterior hippocampus. These results demonstrate temporally and spatially dynamic hippocampal processing during learning that can lead to the maintenance of associative memory.

Cross-Modal Facilitation of Episodic Memory by Sequential Action Execution

Throughout our lives, the actions we produce are often highly familiar and repetitive (e.g., commuting to work). However, layered upon these routine actions are novel, episodic experiences. Substantial research has shown that prior knowledge can facilitate learning of conceptually related new information. But despite the central role our behavior plays in real-world experience, it remains unclear how engagement in a familiar sequence of actions influences memory for unrelated, nonmotor information coincident with those actions. To investigate this, we had healthy young adults encode novel items while simultaneously following a sequence of actions (key presses) that was either predictable and well-learned or random. Across three experiments (N = 80 each), we found that temporal order memory, but not item memory, was significantly enhanced for novel items encoded while participants executed predictable compared with random action sequences. These results suggest that engaging in familiar behaviors during novel learning scaffolds within-event temporal memory, an essential feature of episodic experiences.

The neural representations underlying asymmetric cross-modal prediction of words

Cross-modal prediction serves a crucial adaptive role in the multisensory world, yet the neural mechanisms underlying this prediction are poorly understood. The present study addressed this important question by combining a novel audiovisual sequence memory task, functional magnetic resonance imaging (fMRI), and multivariate neural representational analyses. Our behavioral results revealed a reliable asymmetric cross-modal predictive effect, with a stronger prediction from visual to auditory (VA) modality than auditory to visual (AV) modality. Mirroring the behavioral pattern, we found the superior parietal lobe (SPL) showed higher pattern similarity for VA than AV pairs, and the strength of the predictive coding in the SPL was positively correlated with the behavioral predictive effect in the VA condition. Representational connectivity analyses further revealed that the SPL mediated the neural pathway from the visual to the auditory cortex in the VA condition but was not involved in the auditory to visual cortex pathway in the AV condition. Direct neural pathways within the unimodal regions were found for the visual-to-visual and auditory-to-auditory predictions. Together, these results provide novel insights into the neural mechanisms underlying cross-modal sequence prediction.

Impaired cognitive performance in older adults is associated with deficits in item memory and memory for object features

Amnestic mild cognitive impairment (aMCI) is associated with damage to the perirhinal/entorhinal cortex, and consequently, deficits in item/object memory. However, cognitive assessments commonly used to identify individuals with aMCI require a clinician to administer and interpret the test. We developed a novel self-administered global cognitive assessment, called the Cognitive Assessment via Keyboard (CAKe). To assess the relationship between CAKe performance and perirhinal/entorhinal cortex-dependent memory function, participants completed the CAKe, a feature source memory task, and a context memory task. During the memory tasks, participants studied line drawings with either a green or orange internal color (feature memory runs) or external color (context memory runs) and then classified each item as old and previously presented with a "green" or "orange" color, or "new". CAKe scores were correlated with item memory accuracies and source memory accuracies on both tasks. Participants with 'impaired' CAKe performance had worse item memory and worse feature source memory accuracies than those with 'normal' CAKe performance. These results demonstrate specific deficits in item memory and feature source memory and suggest that our assessments may be a valid predictor of aMCI memory deficits.

Mnemonic Content and Hippocampal Patterns Shape Judgments of Time

Our experience of time can feel dilated or compressed, rather than reflecting true "clock time." Although many contextual factors influence the subjective perception of time, it is unclear how memory accessibility plays a role in constructing our experience of and memory for time. Here, we used a combination of behavioral and functional MRI measures in healthy young adults (N = 147) to ask the question of how memory is incorporated into temporal duration judgments. Behaviorally, we found that event boundaries, which have been shown to disrupt ongoing memory integration processes, result in the temporal compression of duration judgments. Additionally, using a multivoxel pattern similarity analysis of functional MRI data, we found that greater temporal pattern change in the left hippocampus within individual trials was associated with longer duration judgments. Together, these data suggest that mnemonic processes play a role in constructing representations of time.

Temporal context and latent state inference in the hippocampal splitter signal

The hippocampus is thought to enable the encoding and retrieval of ongoing experience, the organization of that experience into structured representations like contexts, maps, and schemas, and the use of these structures to plan for the future. A central goal is to understand what the core computations supporting these functions are, and how these computations are realized in the collective action of single neurons. A potential access point into this issue is provided by 'splitter cells', hippocampal neurons that fire differentially on the overlapping segment of trajectories that differ in their past and/or future. However, the literature on splitter cells has been fragmented and confusing, owing to differences in terminology, behavioral tasks, and analysis methods across studies. In this review, we synthesize consistent findings from this literature, establish a common set of terms, and translate between single-cell and ensemble perspectives. Most importantly, we examine the combined findings through the lens of two major theoretical ideas about hippocampal function: representation of temporal context and latent state inference. We find that unique signature properties of each of these models are necessary to account for the data, but neither theory, by itself, explains all of its features. Specifically, the temporal gradedness of the splitter signal is strong support for temporal context, but is hard to explain using state models, while its flexibility and task-dependence is naturally accounted for using state inference, but poses a challenge otherwise. These theories suggest a number of avenues for future work, and we believe their application to splitter cells is a timely and informative domain for testing and refining theoretical ideas about hippocampal function.

Memory-Based Prediction Deficits and Dorsolateral Prefrontal Dysfunction in Schizophrenia

BACKGROUND

Theories suggest that people with schizophrenia (SZ) have problems generating predictions based on past experiences. The dorsolateral prefrontal cortex (DLPFC) and hippocampus participate in memory-based prediction. We used functional magnetic resonance imaging to investigate DLPFC and hippocampal function in healthy control (HC) subjects and people with SZ during memory-based prediction.

METHODS

Prior to scanning, HC subjects (n = 54) and people with SZ (n = 31) learned 5-object sequences presented in fixed or random orders on each repetition. During scanning, participants made semantic decisions (e.g., "Can this object fit in a shoebox?") on a continuous stream of objects from fixed and random sequences. Sequence prediction was demonstrated by faster semantic decisions for objects in fixed versus random sequences because memory could be used to anticipate and more efficiently process semantic information about upcoming objects in fixed sequences. Representational similarity analyses were used to determine how each sequence type was represented in the posterior hippocampus and DLPFC.

RESULTS

Sequence predictions were reduced in individuals with SZ relative to HC subjects. Representational similarity analyses revealed stronger memory-based predictions in the DLPFC of HC subjects than people with SZ, and DLPFC representations correlated with more successful predictions in HC subjects only. For the posterior hippocampus, voxel pattern similarity was increased for fixed versus random sequences in HC subjects only, but no significant between-group differences or correlations with prediction success were observed.

CONCLUSIONS

Individuals with SZ are capable of learning temporal sequences; however, they are impaired using memory to predict upcoming events as efficiently as HC subjects. This deficit appears related to disrupted neural representation of sequence information in the DLPFC.

The tie that binds: temporal coding and adaptive emotion

Emotions are temporally dynamic, but the persistence of emotions outside of their appropriate temporal context is detrimental to health and well-being. Yet, precisely how temporal coding and emotional processing interact remains unclear. Recently unveiled temporal context representations in the hippocampus, entorhinal cortex (EC), and prefrontal cortex (PFC) support memory for what happened when. Here, we discuss how these neural temporal representations may interact with densely interconnected amygdala circuitry to shape emotional functioning. We propose a neuroanatomically informed framework suggesting that high-fidelity temporal representations linked to dynamic experiences promote emotion regulation and adaptive emotional memories. Then, we discuss how newly-identified synaptic and molecular features of amygdala-hippocampal projections suggest that intense, amygdala-dependent emotional responses may distort temporal-coding mechanisms. We conclude by identifying key avenues for future research.

Non-spatial similarity can bias spatial distances in a cognitive map

The cognitive map theory suggests the hippocampal-entorhinal system has a representation of space that encodes geometric properties. There is also evidence that the hippocampus plays a critical role in supporting declarative memory, and recent theories have hypothesized the mechanism for encoding space is the same as that for processing memory. If space is not represented independently, it might be influenced by non-spatial properties. This study tested whether connections between non-spatial properties can distort judgments about spatial distance. In virtual reality, subjects navigated through an environment to learn the locations of target houses, and then were tested on their ability to judge the pairwise distances between houses and reconstruct a map of the environment. The environment was constructed to have pairs of houses with the same spatial distance but either the same or different color. If memory for spatial and non-spatial properties interact, similar houses would be expected to be judged as closer. In Experiment 1, the similar pairs all had the same color, while in Experiment 2, each pair had a different color to make the pairs more distinctive. We observed that similar houses were drawn closer on reconstructed maps in both experiments, and pairwise distance judgments were smaller for similar houses in Experiment 2. Biases from color similarity are difficult to reconcile with independent representation of space. Our results support theories that space is represented with other properties, and the mechanisms for encoding space in the hippocampal-entorhinal system have a broader function.

The neural bases for timing of durations

Durations are defined by a beginning and an end, and a major distinction is drawn between durations that start in the present and end in the future ('prospective timing') and durations that start in the past and end either in the past or the present ('retrospective timing'). Different psychological processes are thought to be engaged in each of these cases. The former is thought to engage a clock-like mechanism that accurately tracks the continuing passage of time, whereas the latter is thought to engage a reconstructive process that utilizes both temporal and non-temporal information from the memory of past events. We propose that, from a biological perspective, these two forms of duration 'estimation' are supported by computational processes that are both reliant on population state dynamics but are nevertheless distinct. Prospective timing is effectively carried out in a single step where the ongoing dynamics of population activity directly serve as the computation of duration, whereas retrospective timing is carried out in two steps: the initial generation of population state dynamics through the process of event segmentation and the subsequent computation of duration utilizing the memory of those dynamics.

From remembering to reconstruction: The transformative neural representation of episodic memory

Although memory has long been recognized as a generative process, neural research of memory in recent decades has been predominantly influenced by Tulving's "mental time traveling" perspective and focused on the reactivation and consolidation of encoded memory representations. With the development of multiple powerful analytical approaches to characterize the contents and formats of neural representations, recent studies are able to provide detailed examinations of the representations at various processing stages and have provided exciting new insights into the transformative nature of episodic memory. These studies have revealed the rapid, substantial, and continuous transformation of memory representation during the encoding, maintenance, consolidation, and retrieval of both single and multiple events, as well as event sequences. These transformations are characterized by the abstraction, integration, differentiation, and reorganization of memory representations, enabling the long-term retention and generalization of memory. These studies mark a significant shift in perspective from remembering to reconstruction, which might better reveal the nature of memory and its roles in supporting more effective learning, adaptive decision-making, and creative problem solving.

Assessing mild cognitive impairment using object-location memory in immersive virtual environments

Pathological changes in the medial temporal lobe (MTL) are found in the early stages of Alzheimer's disease (AD) and aging. The earliest pathological accumulation of tau colocalizes with the areas of the MTL involved in object processing as part of a wider anterolateral network. Here, we sought to assess the diagnostic potential of memory for object locations in iVR environments in individuals at high risk of AD dementia (amnestic mild cognitive impairment [aMCI] n = 23) as compared to age-related cognitive decline. Consistent with our primary hypothesis that early AD would be associated with impaired object location, aMCI patients exhibited impaired spatial feature binding. Compared to both older (n = 24) and younger (n = 53) controls, aMCI patients, recalled object locations with significantly less accuracy (p < .001), with a trend toward an impaired identification of the object's correct context (p = .05). Importantly, these findings were not explained by deficits in object recognition (p = .6). These deficits differentiated aMCI from controls with greater accuracy (AUC = 0.89) than the standard neuropsychological tests. Within the aMCI group, 16 had CSF biomarkers indicative of their likely AD status (MCI+ n = 9 vs. MCI- n = 7). MCI+ showed lower accuracy in the object-context association than MCI- (p = .03) suggesting a selective deficit in object-context binding postulated to be associated with anterior-temporal areas. MRI volumetric analysis across healthy older participants and aMCI revealed that test performance positively correlates with lateral entorhinal cortex volumes (p < .05) and hippocampus volumes (p < .01), consistent with their hypothesized role in binding contextual and spatial information with object identity. Our results indicate that tests relying on the anterolateral object processing stream, and in particular requiring successful binding of an object with spatial information, may aid detection of pre-dementia AD due to the underlying early spread of tau pathology.

Virtual navigation in healthy aging: Activation during learning and deactivation during retrieval predicts successful memory for spatial locations

Spatial navigation and spatial memory are two important skills for independent living, and are known to be compromised with age. Here, we investigate the neural correlates of successful spatial memory in healthy older adults in order to learn more about the neural underpinnings of maintenance of navigation skill into old age. Healthy older adults watched a video shot by a person navigating a route and were asked to remember objects along the route and then attempted to remember object locations by virtually pointing to the location of hidden objects from several locations along the route. Brain activity during watching and pointing was recorded with functional MRI. Larger activations in temporal and frontal regions during watching, and larger deactivations in superior parietal cortex and intraparietal sulcus during pointing, were associated with smaller location errors. These findings suggest that larger evoked responses during learning of spatial information coupled with larger deactivation of canonical spatial memory regions at retrieval are important for effective spatial memory in late life.

Contextual Expectations Shape Cortical Reinstatement of Sensory Representations

When making a turn at a familiar intersection, we know what items and landmarks will come into view. These perceptual expectations, or predictions, come from our knowledge of the context; however, it is unclear how memory and perceptual systems interact to support the prediction and reactivation of sensory details in cortex. To address this, human participants learned the spatial layout of animals positioned in a cross maze. During fMRI, participants of both sexes navigated between animals to reach a target, and in the process saw a predictable sequence of five animal images. Critically, to isolate activity patterns related to item predictions, rather than bottom-up inputs, one-fourth of trials ended early, with a blank screen presented instead. Using multivariate pattern similarity analysis, we reveal that activity patterns in early visual cortex, posterior medial regions, and the posterior hippocampus showed greater similarity when seeing the same item compared with different items. Further, item effects in posterior hippocampus were specific to the sequence context. Critically, activity patterns associated with seeing an item in visual cortex and posterior medial cortex, were also related to activity patterns when an item was expected, but omitted, suggesting sequence predictions were reinstated in these regions. Finally, multivariate connectivity showed that patterns in the posterior hippocampus at one position in the sequence were related to patterns in early visual cortex and posterior medial cortex at a later position. Together, our results support the idea that hippocampal representations facilitate sensory processing by modulating visual cortical activity in anticipation of expected items.SIGNIFICANCE STATEMENT Our visual world is a series of connected events, where we can predict what we might see next based on our recent past. Understanding the neural circuitry and mechanisms of the perceptual and memory systems that support these expectations is fundamental to revealing how we perceive and act in our world. Using brain imaging, we studied what happens when we expect to see specific visual items, and how such expectations relate to top-down memory signals. We find both visual and memory systems reflect item predictions, and moreover, we show that hippocampal activity supports predictions of future expected items. This demonstrates that the hippocampus acts to predict upcoming items, and reinstates such predictions in cortex.

Stronger memory representation after memory reinstatement during retrieval in the human hippocampus

Memory retrieval allows us to reinstate previously encoded information but is also considered to contribute to memory enhancement. Retrieval-induced enhancement may involve processing to strengthen memory traces, but neural processing beyond reinstatement during retrieval remains elusive. Here, we show that hippocampal processing, different from memory reinstatement, exists during retrieval in the human brain. By tracking changes in the response patterns in the selected hippocampal and cortical regions over time during retrieval based on functional MRI, we found that the representation of associative memory in CA3/DG became stronger even after cortical memory reinstatement, while CA1 showed significant memory representation at retrieval onset with the cortical reinstatement, but not afterwards. This tendency was not observed in the condition without active retrieval. Moreover, subsequent long-term memory performance depended on the delayed CA3/DG representation during retrieval. These findings suggest that CA3/DG contributes to neural processing beyond memory reinstatement during retrieval, which may lead to memory enhancement.

Mnemonic construction and representation of temporal structure in the hippocampal formation

The hippocampal-entorhinal region supports memory for episodic details, such as temporal relations of sequential events, and mnemonic constructions combining experiences for inferential reasoning. However, it is unclear whether hippocampal event memories reflect temporal relations derived from mnemonic constructions, event order, or elapsing time, and whether these sequence representations generalize temporal relations across similar sequences. Here, participants mnemonically constructed times of events from multiple sequences using infrequent cues and their experience of passing time. After learning, event representations in the anterior hippocampus reflected temporal relations based on constructed times. Temporal relations were generalized across sequences, revealing distinct representational formats for events from the same or different sequences. Structural knowledge about time patterns, abstracted from different sequences, biased the construction of specific event times. These findings demonstrate that mnemonic construction and the generalization of relational knowledge combine in the hippocampus, consistent with the simulation of scenarios from episodic details and structural knowledge.

Activity patterns in the hippocampus resemble temporal relations of learned event sequences. Here, the authors show that these relational memories arise through mnemonic construction and are generalized to reflect the temporal event structure.

Automatic and effortful control of interference in working memory can be distinguished by unique behavioral and functional brain representations

Goal-irrelevant information in working memory (WM) may enter the focus of attention (FOA) during a task and cause proactive interference (PI). In the current study we used fMRI to test several hypotheses concerning the boundary conditions of PI in WM using a modified verbal 2-back task. Temporal distance between item and lure presentation was manipulated to evaluate potential differences among hypothesized states of FOA, short-term memory and long-term memory. PI was present for the most proximal 3-back lures but dissipated with lure distance along with increased activation in brain regions critical for memory recollection, such as right prefrontal cortex, parietal cortex, and hippocampus. Reduced PI and less IFG activation were also observed after repeated item presentation, supporting the notion that a rehearsed encoding of item-context information reduces the need for interference control. Moreover, a trial-by-trial approach revealed activity in ACC, insula, IFG, and parietal cortex with increasing lure trial interference regardless of distance. The current results are first evidence for an observable transition of cognitive control, to include MTL regions involved in recalling task-relevant information from outside the FOA when resolving PI in WM.

Event segmentation reveals working memory forgetting rate

We encounter the world as a continuous flow and effortlessly segment sequences of events into episodes. This process of event segmentation engages working memory (WM) for tracking the flow of events and impacts subsequent memory accuracy. WM is limited in how much information (i.e., WM capacity) and for how long the information is retained (i.e., forgetting rate). In this study, across multiple tasks, we estimated participants’ WM capacity and forgetting rate in a dynamic context and evaluated their relationship to event segmentation. A U-shaped relationship across tasks shows that individuals who segmented the movie more finely or coarsely than the average have a faster WM forgetting rate. A separate task assessing long-term memory retrieval revealed that the coarse-segmenters have better recognition of temporal order of events compared to the fine-segmenters. These findings show that event segmentation employs dissociable memory strategies and correlates with how long information is retained in WM

•

The event segmentation grain is variable across individuals

•

The event segmentation grain has a U-shaped relationship with the WM forgetting rate

•

The temporal order memory accuracy decreases with the increasing event segmentation

•

The number of recalled events increases with the increasing event segmentation

Consistent population activity on the scale of minutes in the mouse hippocampus

Neurons in the hippocampus fire in consistent sequence over the timescale of seconds during the delay period of some memory experiments. For longer timescales, the firing of hippocampal neurons also changes slowly over minutes within experimental sessions. It was thought that these slow dynamics are caused by stochastic drift or a continuous change in the representation of the episode, rather than consistent sequences unfolding over minutes. This paper studies the consistency of contextual drift in three chronic calcium imaging recordings from the hippocampus CA1 region in mice. Computational measures of consistency show reliable sequences within experimental trials at the scale of seconds as one would expect from time cells or place cells during the trial, as well as across experimental trials on the scale of minutes within a recording session. Consistent sequences in the hippocampus are observed over a wide range of time scales, from seconds to minutes. The hippocampal activity could reflect a scale-invariant spatiotemporal context as suggested by theories of memory from cognitive psychology.

Increasing stimulus similarity drives nonmonotonic representational change in hippocampus

Studies of hippocampal learning have obtained seemingly contradictory results, with manipulations that increase coactivation of memories sometimes leading to differentiation of these memories, but sometimes not. These results could potentially be reconciled using the nonmonotonic plasticity hypothesis, which posits that representational change (memories moving apart or together) is a U-shaped function of the coactivation of these memories during learning. Testing this hypothesis requires manipulating coactivation over a wide enough range to reveal the full U-shape. To accomplish this, we used a novel neural network image synthesis procedure to create pairs of stimuli that varied parametrically in their similarity in high-level visual regions that provide input to the hippocampus. Sequences of these pairs were shown to human participants during high-resolution fMRI. As predicted, learning changed the representations of paired images in the dentate gyrus as a U-shaped function of image similarity, with neural differentiation occurring only for moderately similar images.

Multivariate FMRI Signatures of Learning in a Hebb Repetition Paradigm With Tone Sequences

Important information from the environment often arrives to the brain in temporally extended sequences. Language, music, actions, and complex events generally unfold over time. When such informational sequences exceed the limited capacity of working memory, the human brain relies on its ability to accumulate information in long-term memory over several encounters with a complex stimulus. A longstanding question in psychology and neuroscience is whether the neural structures associated with working memory storage—often viewed as capacity limited and temporary—have any builtin ability to store information across longer temporal delays. According to the classic Hebbian dual memory theory, temporally local “activity traces” underlie immediate perception and working memory, whereas “structural traces” undergird long-term learning. Here we examine whether brain structures known to be involved in working maintenance of auditory sequences, such as area Spt, also show evidence of memory persistence across trials. We used representational similarity analysis (RSA) and the Hebb repetition paradigm with supracapacity tonal sequences to test whether repeated sequences have distinguishable multivoxel activity patterns in the auditory-motor networks of the brain. We found that, indeed, area Spt and other nodes of the auditory dorsal stream show multivoxel patterns for tone sequences that become gradually more distinct with repetition during working memory for supracapacity tone-sequences. The findings suggest that the structures are important for working memory are not “blank slates,” wiped clean from moment to moment, but rather encode information in a way can lead to cross-trial persistence.

Topography of hippocampal connectivity with sensorimotor cortex revealed by optimizing smoothing kernel and voxel size

Studies of the hippocampus use smaller voxel sizes and smoothing kernels than cortical activation studies, typically using a multivoxel seed with specified radius for connectivity analysis. This study identified optimal processing parameters for evaluating hippocampal connectivity with sensorimotor cortex (SMC), comparing effectiveness by varying parameters during both activation and connectivity analysis. Using both 3mm and 4mm isovoxels, smoothing kernels of 0-10mm were evaluated on the amplitude and extent of motor activation and hippocampal connectivity with SMC. Psychophysiological interactions (PPI) identified hippocampal connectivity with SMC during volitional movements, and connectivity effects from multivoxel seeds were compared with alternate methods; a structural seed represented the mean connectivity map from all voxels within a region, whereas a functional seed represented the regional voxel with maximal SMC connectivity. With few exceptions, the same parameters were optimal for activation and connectivity. Larger isovoxels showed larger activation volumes in both SMC and the hippocampus; connectivity volumes from structural seeds were also larger, except from the posterior hippocampus. Regardless of voxel size, the 10mm smoothing kernel generated larger activation and connectivity volumes from structural seeds, as well as larger beta estimates at connectivity maxima; structural seeds also produced larger connectivity volumes than multivoxel seeds. Functional seeds showed lesser effects from voxel size and smoothing kernels. Optimal parameters revealed topography in structural seed connectivity along both the longitudinal axis and mediolateral axis of the hippocampus. These results indicate larger voxels and smoothing kernels can improve sensitivity for detecting both cortical activation and hippocampal connectivity.

Partially overlapping spatial environments trigger reinstatement in hippocampus and schema representations in prefrontal cortex

When we remember a city that we have visited, we retrieve places related to finding our goal but also non-target locations within this environment. Yet, understanding how the human brain implements the neural computations underlying holistic retrieval remains unsolved, particularly for shared aspects of environments. Here, human participants learned and retrieved details from three partially overlapping environments while undergoing high-resolution functional magnetic resonance imaging (fMRI). Our findings show reinstatement of stores even when they are not related to a specific trial probe, providing evidence for holistic environmental retrieval. For stores shared between cities, we find evidence for pattern separation (representational orthogonalization) in hippocampal subfield CA2/3/DG and repulsion in CA1 (differentiation beyond orthogonalization). Additionally, our findings demonstrate that medial prefrontal cortex (mPFC) stores representations of the common spatial structure, termed schema, across environments. Together, our findings suggest how unique and common elements of multiple spatial environments are accessed computationally and neurally.

Affective Episodic Memory System for Virtual Creatures: The First Step of Emotion-Oriented Memory

Episodic memory and emotions are considered essential functions in human cognition. Both allow us to acquire new knowledge from the environment, ranging from the objects around us to how we feel towards them. These qualities make them crucial functions for systems trying to create human-like behaviour. In the field of cognitive architectures (CAs), there are multiple studies covering memory and emotions. However, most of them treat these subjects in an isolated manner, considering emotions only as a reward signal unrelated to a retrieved experience. To address this lack of direct interaction, we propose a computational model that covers the common processes that are related to memory and emotions. Specifically, this proposal focuses on affective evaluations of episodic memories. Neurosciences and psychology are the bases of this model. That is, the model's components and the processes that they carry out on the information they receive are designed based on evidence from these cognitive sciences. The proposed model is a part of Cuáyóllótl, a cognitive architecture for cybernetic entities such as virtual creatures and robots. Case studies validate our proposal. They show the relevance of the integration of emotions and memory in a virtual creature. The virtual creature endowed with our emotional episodic model improves its learning and modifies its behaviour according to planning and decision-making processes.

Brain Mechanisms Underlying the Subjective Experience of Remembering

The ability to remember events in vivid, multisensory detail is a significant part of human experience, allowing us to relive previous encounters and providing us with the store of memories that shape our identity. Recent research has sought to understand the subjective experience of remembering, that is, what it feels like to have a memory. Such remembering involves reactivating sensory-perceptual features of an event and the thoughts and feelings we had when the event occurred, integrating them into a conscious first-person experience. It allows us to reflect on the content of our memories and to understand and make judgments about them, such as distinguishing events that actually occurred from those we might have imagined or been told about. In this review, we consider recent evidence from functional neuroimaging in healthy participants and studies of neurological and psychiatric conditions, which is shedding new light on how we subjectively experience remembering. Expected final online publication date for the Annual Review of Psychology, Volume 73 is January 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Neurophysiology of Remembering

By linking the past with the future, our memories define our sense of identity. Because human memory engages the conscious realm, its examination has historically been approached from language and introspection and proceeded largely along separate parallel paths in humans and other animals. Here, we first highlight the achievements and limitations of this mind-based approach and make the case for a new brain-based understanding of declarative memory with a focus on hippocampal physiology. Next, we discuss the interleaved nature and common physiological mechanisms of navigation in real and mental spacetime. We suggest that a distinguishing feature of memory types is whether they subserve actions for single or multiple uses. Finally, in contrast to the persisting view of the mind as a highly plastic blank slate ready for the world to make its imprint, we hypothesize that neuronal networks are endowed with a reservoir of neural trajectories, and the challenge faced by the brain is how to select and match preexisting neuronal trajectories with events in the world. Expected final online publication date for the Annual Review of Psychology, Volume 73 is January 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

How do we generalize?

Humans and animals are able to generalize or transfer information from previous experience so that they can behave appropriately in novel situations. What mechanisms-computations, representations, and neural systems-give rise to this remarkable ability? The members of this Generative Adversarial Collaboration (GAC) come from a range of academic backgrounds but are all interested in uncovering the mechanisms of generalization. We started out this GAC with the aim of arbitrating between two alternative conceptual accounts: (1) generalization stems from integration of multiple experiences into summary representations that reflect generalized knowledge, and (2) generalization is computed on-the-fly using separately stored individual memories. Across the course of this collaboration, we found that-despite using different terminology and techniques, and although some of our specific papers may provide evidence one way or the other-we in fact largely agree that both of these broad accounts (as well as several others) are likely valid. We believe that future research and theoretical synthesis across multiple lines of research is necessary to help determine the degree to which different candidate generalization mechanisms may operate simultaneously, operate on different scales, or be employed under distinct conditions. Here, as the first step, we introduce some of these candidate mechanisms and we discuss the issues currently hindering better synthesis of generalization research. Finally, we introduce some of our own research questions that have arisen over the course of this GAC, that we believe would benefit from future collaborative efforts.

Hippocampal Representations of Event Structure and Temporal Context during Episodic Temporal Order Memory

The hippocampus plays an important role in representing spatial locations and sequences and in transforming representations. How these representational structures and operations support memory for the temporal order of random items is still poorly understood. We addressed this question by leveraging the method of loci, a powerful mnemonic strategy for temporal order memory that particularly recruits hippocampus-dependent computations of spatial locations and associations. Applying representational similarity analysis to functional magnetic resonance imaging activation patterns revealed that hippocampal subfields contained representations of multiple features of sequence structure, including spatial locations, location distance, and sequence boundaries, as well as episodic-like temporal context. Critically, the hippocampal CA1 exhibited spatial transformation of representational patterns, showing lower pattern similarity for items in same locations than closely matched different locations during retrieval, whereas the CA23DG exhibited sequential transformation of representational patterns, showing lower pattern similarity for items in near locations than in far locations during encoding. These transformations enabled the encoding of multiple items in the same location and disambiguation of adjacent items. Our results suggest that the hippocampus can flexibly reconfigure multiplexed event structure representations to support accurate temporal order memory.

The Neural Representation of Ordinal Information: Domain-Specific or Domain-General?

Ordinal processing allows for the representation of the sequential relations between stimuli and is a fundamental aspect of different cognitive domains such as verbal working memory (WM), language and numerical cognition. Several studies suggest common ordinal coding mechanisms across these different domains but direct between-domain comparisons of ordinal coding are rare and have led to contradictory evidence. This fMRI study examined the commonality of ordinal representations across the WM, the number, and the letter domains by using a multivoxel pattern analysis approach and by focusing on triplet stimuli associated with robust ordinal distance effects. Neural patterns in fronto-parietal cortices distinguished ordinal distance in all domains. Critically, between-task predictions of ordinal distance in fronto-parietal cortices were robust between serial order WM, alphabetical order judgment but not when involving the numerical order judgment tasks. Moreover, frontal ROIs further supported between-task prediction of distance for the luminance judgment control task, the serial order WM, and the alphabetical tasks. These results suggest that common neural substrates characterize processing of ordinal information in WM and alphabetical but not numerical domains. This commonality, particularly in frontal cortices, may however reflect attentional control processes involved in judging ordinal distances rather than the intervention of domain-general ordinal codes.

Distinct Neural Representations of Content and Ordinal Structure in Auditory Sequence Memory

Two forms of information, frequency (content) and ordinal position (structure), have to be stored when retaining a sequence of auditory tones in working memory (WM). However, the neural representations and coding characteristics of content and structure, particularly during WM maintenance, remain elusive. Here, in two EEG studies in human participants (both sexes), by transiently perturbing the "activity-silent" WM retention state and decoding the reactivated WM information, we demonstrate that content and structure are stored in a dissociative manner with distinct characteristics throughout WM process. First, each tone in the sequence is associated with two codes in parallel, characterizing its frequency and ordinal position, respectively. Second, during retention, a structural retrocue successfully reactivates structure but not content, whereas a following white noise triggers content but not structure. Third, structure representation remains stable, whereas content code undergoes a dynamic transformation through memory progress. Finally, the noise-triggered content reactivations during retention correlate with subsequent WM behavior. Overall, our results support distinct content and structure representations in auditory WM and provide an efficient approach to access the silently stored WM information in the human brain. The dissociation of content and structure could facilitate efficient memory formation via generalizing stable structure to new auditory contents.SIGNIFICANCE STATEMENT In memory experiences, contents do not exist independently but are linked with each other via ordinal structure. For instance, recalling a piece of favorite music relies on correct ordering (sequence structure) of musical tones (content). How are the structure and content for an auditory temporally structured experience maintained in working memory? Here, by using impulse-response approach and time-resolved representational dissimilarity analysis on human EEG recordings in an auditory working memory task, we reveal that content and structure are stored in a dissociated way, which would facilitate efficient and rapid memory formation through generalizing stable structure knowledge to new auditory inputs.