Hippocampal contributions to serial-order memory

Learning of object-in-context sequences in freely-moving macaques

Flexible learning is a hallmark of primate cognition, which arises through interactions with changing environments. Studies of the neural basis for this flexibility are typically limited by laboratory settings that use minimal environmental cues and restrict interactions with the environment, including active sensing and exploration. To address this, we constructed a 3-D enclosure containing touchscreens on its walls, for studying cognition in freely moving macaques. To test flexible learning, two monkeys completed trials consisting of a regular sequence of object selections across four touchscreens. On each screen, the monkeys had to select by touching the sole correct object item ('target') among a set of four items, irrespective of their positions on the screen. Each item was the target on exactly one screen of the sequence, making correct performance conditioned on the spatiotemporal sequence rule across screens. Both monkeys successfully learned multiple 4-item sets (N=14 and 22 sets), totaling over 50 and 80 unique, conditional item-context memoranda, with no indication of capacity limits. The enclosure allowed freedom of movements leading up to and following the touchscreen interactions. To determine whether movement economy changed with learning, we reconstructed 3D position and movement dynamics using markerless tracking software and gyroscopic inertial measurements. Whereas general body positions remained consistent across repeated sequences, fine head movements varied as monkeys learned, within and across sequence sets, demonstrating learning set or "learning to learn". These results demonstrate monkeys' rapid, capacious, and flexible learning within an integrated, multisensory 3-D space. Furthermore, this approach enables the measurement of continuous behavior while ensuring precise experimental control and behavioral repetition of sequences over time. Overall, this approach harmonizes the design features that are needed for electrophysiological studies with tasks that showcase fully situated, flexible cognition.

What neural oscillations can and cannot do for syntactic structure building

Understanding what someone says requires relating words in a sentence to one another as instructed by the grammatical rules of a language. In recent years, the neurophysiological basis for this process has become a prominent topic of discussion in cognitive neuroscience. Current proposals about the neural mechanisms of syntactic structure building converge on a key role for neural oscillations in this process, but they differ in terms of the exact function that is assigned to them. In this Perspective, we discuss two proposed functions for neural oscillations - chunking and multiscale information integration - and evaluate their merits and limitations taking into account a fundamentally hierarchical nature of syntactic representations in natural languages. We highlight insights that provide a tangible starting point for a neurocognitive model of syntactic structure building.

No evidence for a preferential role of sleep in episodic memory abstraction

Substantial evidence suggests that sleep has a role in declarative memory consolidation. An influential notion holds that such sleep-related memory consolidation is associated with a process of abstraction. The neural underpinnings of this putative process are thought to involve a hippocampo-neocortical dialogue. Specifically, the idea is that, during sleep, the statistical contingencies across episodes are re-coded to a less hippocampus-dependent format, while at the same time losing configural information. Two previous studies from our lab, however, failed to show a preferential role of sleep in either episodic memory decontextualisation or the formation of abstract knowledge across episodic exemplars. Rather these processes occurred over sleep and wake time alike. Here, we present two experiments that replicate and extend these previous studies and exclude some alternative interpretations. The combined data show that sleep has no preferential function in this respect. Rather, hippocampus-dependent memories are generalised to an equal extent across both wake and sleep time. The one point on which sleep outperforms wake is actually the preservation of episodic detail of memories stored prior to sleep.

Intact high-resolution working memory binding in a patient with developmental amnesia and selective hippocampal damage

Debate continues regarding the possible role of the hippocampus across short-term and working memory tasks. The current study examined the possibility of a hippocampal contribution to precise, high-resolution cognition and conjunctive memory. We administered visual working memory tasks featuring a continuous response component to a well-established developmental amnesic patient with relatively selective bilateral hippocampal damage (Jon) and healthy controls. The patient was able to produce highly accurate response judgments regarding conjunctions of color and orientation or color and location, using simultaneous or sequential presentation of stimuli, with no evidence of any impairment in working memory binding, categorical accuracy, or continuous precision. These findings indicate that hippocampal damage does not necessarily lead to deficits in high-resolution cognitive performance, even when the damage is severe and bilateral.

Neural Patterns in Parietal Cortex and Hippocampus Distinguish Retrieval of Start versus End Positions in Working Memory

Coding serial order of information is a fundamental ability of our cognitive system, and still, little is known about its neural substrate. This study examined the neural substrates involved in the retrieval of information that is serially stored in verbal working memory task using a sensitive multivariate analysis approach. We compared neural activity for memorized items stemming from the beginning versus the end of a memory list assessing the degree of neural pattern discordance between order positions (beginning vs. end). The present results confirmed and refined the role of the intraparietal sulcus in the processing of serial order information in working memory. An important finding is that the hippocampus showed sensitivity to serial order information. Our results indicate that the representation of serial order information relies on a broader set of neural areas and highlight the role of the intraparietal sulcus and the hippocampus, in addition to the supramarginal gyrus and the SMA. The contribution of different neural regions might reflect the involvement of distinct levels of serial order coding (i.e., spatial, attentional, temporal) that support the representation of serial order information.

Examining the transition of novel information toward familiarity

Throughout their lives, humans encounter multiple instances of new information that can be inconsistent with prior knowledge (novel). Over time, the once-novel information becomes integrated into their established knowledge base, shifting from novelty to familiarity. In this study, we investigated the processes by which the first steps of this transition take place. We hypothesized that the neural representations of initially novel items gradually change over the course of repeated presentations, expressing a shift toward familiarity. We further assumed that this shift could be traced by examining neural patterns using fMRI. In two experiments, while being scanned, participants read noun-adjective word pairs that were either consistent or inconsistent with their prior knowledge. Stimuli were repeated 3-6 times within the scans. Employing mass univariate and multivariate similarity analyses, we showed that the neural representations associated with the initial presentation of familiar versus novel objects differed in lateral frontal and temporal regions, the medial prefrontal cortex, and the medial temporal lobe. Importantly, the neural representations of novel stimuli gradually changed throughout repetitions until they became indistinguishable from their respective familiar items. We interpret these findings as indicating that an early phase of familiarization can be completed within a few repetitions. This initial familiarization can then serve as the prerequisite to the integration of novel items into existing knowledge. Future empirical and theoretical works can build on the current findings to develop a comprehensive model of the transition from novelty to familiarity.

The morning after the night before: Alcohol-induced blackouts impair next day recall in sober young adults

Binge-drinking in adolescents and young adults is a widespread problem, however, an often unreported consequence of binge-drinking behaviour is an alcohol-induced memory blackout (MBO). An MBO is a transient amnesic event resulting from rapid, excessive alcohol consumption. Here, we examine the short-term impact of an alcohol-induced MBO event (testing < 20 hours after blackout) on memory performance in people who have experienced a high volume of MBOs. In addition, we aimed to test the hypothesis that people who experience a high volume of MBOs may have poorer recall than non-blackout controls in either sober or intoxicated states. Three episodic memory paradigms consisting of free recall, serial recall, and depth of encoding tasks, were conducted by a group of alcohol drinkers who had never experienced a memory blackout, and those who reported at least 9 in the preceding 12-months. Studies were completed sober and after alcohol by all participants, and sober but after blackout by the experimental group. Accuracy of recall was assessed with linear mixed effects modelling for all experiments and conditions. Recall rate both before and after alcohol consumption was similar between groups, with poorer recall after drinking alcohol by all participants in all three studies. After blackout, MBO participants showed no significant improvement from their intoxicated state in serial recall and depth of encoding tasks, but an improvement in free recall. Further analysis of these findings revealed that 10 out of 23 participants showed significantly impaired performance after blackout during free recall, extending up to 17 participants in serial recall. In general, alcohol reduced recall rate in both blackout and control participants similarly, but recall following MBO remained poor. Our evidence suggests that alcohol-induced blackouts impair memory functioning the next day, and future research should establish the duration of deficits after an acute alcohol-induced blackout episode.

Synaptic learning rules for sequence learning

Remembering the temporal order of a sequence of events is a task easily performed by humans in everyday life, but the underlying neuronal mechanisms are unclear. This problem is particularly intriguing as human behavior often proceeds on a time scale of seconds, which is in stark contrast to the much faster millisecond time-scale of neuronal processing in our brains. One long-held hypothesis in sequence learning suggests that a particular temporal fine-structure of neuronal activity - termed 'phase precession' - enables the compression of slow behavioral sequences down to the fast time scale of the induction of synaptic plasticity. Using mathematical analysis and computer simulations, we find that - for short enough synaptic learning windows - phase precession can improve temporal-order learning tremendously and that the asymmetric part of the synaptic learning window is essential for temporal-order learning. To test these predictions, we suggest experiments that selectively alter phase precession or the learning window and evaluate memory of temporal order.

Medial prefrontal cortex and hippocampal activity differentially contribute to ordinal and temporal context retrieval during sequence memory

Remembering sequences of events defines episodic memory, but retrieval can be driven by both ordinality and temporal contexts. Whether these modes of retrieval operate at the same time or not remains unclear. Theoretically, medial prefrontal cortex (mPFC) confers ordinality, while the hippocampus (HC) associates events in gradually changing temporal contexts. Here, we looked for evidence of each with BOLD fMRI in a sequence task that taxes both retrieval modes. To test ordinal modes, items were transferred between sequences but retained their position (e.g., AB3). Ordinal modes activated mPFC, but not HC. To test temporal contexts, we examined items that skipped ahead across lag distances (e.g., ABD). HC, but not mPFC, tracked temporal contexts. There was a mPFC and HC by retrieval mode interaction. These current results suggest that the mPFC and HC are concurrently engaged in different retrieval modes in support of remembering when an event occurred.

Time cells in the human hippocampus and entorhinal cortex support episodic memory

The organization of temporal information is critical for the encoding and retrieval of episodic memories. In the rodent hippocampus and entorhinal cortex, evidence accumulated over the last decade suggests that populations of "time cells" in the hippocampus encode temporal information. We identify time cells in humans using intracranial microelectrode recordings obtained from 27 human epilepsy patients who performed an episodic memory task. We show that time cell activity predicts the temporal organization of retrieved memory items. We also uncover evidence of ramping cell activity in humans, which represents a complementary type of temporal information. These findings establish a cellular mechanism for the representation of temporal information in the human brain needed to form episodic memories.

SERIAL-ORDER recall in working memory across the cognitive spectrum of Parkinson's disease and neuroimaging correlates

We sought to determine if Parkinson's disease (PD) with mild cognitive impairment (MCI) is associated with a greater SERIAL-ORDER (mental manipulation) than ANY-ORDER (auditory span, storage) deficit in working memory (WM). We investigated WM combining neuropsychological measures with the study of brain functional connectivity. A cohort of 160 patients with idiopathic PD, classified as PD-MCI (n = 87) or PD with normal cognition (PD-NC; n = 73), and 70 matched healthy controls were studied. Verbal WM was assessed with the Backward Digit Span Task (BDT; Lamar et al., 2007, Neuropsychologia, 45, 245), measuring SERIAL-ORDER and ANY-ORDER recall. Resting-state MRI data were collected for 15 PD-MCI, 15 PD-NC and 30 controls. Hypothesis-driven seed-based functional connectivity of the dorsolateral prefrontal cortex (DLPFC) was compared between the three groups and correlated with BDT performance. We found the main effect of the test (impairment in SERIAL ORDER > ANY ORDER) and group ((NC = PD-NC) > PD-MCI) in BDT performance that was even more pronounced in SERIAL ORDER when controlling for ANY ORDER variability but not vice versa. Furthermore, PD-MCI compared to other groups were characterized by the functional disconnection between the bilateral DLPFC and the cerebellum. In functional correlations, DLPFC connectivity was positively related to both SERIAL- and ANY-ORDER performance. In conclusion, PD-MCI patients evidenced greater SERIAL-ORDER (manipulation and cognitive control) than ANY-ORDER (storage) working memory impairment than PD-NC and controls with a disrupted DLPFC resting-state connectivity that was also related to the verbal WM performance.

Behavioral evidence for memory replay of video episodes in the macaque

Humans recall the past by replaying fragments of events temporally. Here, we demonstrate a similar effect in macaques. We trained six rhesus monkeys with a temporal-order judgement (TOJ) task and collected 5000 TOJ trials. In each trial, the monkeys watched a naturalistic video of about 10 s comprising two across-context clips, and after a 2 s delay, performed TOJ between two frames from the video. The data are suggestive of a non-linear, time-compressed forward memory replay mechanism in the macaque. In contrast with humans, such compression of replay is, however, not sophisticated enough to allow these monkeys to skip over irrelevant information by compressing the encoded video globally. We also reveal that the monkeys detect event contextual boundaries, and that such detection facilitates recall by increasing the rate of information accumulation. Demonstration of a time-compressed, forward replay-like pattern in the macaque provides insights into the evolution of episodic memory in our lineage.

The hippocampus, prefrontal cortex, and perirhinal cortex are critical to incidental order memory

Considerable research in rodents and humans indicates the hippocampus and prefrontal cortex are essential for remembering temporal relationships among stimuli, and accumulating evidence suggests the perirhinal cortex may also be involved. However, experimental parameters differ substantially across studies, which limits our ability to fully understand the fundamental contributions of these structures. In fact, previous studies vary in the type of temporal memory they emphasize (e.g., order, sequence, or separation in time), the stimuli and responses they use (e.g., trial-unique or repeated sequences, and incidental or rewarded behavior), and the degree to which they control for potential confounding factors (e.g., primary and recency effects, or order memory deficits secondary to item memory impairments). To help integrate these findings, we developed a new paradigm testing incidental memory for trial-unique series of events, and concurrently assessed order and item memory in animals with damage to the hippocampus, prefrontal cortex, or perirhinal cortex. We found that this new approach led to robust order and item memory, and that hippocampal, prefrontal and perirhinal damage selectively impaired order memory. These findings suggest the hippocampus, prefrontal cortex and perirhinal cortex are part of a broad network of structures essential for incidentally learning the order of events in episodic memory.

Structured sequence processing and combinatorial binding: neurobiologically and computationally informed hypotheses

Understanding how the brain forms representations of structured information distributed in time is a challenging endeavour for the neuroscientific community, requiring computationally and neurobiologically informed approaches. The neural mechanisms for segmenting continuous streams of sensory input and establishing representations of dependencies remain largely unknown, as do the transformations and computations occurring between the brain regions involved in these aspects of sequence processing. We propose a blueprint for a neurobiologically informed and informing computational model of sequence processing (entitled: Vector-symbolic Sequencing of Binding INstantiating Dependencies, or VS-BIND). This model is designed to support the transformation of serially ordered elements in sensory sequences into structured representations of bound dependencies, readily operates on multiple timescales, and encodes or decodes sequences with respect to chunked items wherever dependencies occur in time. The model integrates established vector symbolic additive and conjunctive binding operators with neurobiologically plausible oscillatory dynamics, and is compatible with modern spiking neural network simulation methods. We show that the model is capable of simulating previous findings from structured sequence processing tasks that engage fronto-temporal regions, specifying mechanistic roles for regions such as prefrontal areas 44/45 and the frontal operculum during interactions with sensory representations in temporal cortex. Finally, we are able to make predictions based on the configuration of the model alone that underscore the importance of serial position information, which requires input from time-sensitive cells, known to reside in the hippocampus and dorsolateral prefrontal cortex. This article is part of the theme issue 'Towards mechanistic models of meaning composition'.

Causes and consequences of representational drift

The nervous system learns new associations while maintaining memories over long periods, exhibiting a balance between flexibility and stability. Recent experiments reveal that neuronal representations of learned sensorimotor tasks continually change over days and weeks, even after animals have achieved expert behavioral performance. How is learned information stored to allow consistent behavior despite ongoing changes in neuronal activity? What functions could ongoing reconfiguration serve? We highlight recent experimental evidence for such representational drift in sensorimotor systems, and discuss how this fits into a framework of distributed population codes. We identify recent theoretical work that suggests computational roles for drift and argue that the recurrent and distributed nature of sensorimotor representations permits drift while limiting disruptive effects. We propose that representational drift may create error signals between interconnected brain regions that can be used to keep neural codes consistent in the presence of continual change. These concepts suggest experimental and theoretical approaches to studying both learning and maintenance of distributed and adaptive population codes.