Immediate memory for "when, where and what": Short-delay retrieval using dynamic naturalistic material

Scene construction processes in the anterior hippocampus during temporal episodic memory retrieval

Although the hippocampus has been implicated in both the temporal organization of memories and association of scene elements, some theoretical accounts posit that the role of the hippocampus in episodic memory is largely atemporal. In this study, we set out to explore this discrepancy by identifying hippocampal activity patterns related to scene construction while participants performed a temporal order memory task. Participants in the fMRI scanner were shown a sequence of photographs, each consisting of a central object and a contextual background scene. On each retrieval trial, participants were shown a pair of the original photographs (FULL), objects from the scenes without the background (OBJ), or background contexts without the main foreground object (BACK). In the temporal order judgment (TOJ) task, participants judged the temporal order of the pair of scenes; in the Viewing trials, two identical scenes were shown without any task. First, we found that the anterior hippocampus-particularly the CA1 and subiculum-showed similar patterns of activation between the BACK and OBJ conditions, suggesting that scene construction occurred spontaneously during both TOJ and Viewing. Furthermore, neural markers of scene construction in the anterior hippocampus did not apply to incorrect trials, showing that successful temporal memory retrieval was functionally linked to scene construction. In the cortex, time-processing areas, such as the supplementary motor area and the precuneus, and scene-processing areas, such as the parahippocampal cortex, were activated and functionally connected with the hippocampus. Together, these results support the view that the hippocampus is concurrently involved in scene construction and temporal organization of memory and propose a model of hippocampal episodic memory that takes both processes into account.

Brain representations of space and time in episodic memory: A systematic review and meta-analysis

All experiences preserved within episodic memory contain information on the space and time of events. The hippocampus is the main brain region involved in processing spatial and temporal information for incorporation within episodic memory representations. However, the other brain regions involved in the encoding and retrieval of spatial and temporal information within episodic memory are unclear, because a systematic review of related studies is lacking and the findings are scattered. The present study was designed to integrate the results of functional magnetic resonance imaging and positron emission tomography studies by means of a systematic review and meta-analysis to provide converging evidence. In particular, we focused on identifying the brain regions involved in the retrieval of spatial and temporal information. We identified a spatial retrieval network consisting of the inferior temporal gyrus, parahippocampal gyrus, superior parietal lobule, angular gyrus, and precuneus. Temporal context retrieval was supported by the dorsolateral prefrontal cortex. Thus, the retrieval of spatial and temporal information is supported by different brain regions, highlighting their different natures within episodic memory.

The neural representations underlying asymmetric cross-modal prediction of words

Cross-modal prediction serves a crucial adaptive role in the multisensory world, yet the neural mechanisms underlying this prediction are poorly understood. The present study addressed this important question by combining a novel audiovisual sequence memory task, functional magnetic resonance imaging (fMRI), and multivariate neural representational analyses. Our behavioral results revealed a reliable asymmetric cross-modal predictive effect, with a stronger prediction from visual to auditory (VA) modality than auditory to visual (AV) modality. Mirroring the behavioral pattern, we found the superior parietal lobe (SPL) showed higher pattern similarity for VA than AV pairs, and the strength of the predictive coding in the SPL was positively correlated with the behavioral predictive effect in the VA condition. Representational connectivity analyses further revealed that the SPL mediated the neural pathway from the visual to the auditory cortex in the VA condition but was not involved in the auditory to visual cortex pathway in the AV condition. Direct neural pathways within the unimodal regions were found for the visual-to-visual and auditory-to-auditory predictions. Together, these results provide novel insights into the neural mechanisms underlying cross-modal sequence prediction.

Mapping the long-term delayed recall-based cortex-hippocampus network constrained by the structural and functional connectome: a case-control multimodal MRI study

Background

Connectome mapping may reveal new treatment targets for patients with neurological and psychiatric diseases. However, the long-term delayed recall based-network with structural and functional connectome is still largely unknown. Our objectives were to (1) identify the long-term delayed recall-based cortex-hippocampus network with structural and functional connectome and (2) investigate its relationships with various cognitive functions, age, and activities of daily living.

Methods

This case-control study enrolled 131 subjects (73 amnestic mild cognitive impairment [aMCI] patients and 58 age- and education-matched healthy controls [HCs]). All subjects completed a neuropsychological battery, activities of daily living assessment, and multimodal magnetic resonance imaging. Nodes of the cortical-hippocampal network related to long-term delayed recall were identified by probabilistic fiber tracking and functional connectivity (FC) analysis. Then, the main and interaction effects of the network on cognitive functions were assessed by a generalized linear model. Finally, the moderating effects of the network on the relationships between long-term delayed recall and clinical features were analyzed by multiple regression and Hayes’ bootstrap method. All the effects of cortex-hippocampus network were analyzed at the connectivity and network levels.

Results

The result of a generalized linear model showed that the bilateral hippocampus, left dorsolateral superior frontal gyrus, right supplementary motor area, left lingual gyrus, left superior occipital gyrus, left superior parietal gyrus, left precuneus, and right temporal pole (superior temporal gyrus) are the left and right cortex-hippocampus network nodes related to long-term delayed recall (P < 0.05). Significant interaction effects were found between the Auditory Verbal Learning Test Part 5 (AVLT 5) scores and global properties of the left cortex-hippocampus network [hierarchy, clustering coefficient, characteristic path length, global efficiency, local efficiency, Sigma and synchronization (P < 0.05 Bonferroni corrected)]. Significant interaction effects were found between the general cognitive function/executive function/language and global properties of the left cortex-hippocampus network [Sigma and synchronization (P < 0.05 Bonferroni corrected)].

Conclusion

This study introduces a novel symptom-based network and describes relationships among cognitive functions, brain function, and age. The cortex–hippocampus network constrained by the structural and functional connectome is closely related to long-term delayed recall.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13195-023-01197-7.

The Relational Trip Task, a novel ecological measure of relational memory: data from a schizophrenia sample

INTRODUCTION

Relational memory (RM) is severely impaired in schizophrenia. Unitisation can circumvent RM impairments in clinical populations as measured by the transverse-patterning (TP) task, a well-established measure of RM capacity. We compared memory performance on a new ecological RM measure, the Relational Trip Task (RTT), to that of TP at baseline and examined the effects of a unitisation intervention in RTT performance. RTT involves learning relational information of real-life stimuli, such as the relationship between people and places or objects.

METHODS

TP and RTT performances were examined in 45 individuals with schizophrenia. TP-impaired participants (n = 22) were randomised to either the intervention or an active control group. TP and RTT were administered again after unitisation training. Task validity and reliability were assessed. Intervention group's pre- and post-RTT accuracies were compared and contrasted to that in the control group.

RESULTS

RTT and TP were moderately correlated. TP non-learners had inferior performance in RTT at baseline. Improvement in RTT performance after unitisation training was observed in the intervention group; no pre-post improvement was observed in the control group.

CONCLUSION

RTT has an acceptable criterion validity and excellent alternate-form reliability. Unitisation seemed to be successfully generalized to support associations of real-life stimuli.

Stairway to memory: Left-hemispheric alpha dynamics index the progressive loading of items into a short-term store

The encoding, maintenance, and subsequent retrieval of memories over short time intervals is an essential cognitive function. Load effects on the neural dynamics supporting the maintenance of short-term memories have been well studied, but experimental design limitations have hindered the study of similar effects during the encoding of information into online memory stores. Theoretically, the active encoding of complex visual stimuli into memory must also recruit neural resources in a manner that scales with memory load. Understanding the neural systems supporting this encoding load effect is of particular importance, as some patient populations exhibit difficulties specifically with the encoding, and not the maintenance, of short-term memories. Using magnetoencephalography, a visual sequence memory paradigm, and a novel encoding slope analysis, we provide evidence for a left-lateralized network of regions, oscillating in the alpha frequency range, that exhibit a progressive loading effect of complex visual stimulus information during memory encoding. This progressive encoding load effect significantly tracked the eventual retrieval of item-order memories at the single trial level, and neural activity in these regions was functionally dissociated from that of earlier visual networks. These findings suggest that the active encoding of stimulus information into short-term stores recruits a left-lateralized network of frontal, parietal, and temporal regions, and might be susceptible to modulation (e.g., using non-invasive stimulation) in the alpha band.

Movies and narratives as naturalistic stimuli in neuroimaging

Using movies and narratives as naturalistic stimuli in human neuroimaging studies has yielded significant advances in understanding of cognitive and emotional functions. The relevant literature was reviewed, with emphasis on how the use of naturalistic stimuli has helped advance scientific understanding of human memory, attention, language, emotions, and social cognition in ways that would have been difficult otherwise. These advances include discovering a cortical hierarchy of temporal receptive windows, which supports processing of dynamic information that accumulates over several time scales, such as immediate reactions vs. slowly emerging patterns in social interactions. Naturalistic stimuli have also helped elucidate how the hippocampus supports segmentation and memorization of events in day-to-day life and have afforded insights into attentional brain mechanisms underlying our ability to adopt specific perspectives during natural viewing. Further, neuroimaging studies with naturalistic stimuli have revealed the role of the default-mode network in narrative-processing and in social cognition. Finally, by robustly eliciting genuine emotions, these stimuli have helped elucidate the brain basis of both basic and social emotions apparently manifested as highly overlapping yet distinguishable patterns of brain activity.

Behavioral evidence for memory replay of video episodes in the macaque

Humans recall the past by replaying fragments of events temporally. Here, we demonstrate a similar effect in macaques. We trained six rhesus monkeys with a temporal-order judgement (TOJ) task and collected 5000 TOJ trials. In each trial, the monkeys watched a naturalistic video of about 10 s comprising two across-context clips, and after a 2 s delay, performed TOJ between two frames from the video. The data are suggestive of a non-linear, time-compressed forward memory replay mechanism in the macaque. In contrast with humans, such compression of replay is, however, not sophisticated enough to allow these monkeys to skip over irrelevant information by compressing the encoded video globally. We also reveal that the monkeys detect event contextual boundaries, and that such detection facilitates recall by increasing the rate of information accumulation. Demonstration of a time-compressed, forward replay-like pattern in the macaque provides insights into the evolution of episodic memory in our lineage.

Apolipoprotein E ε4 Allele Is Associated with Reduced Retention of the "Where" Memory Component in Cognitively Intact Older Adults

OBJECTIVE

The present study investigated the effect of the apolipoprotein E (ApoE) ε4 allele on the four memory components (i.e., who, when, where, and what) among cognitively intact older adults.

METHODS

Participants comprised 47 cognitively intact older adults, who were classified into 2 groups based on the presence or absence of at least 1 ApoE ε4 allele. All participants completed standardized neuropsychological tests, including the Logical Memory subtest of the Wechsler Memory Scale-III with a revised scoring method.

RESULTS

The results revealed that recollection for each component followed a pattern of who > what > when = where. Furthermore, a significant group-by-component-by-condition interaction indicated that the presence of the ApoE ε4 allele resulted in a disproportionately detrimental effect on the where component retention in the verbal episodic memory task; this finding was significantly correlated with hippocampal volumes.

CONCLUSION

These results highlighted the importance of evaluating the subcomponents of verbal episodic memory to detect subtle cognitive differences related to ApoE ε4 status, which could help elucidate the mechanism behind the cascades caused by ApoE ε4 in the trajectories of cognitive aging.

Context-Dependent Coding of Temporal Distance Between Cinematic Events in the Human Precuneus

How temporal and contextual information interactively impact on behavior and brain activity during the retrieval of temporal order about naturalistic episodes remains incompletely understood. Here, we used fMRI to examine the effects of contextual signals derived from the content of the movie on the neural correlates underlying memory retrieval of temporal-order in human subjects of both sexes. By contrasting SAME versus DIFF storyline conditions during the retrieval of the temporal order of cinematic events, we found that the activation in the precuneus, as well as behavior, are significantly modulated according to storyline condition, supporting our prediction of contextual information contributing to temporal retrieval. We suggest that the precuneus engages in memory retrieval via reconstructive mechanisms, entailing search within a movie-specific, situational knowledge-structure. Furthermore, information-based analyses of multivoxel activity revealed that the precuneus also contains a context-independent linear representation of temporal distances, consistent with a chronological organization of memory traces. We thus put forward that the retrieval of the temporal-order of naturalistic events encoded in rich and dynamic contexts relies on the joint contribution of chronological and reconstructive mechanisms, both of which rely on the medioposterior parietal cortex in humans.SIGNIFICANCE STATEMENT Successful retrieval of episodic memory is dependent on both temporal and contextual signals. However, when contextual signals derived from multiple storylines or narratives are complex and intertwined, the behavioral and neural correlates underpinning the interplay between time and context is not completely understood. Here we characterized the activation level and multivoxel pattern of BOLD signals underlying the modulation of such contextual information during temporal order judgment in the precuneus. Our findings provide us with an elucidation of subprocesses implicating the medial parietal cortex in realizing temporal organization of episodic details.

Load effects on spatial working memory performance are linked to distributed alpha and beta oscillations

Increasing spatial working memory (SWM) load is generally associated with declines in behavioral performance, but the neural correlates of load-related behavioral effects remain poorly understood. Herein, we examine the alterations in oscillatory activity that accompany such performance changes in 22 healthy adults who performed a two- and four-load SWM task during magnetoencephalography (MEG). All MEG data were transformed into the time-frequency domain and significant oscillatory responses were imaged separately per load using a beamformer. Whole-brain correlation maps were computed using the load-related beamformer difference images and load-related accuracy effects on the SWM task. The results indicated that load-related differences in left inferior frontal alpha activity during encoding and maintenance were negatively correlated with load-related accuracy differences on the SWM task. That is, individuals who had more substantial decreases in prefrontal alpha during high-relative to low-load SWM trials tended to have smaller performance decrements on the high-load condition (i.e., they performed more accurately). The same pattern of neurobehavioral correlations was observed during the maintenance period for right superior temporal alpha activity and right superior parietal beta activity. Importantly, this is the first study to employ a voxel-wise whole-brain approach to significantly link load-related oscillatory differences and load-related SWM performance differences.

Slow Down: Behavioural and Physiological Effects of Reducing Eating Rate

Slowing eating rate appears to be an effective strategy for reducing food intake. This feasibility study investigated the effect of eating rate on post-meal responses using functional magnetic resonance imaging (fMRI), plasma gastrointestinal hormone concentrations, appetite ratings, memory for recent eating, and snack consumption. Twenty-one participants (mean age 23 years with healthy body mass index) were randomly assigned to consume a 600 kcal meal at either a “normal” or “slow” rate (6 vs. 24 min). Immediately afterwards, participants rated meal enjoyment and satisfaction. FMRI was performed 2-h post-meal during a memory task about the meal. Appetite, peptide YY, and ghrelin were measured at baseline and every 30 min for 3 h. Participants were given an ad-libitum snack three hours post-meal. Results were reported as effect sizes (Cohen’s d) due to the feasibility sample size. The normal rate group found the meal more enjoyable (effect size = 0.5) and satisfying (effect size = 0.6). Two hours post-meal, the slow rate group reported greater fullness (effect size = 0.7) and more accurate portion size memory (effect sizes = 0.4), with a linear relationship between time taken to make portion size decisions and the BOLD response in satiety and reward brain regions. Ghrelin suppression post-meal was greater in the slow rate group (effect size = 0.8). Three hours post-meal, the slow rate group consumed on average 25% less energy from snacks (effect size = 0.5). These data offer novel insights about mechanisms underlying how eating rate affects food intake and have implications for the design of effective weight-management interventions.

Beta Oscillatory Dynamics in the Prefrontal and Superior Temporal Cortices Predict Spatial Working Memory Performance

The oscillatory dynamics serving spatial working memory (SWM), and how such dynamics relate to performance, are poorly understood. To address these topics, the present study recruited 22 healthy adults to perform a SWM task during magnetoencephalography (MEG). The resulting MEG data were transformed into the time-frequency domain, and significant oscillatory responses were imaged using a beamformer. Voxel time series data were extracted from the cluster peaks to quantify the dynamics, while whole-brain partial correlation maps were computed to identify regions where oscillatory strength varied with accuracy on the SWM task. The results indicated transient theta oscillations in spatially distinct subregions of the prefrontal cortices at the onset of encoding and maintenance, which may underlie selection of goal-relevant information. Additionally, strong and persistent decreases in alpha and beta oscillations were observed throughout encoding and maintenance in parietal, temporal, and occipital regions, which could serve sustained attention and maintenance processes during SWM performance. The neuro-behavioral correlations revealed that beta activity within left dorsolateral prefrontal control regions and bilateral superior temporal integration regions was negatively correlated with SWM accuracy. Notably, this is the first study to employ a whole-brain approach to significantly link neural oscillations to behavioral performance in the context of SWM.

A Real-world What-Where-When Memory Test

Episodic memory is a complex memory system which allows recall and mental re-experience of previous episodes from one's own life. Real-life episodic memories are about events in their spatiotemporal context and are typically visuospatial, rather than verbal. Yet often, tests of episodic memory use verbal material to be recalled (word lists, stories). The Real-World What-Where-When memory test requires participants to hide a total of 16 different objects in 16 different locations over two temporal occasions, 2 h apart. Another two hours later, they are then asked to recall which objects (What) they had hidden in which locations (Where) and on which of the two occasions (When). In addition to counting the number of correctly recalled complete what-where-when combinations, this task can also be used to test real-world spatial memory and object memory. This task is sensitive to normal cognitive aging, and correlates well with performance on other episodic memory tasks, while at the same time providing more ecological validity and being cheap and easy to run.

Obesity and insulin resistance are associated with reduced activity in core memory regions of the brain

Increasing research in animals and humans suggests that obesity may be associated with learning and memory deficits, and in particular with reductions in episodic memory. Rodent models have implicated the hippocampus in obesity-related memory impairments, but the neural mechanisms underlying episodic memory deficits in obese humans remain undetermined. In the present study, lean and obese human participants were scanned using fMRI while completing a What-Where-When episodic memory test (the “Treasure-Hunt Task”) that assessed the ability to remember integrated item, spatial, and temporal details of previously encoded complex events. In lean participants, the Treasure-Hunt task elicited significant activity in regions of the brain known to be important for recollecting episodic memories, such as the hippocampus, angular gyrus, and dorsolateral prefrontal cortex. Both obesity and insulin resistance were associated with significantly reduced functional activity throughout the core recollection network. These findings indicate that obesity is associated with reduced functional activity in core brain areas supporting episodic memory and that insulin resistance may be a key player in this association.

•

Obesity associated with reduced activity in core recollection network during episodic memory.

•

Insulin resistance associated with reduced activity in core recollection network during episodic memory.

•

Insulin resistance, but not obesity, associated with poorer memory performance.

Aberrant Functional Connectivity between the Amygdala and the Temporal Pole in Drug-Free Generalized Anxiety Disorder

The amygdala and the dorsolateral prefrontal cortex (DLPFC) play important roles in “emotion dysregulation,” which has a profound impact on etiologic research of generalized anxiety disorder (GAD). The present study analyzed both eyes-open and eyes-closed resting state functional MRI (rs-fMRI) of 43 subjects (21 GAD patients with medicine free and 22 matched healthy controls). The amygdala and the DLPFC were defined as regions of interest (ROI) to analyze functional connectivity (FC) in GAD patients compared with healthy controls. The main findings revealed GAD patients had increased FC between the amygdala and the temporal pole compared to healthy controls, which was found in both eyes-open and eyes-closed rs-fMRI. And altered FC between the ROIs and brain regions that mainly belonged to the default mode network (DMN) were found. These findings suggest that the abnormal FC between the amygdala and the temporal pole may contribute to the pathophysiology of GAD, and provide insights into the current understanding of the emotion dysregulation of anxiety disorders.

Language in Context: MEG Evidence for Modality-General and -Specific Responses to Reference Resolution

Successful language comprehension critically depends on our ability to link linguistic expressions to the entities they refer to. Without reference resolution, newly encountered language cannot be related to previously acquired knowledge. The human experience includes many different types of referents, some visual, some auditory, some very abstract. Does the neural basis of reference resolution depend on the nature of the referents, or do our brains use a modality-general mechanism for linking meanings to referents? Here we report evidence for both. Using magnetoencephalography (MEG), we varied both the modality of referents, which consisted either of visual or auditory objects, and the point at which reference resolution was possible within sentences. Source-localized MEG responses revealed brain activity associated with reference resolution that was independent of the modality of the referents, localized to the medial parietal lobe and starting ∼415 ms after the onset of reference resolving words. A modality-specific response to reference resolution in auditory domains was also found, in the vicinity of auditory cortex. Our results suggest that referential language processing cannot be reduced to processing in classical language regions and representations of the referential domain in modality-specific neural systems. Instead, our results suggest that reference resolution engages medial parietal cortex, which supports a mechanism for referential processing regardless of the content modality.

Differential Neural Correlates Underlie Judgment of Learning and Subsequent Memory Performance

Judgment of learning (JOL) plays a pivotal role in self-regulated learning. Although the JOLs are in general accurate, important deviations from memory performance are often reported, especially when the JOLs are made immediately after learning. Nevertheless, existing studies have not clearly dissociated the neural processes underlying subjective JOL and objective memory. In the present study, participants were asked to study a list of words that would be tested 1 day later. Immediately after learning an item, participants predicted how likely they would remember that item. Critically, the JOL was performed on only half of the studied items to avoid its contamination on subsequent memory. We found that during encoding, compared to items later judged as "will be forgotten," those judged as "will be remembered" showed stronger activities in the default-mode network, including the ventromedial prefrontal cortex (PFC) and posterior cingulate cortex, as well as weaker functional connectivity between the left dorsolateral PFC and the visual cortex. The exact opposite pattern was found when comparing items that were actually remembered with those that were later forgotten. These important neural dissociations between JOL and memory performance shed light on the neural mechanisms of human metamemory bias.

Episodic memory in aspects of large-scale brain networks

Understanding human episodic memory in aspects of large-scale brain networks has become one of the central themes in neuroscience over the last decade. Traditionally, episodic memory was regarded as mostly relying on medial temporal lobe (MTL) structures. However, recent studies have suggested involvement of more widely distributed cortical network and the importance of its interactive roles in the memory process. Both direct and indirect neuro-modulations of the memory network have been tried in experimental treatments of memory disorders. In this review, we focus on the functional organization of the MTL and other neocortical areas in episodic memory. Task-related neuroimaging studies together with lesion studies suggested that specific sub-regions of the MTL are responsible for specific components of memory. However, recent studies have emphasized that connectivity within MTL structures and even their network dynamics with other cortical areas are essential in the memory process. Resting-state functional network studies also have revealed that memory function is subserved by not only the MTL system but also a distributed network, particularly the default-mode network (DMN). Furthermore, researchers have begun to investigate memory networks throughout the entire brain not restricted to the specific resting-state network (RSN). Altered patterns of functional connectivity (FC) among distributed brain regions were observed in patients with memory impairments. Recently, studies have shown that brain stimulation may impact memory through modulating functional networks, carrying future implications of a novel interventional therapy for memory impairment.

Adaptability to changes in temporal structure is fornix-dependent

Recognition memory deficits, even after short delays, are sometimes observed following hippocampal damage. One hypothesis links the hippocampus with processes in updating contextual memory representation. Here, we used fornix transection, which partially disconnects the hippocampal system, and compares the performance of fornix-transected monkeys with normal monkeys on two versions of a delayed-matching-to-position task with short delays. Spatial recognition memory was affected by fornix transection only when the temporal structure of the task changed across trials, while differences in motor control, motivation, perception, or short-term memory were not critical. We attributed the deficit to a compromised ability in tracking changes in task temporal structure.

Scale invariance of temporal order discrimination using complex, naturalistic events

•

RTs in temporal-order judgement increase with temporal similarity between events.

•

Such relation holds across 4 experiments with varying delays and temporal scales.

•

Bayesian evidence for scale-invariance in temporal-order retrieval of memory.

Recent demonstrations of scale invariance in cognitive domains prompted us to investigate whether a scale-free pattern might exist in retrieving the temporal order of events from episodic memory. We present four experiments using an encoding-retrieval paradigm with naturalistic stimuli (movies or video clips). Our studies show that temporal order judgement retrieval times were negatively correlated with the temporal separation between two events in the movie. This relation held, irrespective of whether temporal distances were on the order of tens of minutes (Exp 1−2) or just a few seconds (Exp 3−4). Using the SIMPLE model, we factored in the retention delays between encoding and retrieval (delays of 24 h, 15 min, 1.5–2.5 s, and 0.5 s for Exp 1–4, respectively) and computed a temporal similarity score for each trial. We found a positive relation between similarity and retrieval times; that is, the more temporally similar two events, the slower the retrieval of their temporal order. Using Bayesian analysis, we confirmed the equivalence of the RT/similarity relation across all experiments, which included a vast range of temporal distances and retention delays. These results provide evidence for scale invariance during the retrieval of temporal order of episodic memories.