Reinstatement of memory representations for lifelike events over the course of a week

When we remember an event, the content of that memory is represented across the brain. Detailed memory retrieval is thought to involve the reinstatement of those representations. Functional MRI combined with representational similarity analyses (RSA) of spatial patterns of brain activity has revealed reinstatement of recently-experienced events throughout a core memory retrieval network. In the present study, participants were scanned while they watched, immediately retrieved and then retrieved after a week, 24 short videos. Following the delayed retrieval, they freely recalled all videos outside of the scanner. We observed widespread within- and between-subject reinstatement effects within a posterior midline core memory retrieval network during all phases of the experiment. Within precuneus, bilateral middle temporal gyrus and the left hippocampus, reinstatement effects between the retrieval phases correlated with memory performance. These findings extend previous studies that have only employed short retention periods or highly rehearsed materials, demonstrating that memory representations for unique events are reliably reinstated over longer timeframes that are meaningful in the context of real-world episodic memory.

Computational approaches to fMRI analysis

Analysis methods in cognitive neuroscience have not always matched the richness of fMRI data. Early methods focused on estimating neural activity within individual voxels or regions, averaged over trials or blocks and modeled separately in each participant. This approach mostly neglected the distributed nature of neural representations over voxels, the continuous dynamics of neural activity during tasks, the statistical benefits of performing joint inference over multiple participants and the value of using predictive models to constrain analysis. Several recent exploratory and theory-driven methods have begun to pursue these opportunities. These methods highlight the importance of computational techniques in fMRI analysis, especially machine learning, algorithmic optimization and parallel computing. Adoption of these techniques is enabling a new generation of experiments and analyses that could transform our understanding of some of the most complex-and distinctly human-signals in the brain: acts of cognition such as thoughts, intentions and memories.

Reconstructing Perceived and Retrieved Faces from Activity Patterns in Lateral Parietal Cortex

Recent findings suggest that the contents of memory encoding and retrieval can be decoded from the angular gyrus (ANG), a subregion of posterior lateral parietal cortex. However, typical decoding approaches provide little insight into the nature of ANG content representations. Here, we tested whether complex, multidimensional stimuli (faces) could be reconstructed from ANG by predicting underlying face components from fMRI activity patterns in humans. Using an approach inspired by computer vision methods for face recognition, we applied principal component analysis to a large set of face images to generate eigenfaces. We then modeled relationships between eigenface values and patterns of fMRI activity. Activity patterns evoked by individual faces were then used to generate predicted eigenface values, which could be transformed into reconstructions of individual faces. We show that visually perceived faces were reliably reconstructed from activity patterns in occipitotemporal cortex and several lateral parietal subregions, including ANG. Subjective assessment of reconstructed faces revealed specific sources of information (e.g., affect and skin color) that were successfully reconstructed in ANG. Strikingly, we also found that a model trained on ANG activity patterns during face perception was able to successfully reconstruct an independent set of face images that were held in memory. Together, these findings provide compelling evidence that ANG forms complex, stimulus-specific representations that are reflected in activity patterns evoked during perception and remembering.

SIGNIFICANCE STATEMENT

Neuroimaging studies have consistently implicated lateral parietal cortex in episodic remembering, but the functional contributions of lateral parietal cortex to memory remain a topic of debate. Here, we used an innovative form of fMRI pattern analysis to test whether lateral parietal cortex actively represents the contents of memory. Using a large set of human face images, we first extracted latent face components (eigenfaces). We then used machine learning algorithms to predict face components from fMRI activity patterns and, ultimately, to reconstruct images of individual faces. We show that activity patterns in a subregion of lateral parietal cortex, the angular gyrus, supported successful reconstruction of perceived and remembered faces, confirming a role for this region in actively representing remembered content.

Lower Parietal Encoding Activation Is Associated with Sharper Information and Better Memory

Mean fMRI activation in ventral posterior parietal cortex (vPPC) during memory encoding often negatively predicts successful remembering. A popular interpretation of this phenomenon is that vPPC reflects "off-task" processing. However, recent fMRI studies considering distributed patterns of activity suggest that vPPC actively represents encoded material. Here, we assessed the relationships between pattern-based content representations in vPPC, mean activation in vPPC, and subsequent remembering. We analyzed data from two fMRI experiments where subjects studied then recalled word-face or word-scene associations. For each encoding trial, we measured 1) mean univariate activation within vPPC and 2) the strength of face/scene information as indexed by pattern analysis. Mean activation in vPPC negatively predicted subsequent remembering, but the strength of pattern-based information in the same vPPC voxels positively predicted later memory. Indeed, univariate amplitude averaged across vPPC voxels negatively correlated with pattern-based information strength. This dissociation reflected a tendency for univariate reductions to maximally occur in voxels that were not strongly tuned for the category of encoded stimuli. These results indicate that vPPC activity patterns reflect the content and quality of memory encoding and constitute a striking example of lower univariate activity corresponding to stronger pattern-based information.

Meta-analytic comparison of trial- versus questionnaire-based vividness reportability across behavioral, cognitive and neural measurements of imagery

Vividness is an aspect of consciousness related to mental imagery and prospective episodic memory. Despite being harshly criticized in the past for failing to demonstrate robust correlations with behavioral measures, currently this construct is attracting a resurgent interest in cognitive neuroscience. Therefore, an updated examination of the validity of this construct is timely. A corpus of peer-reviewed literature was analyzed through meta-analysis, which compared the two main formats used to measure vividness [trial-by-trial vividness ratings (VR) and the Vividness of Visual Imagery Questionnaire (VVIQ)]. These two formats were compared in relation to all available behavioral/cognitive (BC) and neuroscience (NS) measures in Phase 1 (3542 statistical observations representing 393 journal articles); and then in relation to all available BC, EEG and fMRI literature in Phase 2 (3624 observations representing 402 articles). Both Phases observed significantly larger effect size estimates (ESEs) for VR than VVIQ, and larger ESEs for NS than BC measures. ESEs for EEG and fMRI were not significantly different in Phase 2, but were greater than BC ESEs. These data suggest VR are a more reliable self-report measure than VVIQ, and may reflect a more direct route of reportability than the latter. Furthermore, both VR and VVIQ are more strongly associated with the neural, than the cognitive and behavioural correlates of imagery. If one establishes neuroscience measures as the criterion variable, then self-reports of vividness show higher construct validity than behavioural/cognitive measures of imagery. We discuss how the present findings contribute to current issues on measurement of reportability; and how this study advances our understanding of vividness as a phenomenological characteristic of imagery, and other forms of conscious experience which do not necessarily involve imagery.

Shared memories reveal shared structure in neural activity across individuals

Our lives revolve around sharing experiences and memories with others. When different people recount the same events, how similar are their underlying neural representations? Participants viewed a 50-min movie, then verbally described the events during functional MRI, producing unguided detailed descriptions lasting up to 40 min. As each person spoke, event-specific spatial patterns were reinstated in default-network, medial-temporal, and high-level visual areas. Individual event patterns were both highly discriminable from one another and similar among people, suggesting consistent spatial organization. In many high-order areas, patterns were more similar between people recalling the same event than between recall and perception, indicating systematic reshaping of percept into memory. These results reveal the existence of a common spatial organization for memories in high-level cortical areas, where encoded information is largely abstracted beyond sensory constraints, and that neural patterns during perception are altered systematically across people into shared memory representations for real-life events.

Decoding the content of recollection within the core recollection network and beyond

Recollection - retrieval of qualitative information about a past event - is associated with enhanced neural activity in a consistent set of neural regions (the 'core recollection network') seemingly regardless of the nature of the recollected content. Here, we employed multi-voxel pattern analysis (MVPA) to assess whether retrieval-related functional magnetic resonance imaging (fMRI) activity in core recollection regions - including the hippocampus, angular gyrus, medial prefrontal cortex, retrosplenial/posterior cingulate cortex, and middle temporal gyrus - contain information about studied content and thus demonstrate retrieval-related 'reinstatement' effects. During study, participants viewed objects and concrete words that were subjected to different encoding tasks. Test items included studied words, the names of studied objects, or unstudied words. Participants judged whether the items were recollected, familiar, or new by making 'remember', 'know', and 'new' responses, respectively. The study history of remembered test items could be reliably decoded using MVPA in most regions, as well as from the dorsolateral prefrontal cortex, a region where univariate recollection effects could not be detected. The findings add to evidence that members of the core recollection network, as well as at least one neural region where mean signal is insensitive to recollection success, carry information about recollected content. Importantly, the study history of recognized items endorsed with a 'know' response could be decoded with equal accuracy. The results thus demonstrate a striking dissociation between mean signal and multi-voxel indices of recollection. Moreover, they converge with prior findings in suggesting that, as it is operationalized by classification-based MVPA, reinstatement is not uniquely a signature of recollection.

Distinct neural mechanisms underlie the success, precision, and vividness of episodic memory

A network of brain regions have been linked with episodic memory retrieval, but limited progress has been made in identifying the contributions of distinct parts of the network. Here, we utilized continuous measures of retrieval to dissociate three components of episodic memory: retrieval success, precision, and vividness. In the fMRI scanner, participants encoded objects that varied continuously on three features: color, orientation, and location. Participants’ memory was tested by having them recreate the appearance of the object features using a continuous dial, and continuous vividness judgments were recorded. Retrieval success, precision, and vividness were dissociable both behaviorally and neurally: successful versus unsuccessful retrieval was associated with hippocampal activity, retrieval precision scaled with activity in the angular gyrus, and vividness judgments tracked activity in the precuneus. The ability to dissociate these components of episodic memory reveals the benefit afforded by measuring memory on a continuous scale, allowing functional parcellation of the retrieval network.

DOI:http://dx.doi.org/10.7554/eLife.18260.001

Remembering is something we do countless times each day. The detail and vividness with which we can remember is part of what makes memories so precious. Given the significance and complexity of memories, it is perhaps unsurprising that several parts of the brain are needed for us to experience them. Indeed, the brain regions involved in memory all work so closely together that it is a challenge to identify what role each region plays.

Richter, Cooper et al. aimed to design a memory task that could separate key characteristics of remembering, which would allow them to study links between each aspect and the different brain regions involved in memory. The resulting test involved showing people images of different objects whilst they were in an MRI medical imaging scanner. The people taking the test were asked to remember several objects that could vary in color, position and orientation. Participants were asked to rate how vividly they remembered the objects and then tried to precisely recreate their color, orientation and position.

The test allowed Richter, Cooper et al. to link specific parts of the brain to certain aspects of remembering. The hippocampus, an area known to be important in memory processing, indicated whether or not information had been remembered. More vivid memories were linked to greater activity in a region called the precuneus, which plays a role in imagination. Lastly, activity in a third region – the angular gyrus – indicated the precision of each memory.

Being able to study different aspects of memory using tests like this that collect detailed measurements could be important in identifying memory problems, for example, in people with brain diseases or head injuries, or after a stroke. Specifically, the methods developed by Richter, Cooper et al. could provide sensitive tools for detecting memory difficulties at an early stage. This may help more people to get treated sooner, potentially minimizing lasting complications.

DOI:http://dx.doi.org/10.7554/eLife.18260.002

Similarities and differences in the default mode network across rest, retrieval, and future imagining

The default mode network (DMN) has been identified reliably during rest, as well as during the performance of tasks such as episodic retrieval and future imagining. It remains unclear why this network is engaged across these seemingly distinct conditions, though many hypotheses have been proposed to account for these effects. Prior to generating hypotheses explaining common DMN involvement, the degree of commonality in the DMN across these conditions, within individuals, must be statistically determined to test whether or not the DMN is truly a unitary network, equally engaged across rest, retrieval and future imagining. To provide such a test, we used comparable paradigms (self-directed, uninterrupted thought of equal duration) across the three conditions (rest, retrieval, and future imagining) in a within-participant design. We found lower than expected pattern similarity in DMN functional connectivity across the three conditions. Similarity in connectivity accounted for only 40-50% of the total variance. Partial Least Squares (PLS) analyses revealed the medial temporal regions of the DMN were preferentially coupled with one another during episodic retrieval and future imagining, whereas the non-medial temporal regions of the DMN (e.g., medial prefrontal cortex, lateral temporal cortex, and temporal pole) were preferentially coupled during rest. These results suggest that DMN connectivity may be more flexible than previously considered. Our findings are in line with emerging evidence that the DMN is not a static network engaged commonly across distinct cognitive processes, but is instead a dynamic system, topographically changing in relation to ongoing cognitive demands. Hum Brain Mapp 38:1155-1171, 2017. © 2016 Wiley Periodicals, Inc.

Decoding fMRI Signatures of Real-world Autobiographical Memory Retrieval

Extant neuroimaging data implicate frontoparietal and medial-temporal lobe regions in episodic retrieval, and the specific pattern of activity within and across these regions is diagnostic of an individual's subjective mnemonic experience. For example, in laboratory-based paradigms, memories for recently encoded faces can be accurately decoded from single-trial fMRI patterns [Uncapher, M. R., Boyd-Meredith, J. T., Chow, T. E., Rissman, J., & Wagner, A. D. Goal-directed modulation of neural memory patterns: Implications for fMRI-based memory detection. Journal of Neuroscience, 35, 8531–8545, 2015; Rissman, J., Greely, H. T., & Wagner, A. D. Detecting individual memories through the neural decoding of memory states and past experience. Proceedings of the National Academy of Sciences, U.S.A., 107, 9849–9854, 2010]. Here, we investigated the neural patterns underlying memory for real-world autobiographical events, probed at 1- to 3-week retention intervals as well as whether distinct patterns are associated with different subjective memory states. For 3 weeks, participants (n = 16) wore digital cameras that captured photographs of their daily activities. One week later, they were scanned while making memory judgments about sequences of photos depicting events from their own lives or events captured by the cameras of others. Whole-brain multivoxel pattern analysis achieved near-perfect accuracy at distinguishing correctly recognized events from correctly rejected novel events, and decoding performance did not significantly vary with retention interval. Multivoxel pattern classifiers also differentiated recollection from familiarity and reliably decoded the subjective strength of recollection, of familiarity, or of novelty. Classification-based brain maps revealed dissociable neural signatures of these mnemonic states, with activity patterns in hippocampus, medial PFC, and ventral parietal cortex being particularly diagnostic of recollection. Finally, a classifier trained on previously acquired laboratory-based memory data achieved reliable decoding of autobiographical memory states. We discuss the implications for neuroscientific accounts of episodic retrieval and comment on the potential forensic use of fMRI for probing experiential knowledge.