1
|
Chen YY, Areti A, Yoshor D, Foster BL. Perception and Memory Reinstatement Engage Overlapping Face-Selective Regions within Human Ventral Temporal Cortex. J Neurosci 2024; 44:e2180232024. [PMID: 38627090 PMCID: PMC11140664 DOI: 10.1523/jneurosci.2180-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 04/03/2024] [Accepted: 04/05/2024] [Indexed: 04/19/2024] Open
Abstract
Humans have the remarkable ability to vividly retrieve sensory details of past events. According to the theory of sensory reinstatement, during remembering, brain regions specialized for processing specific sensory stimuli are reactivated to support content-specific retrieval. Recently, several studies have emphasized transformations in the spatial organization of these reinstated activity patterns. Specifically, studies of scene stimuli suggest a clear anterior shift in the location of retrieval activations compared with the activity observed during perception. However, it is not clear that such transformations occur universally, with inconsistent evidence for other important stimulus categories, particularly faces. One challenge in addressing this question is the careful delineation of face-selective cortices, which are interdigitated with other selective regions, in configurations that spatially differ across individuals. Therefore, we conducted a multisession neuroimaging study to first carefully map individual participants' (nine males and seven females) face-selective regions within ventral temporal cortex (VTC), followed by a second session to examine the activity patterns within these regions during face memory encoding and retrieval. While face-selective regions were expectedly engaged during face perception at encoding, memory retrieval engagement exhibited a more selective and constricted reinstatement pattern within these regions, but did not show any consistent direction of spatial transformation (e.g., anteriorization). We also report on unique human intracranial recordings from VTC under the same experimental conditions. These findings highlight the importance of considering the complex configuration of category-selective cortex in elucidating principles shaping the neural transformations that occur from perception to memory.
Collapse
Affiliation(s)
- Yvonne Y Chen
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104
| | | | - Daniel Yoshor
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104
| | - Brett L Foster
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104
| |
Collapse
|
2
|
Morales-Torres R, Wing EA, Deng L, Davis SW, Cabeza R. Visual Recognition Memory of Scenes Is Driven by Categorical, Not Sensory, Visual Representations. J Neurosci 2024; 44:e1479232024. [PMID: 38569925 PMCID: PMC11112637 DOI: 10.1523/jneurosci.1479-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Revised: 02/07/2024] [Accepted: 02/14/2024] [Indexed: 04/05/2024] Open
Abstract
When we perceive a scene, our brain processes various types of visual information simultaneously, ranging from sensory features, such as line orientations and colors, to categorical features, such as objects and their arrangements. Whereas the role of sensory and categorical visual representations in predicting subsequent memory has been studied using isolated objects, their impact on memory for complex scenes remains largely unknown. To address this gap, we conducted an fMRI study in which female and male participants encoded pictures of familiar scenes (e.g., an airport picture) and later recalled them, while rating the vividness of their visual recall. Outside the scanner, participants had to distinguish each seen scene from three similar lures (e.g., three airport pictures). We modeled the sensory and categorical visual features of multiple scenes using both early and late layers of a deep convolutional neural network. Then, we applied representational similarity analysis to determine which brain regions represented stimuli in accordance with the sensory and categorical models. We found that categorical, but not sensory, representations predicted subsequent memory. In line with the previous result, only for the categorical model, the average recognition performance of each scene exhibited a positive correlation with the average visual dissimilarity between the item in question and its respective lures. These results strongly suggest that even in memory tests that ostensibly rely solely on visual cues (such as forced-choice visual recognition with similar distractors), memory decisions for scenes may be primarily influenced by categorical rather than sensory representations.
Collapse
Affiliation(s)
| | - Erik A Wing
- Rotman Research Institute, Baycrest Health Sciences, Toronto, Ontario M6A 2E1, Canada
| | - Lifu Deng
- Department of Psychology & Neuroscience, Duke University, Durham, North Carolina 27708
| | - Simon W Davis
- Department of Psychology & Neuroscience, Duke University, Durham, North Carolina 27708
- Department of Neurology, Duke University School of Medicine, Durham, North Carolina 27708
| | - Roberto Cabeza
- Department of Psychology & Neuroscience, Duke University, Durham, North Carolina 27708
| |
Collapse
|
3
|
Woodry R, Curtis CE, Winawer J. Feedback scales the spatial tuning of cortical responses during visual memory. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.11.589111. [PMID: 38659957 PMCID: PMC11042180 DOI: 10.1101/2024.04.11.589111] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/26/2024]
Abstract
Perception, working memory, and long-term memory each evoke neural responses in visual cortex, suggesting that memory uses encoding mechanisms shared with perception. While previous research has largely focused on how perception and memory are similar, we hypothesized that responses in visual cortex would differ depending on the origins of the inputs. Using fMRI, we quantified spatial tuning in visual cortex while participants (both sexes) viewed, maintained in working memory, or retrieved from long-term memory a peripheral target. In each of these conditions, BOLD responses were spatially tuned and were aligned with the target's polar angle in all measured visual field maps including V1. As expected given the increasing sizes of receptive fields, polar angle tuning during perception increased in width systematically up the visual hierarchy from V1 to V2, V3, hV4, and beyond. In stark contrast, the widths of tuned responses were broad across the visual hierarchy during working memory and long-term memory, matched to the widths in perception in later visual field maps but much broader in V1. This pattern is consistent with the idea that mnemonic responses in V1 stem from top-down sources. Moreover, these tuned responses when biased (clockwise or counterclockwise of target) predicted matched biases in memory, suggesting that the readout of maintained and reinstated mnemonic responses influences memory guided behavior. We conclude that feedback constrains spatial tuning during memory, where earlier visual maps inherit broader tuning from later maps thereby impacting the precision of memory.
Collapse
Affiliation(s)
- Robert Woodry
- Department of Psychology, New York University, New York City, NY 10003
| | - Clayton E. Curtis
- Department of Psychology, New York University, New York City, NY 10003
- Center for Neural Science, New York University, New York City, NY 10003
| | - Jonathan Winawer
- Department of Psychology, New York University, New York City, NY 10003
- Center for Neural Science, New York University, New York City, NY 10003
| |
Collapse
|
4
|
Huang S, Howard CM, Hovhannisyan M, Ritchey M, Cabeza R, Davis SW. Hippocampal Functions Modulate Transfer-Appropriate Cortical Representations Supporting Subsequent Memory. J Neurosci 2024; 44:e1135232023. [PMID: 38050089 PMCID: PMC10851689 DOI: 10.1523/jneurosci.1135-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Revised: 09/12/2023] [Accepted: 10/14/2023] [Indexed: 12/06/2023] Open
Abstract
The hippocampus plays a central role as a coordinate system or index of information stored in neocortical loci. Nonetheless, it remains unclear how hippocampal processes integrate with cortical information to facilitate successful memory encoding. Thus, the goal of the current study was to identify specific hippocampal-cortical interactions that support object encoding. We collected fMRI data while 19 human participants (7 female and 12 male) encoded images of real-world objects and tested their memory for object concepts and image exemplars (i.e., conceptual and perceptual memory). Representational similarity analysis revealed robust representations of visual and semantic information in canonical visual (e.g., occipital cortex) and semantic (e.g., angular gyrus) regions in the cortex, but not in the hippocampus. Critically, hippocampal functions modulated the mnemonic impact of cortical representations that are most pertinent to future memory demands, or transfer-appropriate representations Subsequent perceptual memory was best predicted by the strength of visual representations in ventromedial occipital cortex in coordination with hippocampal activity and pattern information during encoding. In parallel, subsequent conceptual memory was best predicted by the strength of semantic representations in left inferior frontal gyrus and angular gyrus in coordination with either hippocampal activity or semantic representational strength during encoding. We found no evidence for transfer-incongruent hippocampal-cortical interactions supporting subsequent memory (i.e., no hippocampal interactions with cortical visual/semantic representations supported conceptual/perceptual memory). Collectively, these results suggest that diverse hippocampal functions flexibly modulate cortical representations of object properties to satisfy distinct future memory demands.Significance Statement The hippocampus is theorized to index pieces of information stored throughout the cortex to support episodic memory. Yet how hippocampal processes integrate with cortical representation of stimulus information remains unclear. Using fMRI, we examined various forms of hippocampal-cortical interactions during object encoding in relation to subsequent performance on conceptual and perceptual memory tests. Our results revealed novel hippocampal-cortical interactions that utilize semantic and visual representations in transfer-appropriate manners: conceptual memory supported by hippocampal modulation of frontoparietal semantic representations, and perceptual memory supported by hippocampal modulation of occipital visual representations. These findings provide important insights into the neural mechanisms underlying the formation of information-rich episodic memory and underscore the value of studying the flexible interplay between brain regions for complex cognition.
Collapse
Affiliation(s)
- Shenyang Huang
- Department of Psychology & Neuroscience, Duke University, Durham 27708, North Carolina
| | - Cortney M Howard
- Department of Psychology & Neuroscience, Duke University, Durham 27708, North Carolina
| | | | - Maureen Ritchey
- Department of Psychology, Boston College, 02467 Massachusetts
| | - Roberto Cabeza
- Department of Psychology & Neuroscience, Duke University, Durham 27708, North Carolina
| | - Simon W Davis
- Department of Psychology & Neuroscience, Duke University, Durham 27708, North Carolina
- Department of Neurology, Duke University School of Medicine, Durham 27708, North Carolina
| |
Collapse
|
5
|
Naspi L, Stensholt C, Karlsson AE, Monge ZA, Cabeza R. Effects of Aging on Successful Object Encoding: Enhanced Semantic Representations Compensate for Impaired Visual Representations. J Neurosci 2023; 43:7337-7350. [PMID: 37673674 PMCID: PMC10621770 DOI: 10.1523/jneurosci.2265-22.2023] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 08/09/2023] [Accepted: 08/13/2023] [Indexed: 09/08/2023] Open
Abstract
Although episodic memory and visual processing decline substantially with healthy aging, semantic knowledge is generally spared. There is evidence that older adults' spared semantic knowledge can support episodic memory. Here, we used functional magnetic resonance imaging (fMRI) combined with representational similarity analyses (RSAs) to examine how novel visual and preexisting semantic representations at encoding predict subjective memory vividness at retrieval. Eighteen young and seventeen older adults (female and male participants) encoded images of objects during fMRI scanning and recalled these images while rating the vividness of their memories. After scanning, participants discriminated between studied images and similar lures. RSA based on a deep convolutional neural network and normative concept feature data were used to link patterns of neural activity during encoding to visual and semantic representations. Relative to young adults, the specificity of activation patterns for visual features was reduced in older adults, consistent with dedifferentiation. However, the specificity of activation patterns for semantic features was enhanced in older adults, consistent with hyperdifferentiation. Despite dedifferentiation, visual representations in early visual cortex (EVC) predicted high memory vividness in both age groups. In contrast, semantic representations in lingual gyrus (LG) and fusiform gyrus (FG) were associated with high memory vividness only in the older adults. Intriguingly, data suggests that older adults with lower specificity of visual representations in combination with higher specificity of semantic representations tended to rate their memories as more vivid. Our findings suggest that memory vividness in aging relies more on semantic representations over anterior regions, potentially compensating for age-related dedifferentiation of visual information in posterior regions.SIGNIFICANCE STATEMENT Normal aging is associated with impaired memory for events while semantic knowledge might even improve. We investigated the effects of aging on the specificity of visual and semantic information in the brain when viewing common objects and how this information enables subsequent memory vividness for these objects. Using functional magnetic resonance imaging (fMRI) combined with modeling of the stimuli we found that visual information was represented with less specificity in older than young adults while still supporting memory vividness. In contrast semantic information supported memory vividness only in older adults and especially in those individuals that had the lowest specificity of visual information. These findings provide evidence for a spared semantic memory system increasingly recruited to compensate for degraded visual representations in older age.
Collapse
Affiliation(s)
- Loris Naspi
- Department of Psychology, Humboldt University of Berlin, Berlin 10117, Germany
| | - Charlotte Stensholt
- Department of Psychology, Humboldt University of Berlin, Berlin 10117, Germany
| | - Anna E Karlsson
- Department of Psychology, Humboldt University of Berlin, Berlin 10117, Germany
| | - Zachary A Monge
- Center for Cognitive Neuroscience, Duke University, Durham, North Carolina 27708
| | - Roberto Cabeza
- Department of Psychology, Humboldt University of Berlin, Berlin 10117, Germany
- Center for Cognitive Neuroscience, Duke University, Durham, North Carolina 27708
| |
Collapse
|
6
|
Lee H, Keene PA, Sweigart SC, Hutchinson JB, Kuhl BA. Adding Meaning to Memories: How Parietal Cortex Combines Semantic Content with Episodic Experience. J Neurosci 2023; 43:6525-6537. [PMID: 37596054 PMCID: PMC10513070 DOI: 10.1523/jneurosci.1919-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 07/20/2023] [Accepted: 07/24/2023] [Indexed: 08/20/2023] Open
Abstract
Neuroimaging studies of human memory have consistently found that univariate responses in parietal cortex track episodic experience with stimuli (whether stimuli are 'old' or 'new'). More recently, pattern-based fMRI studies have shown that parietal cortex also carries information about the semantic content of remembered experiences. However, it is not well understood how memory-based and content-based signals are integrated within parietal cortex. Here, in humans (males and females), we used voxel-wise encoding models and a recognition memory task to predict the fMRI activity patterns evoked by complex natural scene images based on (1) the episodic history and (2) the semantic content of each image. Models were generated and compared across distinct subregions of parietal cortex and for occipitotemporal cortex. We show that parietal and occipitotemporal regions each encode memory and content information, but they differ in how they combine this information. Among parietal subregions, angular gyrus was characterized by robust and overlapping effects of memory and content. Moreover, subject-specific semantic tuning functions revealed that successful recognition shifted the amplitude of tuning functions in angular gyrus but did not change the selectivity of tuning. In other words, effects of memory and content were additive in angular gyrus. This pattern of data contrasted with occipitotemporal cortex where memory and content effects were interactive: memory effects were preferentially expressed by voxels tuned to the content of a remembered image. Collectively, these findings provide unique insight into how parietal cortex combines information about episodic memory and semantic content.SIGNIFICANCE STATEMENT Neuroimaging studies of human memory have identified multiple brain regions that not only carry information about "whether" a visual stimulus is successfully recognized but also "what" the content of that stimulus includes. However, a fundamental and open question concerns how the brain integrates these two types of information (memory and content). Here, using a powerful combination of fMRI analysis methods, we show that parietal cortex, particularly the angular gyrus, robustly combines memory- and content-related information, but these two forms of information are represented via additive, independent signals. In contrast, memory effects in high-level visual cortex critically depend on (and interact with) content representations. Together, these findings reveal multiple and distinct ways in which the brain combines memory- and content-related information.
Collapse
Affiliation(s)
- Hongmi Lee
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD 21218
| | - Paul A Keene
- Department of Psychology, University of Oregon, Eugene, OR 97403
| | - Sarah C Sweigart
- Department of Psychology, University of California-Davis, Davis, California 95616
| | | | - Brice A Kuhl
- Department of Psychology, University of Oregon, Eugene, OR 97403
- Institute of Neuroscience, University of Oregon, Eugene, OR 97403
| |
Collapse
|
7
|
Wang Y, Lee H, Kuhl BA. Mapping multidimensional content representations to neural and behavioral expressions of episodic memory. Neuroimage 2023; 277:120222. [PMID: 37327954 PMCID: PMC10424734 DOI: 10.1016/j.neuroimage.2023.120222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2023] [Revised: 06/06/2023] [Accepted: 06/08/2023] [Indexed: 06/18/2023] Open
Abstract
Human neuroimaging studies have shown that the contents of episodic memories are represented in distributed patterns of neural activity. However, these studies have mostly been limited to decoding simple, unidimensional properties of stimuli. Semantic encoding models, in contrast, offer a means for characterizing the rich, multidimensional information that comprises episodic memories. Here, we extensively sampled four human fMRI subjects to build semantic encoding models and then applied these models to reconstruct content from natural scene images as they were viewed and recalled from memory. First, we found that multidimensional semantic information was successfully reconstructed from activity patterns across visual and lateral parietal cortices, both when viewing scenes and when recalling them from memory. Second, whereas visual cortical reconstructions were much more accurate when images were viewed versus recalled from memory, lateral parietal reconstructions were comparably accurate across visual perception and memory. Third, by applying natural language processing methods to verbal recall data, we showed that fMRI-based reconstructions reliably matched subjects' verbal descriptions of their memories. In fact, reconstructions from ventral temporal cortex more closely matched subjects' own verbal recall than other subjects' verbal recall of the same images. Fourth, encoding models reliably transferred across subjects: memories were successfully reconstructed using encoding models trained on data from entirely independent subjects. Together, these findings provide evidence for successful reconstructions of multidimensional and idiosyncratic memory representations and highlight the differential sensitivity of visual cortical and lateral parietal regions to information derived from the external visual environment versus internally-generated memories.
Collapse
Affiliation(s)
- Yingying Wang
- Department of Psychology and Behavioral Sciences, Zhejiang University, Hangzhou 310028, China; Department of Psychology, University of Oregon, Eugene, OR 97403, USA
| | - Hongmi Lee
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Brice A Kuhl
- Department of Psychology, University of Oregon, Eugene, OR 97403, USA.
| |
Collapse
|
8
|
Chen YY, Areti A, Yoshor D, Foster BL. Individual-specific memory reinstatement patterns within human face-selective cortex. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.06.552130. [PMID: 37609262 PMCID: PMC10441346 DOI: 10.1101/2023.08.06.552130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/24/2023]
Abstract
Humans have the remarkable ability to vividly retrieve sensory details of past events. According to the theory of sensory reinstatement, during remembering, brain regions involved in the sensory processing of prior events are reactivated to support this perception of the past. Recently, several studies have emphasized potential transformations in the spatial organization of reinstated activity patterns. In particular, studies of scene stimuli suggest a clear anterior shift in the location of retrieval activations compared with those during perception. However, it is not clear that such transformations occur universally, with evidence lacking for other important stimulus categories, particularly faces. Critical to addressing these questions, and to studies of reinstatement more broadly, is the growing importance of considering meaningful variations in the organization of sensory systems across individuals. Therefore, we conducted a multi-session neuroimaging study to first carefully map individual participants face-selective regions within ventral temporal cortex (VTC), followed by a second session to examine the correspondence of activity patterns during face memory encoding and retrieval. Our results showed distinct configurations of face-selective regions within the VTC across individuals. While a significant degree of overlap was observed between face perception and memory encoding, memory retrieval engagement exhibited a more selective and constricted reinstatement pattern within these regions. Importantly, these activity patterns were consistently tied to individual-specific neural substrates, but did not show any consistent direction of spatial transformation (e.g., anteriorization). To provide further insight to these findings, we also report on unique human intracranial recordings from VTC under the same experimental conditions. Our findings highlight the importance of considering individual variations in functional neuroanatomy in the context of assessing the nature of cortical reinstatement. Consideration of such factors will be important for establishing general principles shaping the neural transformations that occur from perception to memory.
Collapse
Affiliation(s)
- Yvonne Y Chen
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, 19104, USA
| | | | - Daniel Yoshor
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, 19104, USA
| | - Brett L Foster
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, 19104, USA
| |
Collapse
|
9
|
Amer T, Davachi L. Extra-hippocampal contributions to pattern separation. eLife 2023; 12:82250. [PMID: 36972123 PMCID: PMC10042541 DOI: 10.7554/elife.82250] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Accepted: 03/21/2023] [Indexed: 03/29/2023] Open
Abstract
Pattern separation, or the process by which highly similar stimuli or experiences in memory are represented by non-overlapping neural ensembles, has typically been ascribed to processes supported by the hippocampus. Converging evidence from a wide range of studies, however, suggests that pattern separation is a multistage process supported by a network of brain regions. Based on this evidence, considered together with related findings from the interference resolution literature, we propose the 'cortico-hippocampal pattern separation' (CHiPS) framework, which asserts that brain regions involved in cognitive control play a significant role in pattern separation. Particularly, these regions may contribute to pattern separation by (1) resolving interference in sensory regions that project to the hippocampus, thus regulating its cortical input, or (2) directly modulating hippocampal processes in accordance with task demands. Considering recent interest in how hippocampal operations are modulated by goal states likely represented and regulated by extra-hippocampal regions, we argue that pattern separation is similarly supported by neocortical-hippocampal interactions.
Collapse
Affiliation(s)
- Tarek Amer
- Department of Psychology, University of Victoria, Victoria, Canada
| | - Lila Davachi
- Department of Psychology, Columbia University, New York, United States
- Nathan Kline Research Institute, Orangeburg, United States
| |
Collapse
|
10
|
Fang M, Poskanzer C, Anzellotti S. Multivariate connectivity: A brief introduction and an open question. Front Neurosci 2023; 16:1082120. [PMID: 36704006 PMCID: PMC9871770 DOI: 10.3389/fnins.2022.1082120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Accepted: 12/16/2022] [Indexed: 01/11/2023] Open
Affiliation(s)
- Mengting Fang
- University of Pennsylvania, Philadelphia, PA, United States
| | | | - Stefano Anzellotti
- Boston College, Chestnut Hill, MA, United States,*Correspondence: Stefano Anzellotti ✉
| |
Collapse
|
11
|
Cooper RA, Ritchey M. Patterns of episodic content and specificity predicting subjective memory vividness. Mem Cognit 2022; 50:1629-1643. [PMID: 35246786 DOI: 10.3758/s13421-022-01291-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/10/2022] [Indexed: 12/30/2022]
Abstract
The ability to remember and internally represent events is often accompanied by a subjective sense of "vividness". Vividness measures are frequently used to evaluate the experience of remembering and imagining events, yet little research has considered the objective attributes of event memories that underlie this subjective judgment, and individual differences in this mapping. Here, we tested how the content and specificity of event memories support subjectively vivid recollection. Over three experiments, participants encoded events containing a theme word and three distinct elements - a person, a place, and an object. In a memory test, memory for event elements was assessed at two levels of specificity - semantic gist (names) and perceptual details (lure discrimination). We found a strong correspondence between memory vividness and memory for gist information that did not vary by which elements were contained in memory. There was a smaller, additive benefit of remembering specific perceptual details on vividness, which, in one study, was driven by memory for place details. Moreover, we found individual differences in the relationship between memory vividness and objective memory attributes primarily along the specificity dimension, such that one cluster of participants used perceptual detail to inform memory vividness whereas another cluster was more driven by gist information. Therefore, while gist memory appears to drive vividness on average, there were idiosyncrasies in this pattern across participants. When assessing subjective ratings of memory and imagination, research should consider how these ratings map onto objective memory attributes in the context of their study design and population.
Collapse
Affiliation(s)
- Rose A Cooper
- Department of Psychology, Northeastern University, Boston, MA, USA.
- Roux Institute, Northeastern University, Boston, MA, USA.
| | - Maureen Ritchey
- Department of Psychology and Neuroscience, Boston College, Boston, MA, USA
| |
Collapse
|
12
|
Perception and memory have distinct spatial tuning properties in human visual cortex. Nat Commun 2022; 13:5864. [PMID: 36257949 PMCID: PMC9579130 DOI: 10.1038/s41467-022-33161-8] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2020] [Accepted: 09/06/2022] [Indexed: 11/12/2022] Open
Abstract
Reactivation of earlier perceptual activity is thought to underlie long-term memory recall. Despite evidence for this view, it is unclear whether mnemonic activity exhibits the same tuning properties as feedforward perceptual activity. Here, we leverage population receptive field models to parameterize fMRI activity in human visual cortex during spatial memory retrieval. Though retinotopic organization is present during both perception and memory, large systematic differences in tuning are also evident. Whereas there is a three-fold decline in spatial precision from early to late visual areas during perception, this pattern is not observed during memory retrieval. This difference cannot be explained by reduced signal-to-noise or poor performance on memory trials. Instead, by simulating top-down activity in a network model of cortex, we demonstrate that this property is well explained by the hierarchical structure of the visual system. Together, modeling and empirical results suggest that computational constraints imposed by visual system architecture limit the fidelity of memory reactivation in sensory cortex. Neural activity from perception is thought to be reactivated during memory recall. Here, the authors show that memory reactivation in visual cortex exhibits different tuning properties from those observed during perception.
Collapse
|
13
|
Xue G. From remembering to reconstruction: The transformative neural representation of episodic memory. Prog Neurobiol 2022; 219:102351. [PMID: 36089107 DOI: 10.1016/j.pneurobio.2022.102351] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2022] [Revised: 08/23/2022] [Accepted: 09/05/2022] [Indexed: 11/29/2022]
Abstract
Although memory has long been recognized as a generative process, neural research of memory in recent decades has been predominantly influenced by Tulving's "mental time traveling" perspective and focused on the reactivation and consolidation of encoded memory representations. With the development of multiple powerful analytical approaches to characterize the contents and formats of neural representations, recent studies are able to provide detailed examinations of the representations at various processing stages and have provided exciting new insights into the transformative nature of episodic memory. These studies have revealed the rapid, substantial, and continuous transformation of memory representation during the encoding, maintenance, consolidation, and retrieval of both single and multiple events, as well as event sequences. These transformations are characterized by the abstraction, integration, differentiation, and reorganization of memory representations, enabling the long-term retention and generalization of memory. These studies mark a significant shift in perspective from remembering to reconstruction, which might better reveal the nature of memory and its roles in supporting more effective learning, adaptive decision-making, and creative problem solving.
Collapse
Affiliation(s)
- Gui Xue
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, PR China; Chinese Institute for Brain Research, Beijing 102206, PR China.
| |
Collapse
|
14
|
Schultz H, Sommer T, Peters J. Category-sensitive incidental reinstatement in medial temporal lobe subregions during word recognition. Learn Mem 2022; 29:126-135. [PMID: 35428729 PMCID: PMC9053111 DOI: 10.1101/lm.053553.121] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2021] [Accepted: 03/29/2022] [Indexed: 11/25/2022]
Abstract
During associative retrieval, the brain reinstates neural representations that were present during encoding. The human medial temporal lobe (MTL), with its subregions hippocampus (HC), perirhinal cortex (PRC), and parahippocampal cortex (PHC), plays a central role in neural reinstatement. Previous studies have given compelling evidence for reinstatement in the MTL during explicitly instructed associative retrieval. High-confident recognition may be similarly accompanied by recollection of associated information from the encoding context. It is unclear, however, whether high-confident recognition memory elicits reinstatement in the MTL even in the absence of an explicit instruction to retrieve associated information. Here, we addressed this open question using high-resolution fMRI. Twenty-eight male and female human volunteers engaged in a recognition memory task for words that they had previously encoded together with faces and scenes. Using complementary univariate and multivariate approaches, we show that MTL subregions including the PRC, PHC, and HC differentially reinstate category-sensitive representations during high-confident word recognition, even though no explicit instruction to retrieve the associated category was given. This constitutes novel evidence that high-confident recognition memory is accompanied by incidental reinstatement of associated category information in MTL subregions, and supports a functional model of the MTL that emphasizes content-sensitive representations during both encoding and retrieval.
Collapse
Affiliation(s)
- Heidrun Schultz
- Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, 20246 Hamburg, Germany.,Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany
| | - Tobias Sommer
- Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, 20246 Hamburg, Germany
| | - Jan Peters
- Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, 20246 Hamburg, Germany.,Department of Psychology, Biological Psychology, University of Cologne, 50969 Cologne, Germany
| |
Collapse
|
15
|
Neural reactivation and judgements of vividness reveal separable contributions to mnemonic representation. Neuroimage 2022; 255:119205. [PMID: 35427774 DOI: 10.1016/j.neuroimage.2022.119205] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2021] [Revised: 04/04/2022] [Accepted: 04/08/2022] [Indexed: 11/22/2022] Open
Abstract
Mnemonic representations vary in fidelity, sharpness, and strength-qualities that can be examined using both introspective judgements of mental states and objective measures of brain activity. Subjective and objective measures are both valid ways of "reading out" the content of someone's internal mnemonic states, each with different strengths and weaknesses. St-Laurent and colleagues (2015) compared the neural correlates of memory vividness ratings with patterns of neural reactivation evoked during memory recall and found considerable overlap between the two, suggesting a common neural basis underlying these different markers of representational quality. Here we extended this work with meta-analytic methods by pooling together four neuroimaging datasets in order to contrast the neural substrates of neural reactivation and those of vividness judgements. While reactivation and vividness judgements correlated positively with one another and were associated with common univariate activity in the dorsal attention network and anterior hippocampus, some notable differences were also observed. Vividness judgments were tied to stronger activation in the striatum and dorsal attention network, together with activity suppression in default mode network nodes. We also observed a trend for reactivation to be more closely associated with early visual cortex activity. A mediation analysis found support for the hypothesis that neural reactivation is necessary for memory vividness, with activity in the anterior hippocampus associated with greater reactivation. Our results suggest that neural reactivation and vividness judgements reflect common mnemonic processes but differ in the extent to which they engage effortful, attentional processes. Additionally, the similarity between reactivation and vividness appears to arise, partly, through hippocampal engagement during memory retrieval.
Collapse
|
16
|
|
17
|
I remember it like it was yesterday: Age-related differences in the subjective experience of remembering. Psychon Bull Rev 2021; 29:1223-1245. [PMID: 34918271 DOI: 10.3758/s13423-021-02048-y] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/24/2021] [Indexed: 11/08/2022]
Abstract
It has been frequently described that older adults subjectively report the vividness of their memories as being as high, or even higher, than young adults, despite poorer objective memory performance. Here, we review studies that examined age-related differences in the subjective experience of memory vividness. By examining vividness calibration and resolution, studies using different types of approaches converge to suggest that older adults overestimate the intensity of their vividness ratings relative to young adults, and that they rely on retrieved memory details to a lesser extent to judge vividness. We discuss potential mechanisms underlying these observations. Inflation of memory vividness with regard to the richness of memory content may stem from age-differences in vividness criterion or scale interpretation and psycho-social factors. The reduced reliance on episodic memory details in older adults may stem from age-related differences in how they monitor these details to make their vividness ratings. Considered together, these findings emphasize the importance of examining age-differences in memory vividness using different analytical methods and they provide valuable evidence that the subjective experience of remembering is more than the reactivation of memory content. In this vein, we recommend that future studies explore the links between memory vividness and other subjective memory scales (e.g., ratings of details or memory confidence) in healthy aging and/or other populations, as it could be used as a window to better characterize the cognitive processes that underpin the subjective assessment of the quality of recollected events.
Collapse
|
18
|
Bone MB, Buchsbaum BR. Detailed Episodic Memory Depends on Concurrent Reactivation of Basic Visual Features within the Posterior Hippocampus and Early Visual Cortex. Cereb Cortex Commun 2021; 2:tgab045. [PMID: 34414371 PMCID: PMC8370760 DOI: 10.1093/texcom/tgab045] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Revised: 06/22/2021] [Accepted: 06/25/2021] [Indexed: 11/13/2022] Open
Abstract
The hippocampus is a key brain region for the storage and retrieval of episodic memories, but how it performs this function is unresolved. Leading theories posit that the hippocampus stores a sparse representation, or "index," of the pattern of neocortical activity that occurred during perception. During retrieval, reactivation of the index by a partial cue facilitates the reactivation of the associated neocortical pattern. Therefore, episodic retrieval requires joint reactivation of the hippocampal index and the associated neocortical networks. To test this theory, we examine the relation between performance on a recognition memory task requiring retrieval of image-specific visual details and feature-specific reactivation within the hippocampus and neocortex. We show that trial-by-trial recognition accuracy correlates with neural reactivation of low-level features (e.g., luminosity and edges) within the posterior hippocampus and early visual cortex for participants with high recognition lure accuracy. As predicted, the two regions interact, such that recognition accuracy correlates with hippocampal reactivation only when reactivation co-occurs within the early visual cortex (and vice versa). In addition to supporting leading theories of hippocampal function, our findings show large individual differences in the features underlying visual memory and suggest that the anterior and posterior hippocampus represents gist-like and detailed features, respectively.
Collapse
Affiliation(s)
- Michael B Bone
- Rotman Research Institute at Baycrest, Toronto, Ontario, M6A 2E1, Canada
| | | |
Collapse
|
19
|
Zhao Y, Chanales AJH, Kuhl BA. Adaptive Memory Distortions Are Predicted by Feature Representations in Parietal Cortex. J Neurosci 2021; 41:3014-3024. [PMID: 33619210 PMCID: PMC8018893 DOI: 10.1523/jneurosci.2875-20.2021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2020] [Revised: 01/26/2021] [Accepted: 01/27/2021] [Indexed: 11/21/2022] Open
Abstract
Similarity between memories is a primary cause of interference and forgetting. Exaggerating subtle differences between memories is therefore a potential mechanism for reducing interference. Here, we report a human fMRI study (n = 29, 19 female) that tested whether behavioral and neural expressions of memories are adaptively distorted to reduce interference. Participants learned and repeatedly retrieved object images, some of which were identical except for subtle color differences. Behavioral measures of color memory revealed exaggeration of differences between similar objects. Importantly, greater memory exaggeration was associated with lower memory interference. fMRI pattern analyses revealed that color information in parietal cortex was stronger during memory recall when color information was critical for discriminating competing memories. Moreover, greater representational distance between competing memories in parietal cortex predicted greater color memory exaggeration and lower memory interference. Together, these findings reveal that competition between memories induces adaptive, feature-specific distortions in parietal representations and corresponding behavioral expressions.SIGNIFICANCE STATEMENT Similarity between memories is a primary cause of interference and forgetting. Here, we show that, when remembering highly similar objects, subtle differences in the features of these objects are exaggerated in memory to reduce interference. These memory distortions are reflected in, and predicted by, overlap of activity patterns in lateral parietal cortex. These findings provide unique insight into how memory interference is resolved and specifically implicate lateral parietal cortex in representing feature-specific memory distortions.
Collapse
Affiliation(s)
- Yufei Zhao
- Department of Psychology, University of Oregon, Eugene, Oregon 97401
| | - Avi J H Chanales
- Department of Psychology, New York University, New York, New York 10016
| | - Brice A Kuhl
- Department of Psychology, University of Oregon, Eugene, Oregon 97401
| |
Collapse
|
20
|
Conti F, Irish M. Harnessing Visual Imagery and Oculomotor Behaviour to Understand Prospection. Trends Cogn Sci 2021; 25:272-283. [PMID: 33618981 DOI: 10.1016/j.tics.2021.01.009] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Revised: 01/20/2021] [Accepted: 01/20/2021] [Indexed: 12/20/2022]
Abstract
Much of the rich internal world constructed by humans is derived from, and experienced through, visual mental imagery. Despite growing appreciation of visual exploration in guiding episodic memory processes, extant theories of prospection have yet to accommodate the precise role of visual mental imagery in the service of future-oriented thinking. We propose that the construction of future events relies on the assimilation of perceptual details originally experienced, and subsequently reinstantiated, predominantly in the visual domain. Individual differences in the capacity to summon discrete aspects of visual imagery can therefore account for the diversity of content generated by humans during future simulation. Our integrative framework provides a novel testbed to query alterations in future thinking in health and disease.
Collapse
Affiliation(s)
- Federica Conti
- Institut des Neurosciences de la Timone, Aix-Marseille University, 27 Boulevard Jean Moulin, 13005 Marseille, France; The University of Sydney, Brain and Mind Centre and School of Psychology, 94 Mallett Street, Camperdown, NSW 2050, Australia.
| | - Muireann Irish
- The University of Sydney, Brain and Mind Centre and School of Psychology, 94 Mallett Street, Camperdown, NSW 2050, Australia.
| |
Collapse
|
21
|
Context Memory Encoding and Retrieval Temporal Dynamics are Modulated by Attention across the Adult Lifespan. eNeuro 2021; 8:ENEURO.0387-20.2020. [PMID: 33436445 PMCID: PMC7877465 DOI: 10.1523/eneuro.0387-20.2020] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2020] [Revised: 11/11/2020] [Accepted: 11/13/2020] [Indexed: 11/21/2022] Open
Abstract
Episodic memories are multidimensional, including simple and complex features. How we successful encode and recover these features in time, whether these temporal dynamics are preserved across age, even under conditions of reduced memory performance, and the role of attention on these temporal dynamics is unknown. In the current study, we applied time-resolved multivariate decoding to oscillatory electroencephalography (EEG) in an adult lifespan sample to investigate the temporal order of successful encoding and recognition of simple and complex perceptual context features. At encoding, participants studied pictures of black and white objects presented with both color (low-level/simple) and scene (high-level/complex) context features and subsequently made context memory decisions for both features. Attentional demands were manipulated by having participants attend to the relationship between the object and either the color or scene while ignoring the other context feature. Consistent with hierarchical visual perception models, simple visual features (color) were successfully encoded earlier than were complex features (scenes). These features were successfully recognized in the reverse temporal order. Importantly, these temporal dynamics were both dependent on whether these context features were in the focus of one's attention, and preserved across age, despite age-related context memory impairments. These novel results support the idea that episodic memories are encoded and retrieved successively, likely dependent on the input and output pathways of the medial temporal lobe (MTL), and attentional influences that bias activity within these pathways across age.
Collapse
|
22
|
Gilmore AW, Quach A, Kalinowski SE, Gotts SJ, Schacter DL, Martin A. Dynamic Content Reactivation Supports Naturalistic Autobiographical Recall in Humans. J Neurosci 2021; 41:153-166. [PMID: 33203742 PMCID: PMC7786205 DOI: 10.1523/jneurosci.1490-20.2020] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Revised: 10/05/2020] [Accepted: 11/03/2020] [Indexed: 11/30/2022] Open
Abstract
Humans can vividly recall and re-experience events from their past, and these are commonly referred to as episodic or autobiographical memories. fMRI experiments reliably associate autobiographical event recall with activity in a network of "default" or "core" brain regions. However, as prior studies have relied on covert (silent) recall procedures, current understanding may be hampered by methodological limitations that obscure dynamic effects supporting moment-to-moment content retrieval. Here, fMRI participants (N = 40) overtly (verbally) recalled memories for ∼2 min periods. The content of spoken descriptions was categorized using a variant of the Autobiographical Interview (AI) procedure (Levine et al., 2002) and temporally re-aligned with BOLD data so activity accompanying the recall of different details could be measured. Replicating prior work, sustained effects associated with autobiographical recall periods (which are insensitive to the moment-to-moment content of retrieval) fell primarily within canonical default network regions. Spoken descriptions were rich in episodic details, frequently focusing on physical entities, their ongoing activities, and their appearances. Critically, neural activity associated with recalling specific details (e.g., those related to people or places) was transient, broadly distributed, and grounded in category-selective cortex (e.g., regions related to social cognition or scene processing). Thus, although a single network may generally support the process of vivid event reconstruction, the structures required to provide detail-related information shift in a predictable manner that respects domain-level representations across the cortex.SIGNIFICANCE STATEMENT Humans can vividly recall memories of autobiographical episodes, a process thought to involve the reconstruction of numerous distinct event details. Yet how the brain represents a complex episode as it unfolds over time remains unclear and appears inconsistent across experimental traditions. One hurdle is the use of covert (silent) in-scanner recall to study autobiographical memory, which prevents experimenter knowledge of what information is being retrieved, and when, throughout the remembering process. In this experiment, participants overtly described autobiographical memories while undergoing fMRI. Activity associated with the recall and description of specific details was transient, broadly distributed, and grounded in category-selective cortex. Thus, it appears that as events unfold mentally, structures are dynamically reactivated to support vivid recollection.
Collapse
Affiliation(s)
- Adrian W Gilmore
- Section on Cognitive Neuropsychology, Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892
| | - Alina Quach
- Section on Cognitive Neuropsychology, Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892
| | - Sarah E Kalinowski
- Section on Cognitive Neuropsychology, Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892
| | - Stephen J Gotts
- Section on Cognitive Neuropsychology, Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892
| | - Daniel L Schacter
- Department of Psychology, Harvard University, Cambridge, Massachusetts 02138
| | - Alex Martin
- Section on Cognitive Neuropsychology, Laboratory of Brain and Cognition, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892
| |
Collapse
|
23
|
Favila SE, Lee H, Kuhl BA. Transforming the Concept of Memory Reactivation. Trends Neurosci 2020; 43:939-950. [PMID: 33041061 PMCID: PMC7688497 DOI: 10.1016/j.tins.2020.09.006] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2020] [Revised: 08/18/2020] [Accepted: 09/15/2020] [Indexed: 12/18/2022]
Abstract
Reactivation refers to the phenomenon wherein patterns of neural activity expressed during perceptual experience are re-expressed at a later time, a putative neural marker of memory. Reactivation of perceptual content has been observed across many cortical areas and correlates with objective and subjective expressions of memory in humans. However, because reactivation emphasizes similarities between perceptual and memory-based representations, it obscures differences in how perceptual events and memories are represented. Here, we highlight recent evidence of systematic differences in how (and where) perceptual events and memories are represented in the brain. We argue that neural representations of memories are best thought of as spatially transformed versions of perceptual representations. We consider why spatial transformations occur and identify critical questions for future research.
Collapse
Affiliation(s)
- Serra E Favila
- Department of Psychology, Columbia University, New York, NY 10027, USA
| | - Hongmi Lee
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Brice A Kuhl
- Department of Psychology, University of Oregon, Eugene, OR 97403, USA.
| |
Collapse
|