The distinct and overlapping brain networks supporting semantic and spatial constructive scene processing

Using imagination and the contents of memory to create new scene and object representations: A functional MRI study

Humans can use the contents of memory to construct scenarios and events that they have not encountered before, a process colloquially known as imagination. Much of our current understanding of the neural mechanisms mediating imagination is limited by paradigms that rely on participants' subjective reports of imagined content. Here, we used a novel behavioral paradigm that was designed to systematically evaluate the contents of an individual's imagination. Participants first learned the layout of four distinct rooms containing five wall segments with differing geometrical characteristics, each associated with a unique object. During functional MRI, participants were then shown two different wall segments or objects on each trial and asked to first, retrieve the associated objects or walls, respectively (retrieval phase) and then second, imagine the two objects side-by-side or combine the two wall segments (imagination phase). Importantly, the contents of each participant's imagination were interrogated by having them make a same/different judgment about the properties of the imagined objects or scenes. Using univariate and multivariate analyses, we observed widespread activity across occipito-temporal cortex for the retrieval of objects and for the imaginative creation of scenes. Interestingly, a classifier, whether trained on the imagination or retrieval data, was able to successfully differentiate the neural patterns associated with the imagination of scenes from that of objects. Our results reveal neural differences in the cued retrieval of object and scene memoranda, demonstrate that different representations underlie the creation and/or imagination of scene and object content, and highlight a novel behavioral paradigm that can be used to systematically evaluate the contents of an individual's imagination.

A theory of hippocampal function: New developments

We develop further here the only quantitative theory of the storage of information in the hippocampal episodic memory system and its recall back to the neocortex. The theory is upgraded to account for a revolution in understanding of spatial representations in the primate, including human, hippocampus, that go beyond the place where the individual is located, to the location being viewed in a scene. This is fundamental to much primate episodic memory and navigation: functions supported in humans by pathways that build 'where' spatial view representations by feature combinations in a ventromedial visual cortical stream, separate from those for 'what' object and face information to the inferior temporal visual cortex, and for reward information from the orbitofrontal cortex. Key new computational developments include the capacity of the CA3 attractor network for storing whole charts of space; how the correlations inherent in self-organizing continuous spatial representations impact the storage capacity; how the CA3 network can combine continuous spatial and discrete object and reward representations; the roles of the rewards that reach the hippocampus in the later consolidation into long-term memory in part via cholinergic pathways from the orbitofrontal cortex; and new ways of analysing neocortical information storage using Potts networks.

Scene construction in healthy aging - Exploring the interplay between task complexity and oculomotor behaviour

Mounting evidence indicates a close correspondence between episodic memory, mental imagery, and oculomotor behaviour. It remains unclear, however, how oculomotor variables support endogenously driven forms of mental imagery and how this relationship changes across the adult lifespan. In this study we investigated age-related changes in oculomotor signatures during scene construction and explored how task complexity impacts these processes. Younger and cognitively healthy older participants completed a guided scene construction paradigm where scene complexity was manipulated according to the number of elements to be sequentially integrated. We recorded participants' eye movements and collected subjective ratings regarding their phenomenological experience. Overall, older adults rated their constructions as more vivid and more spatially integrated, while also generating more fixations and saccades relative to the younger group, specifically on control trials. Analyses of participants' total scan paths revealed that, in the early stages of scene construction, oculomotor behaviour changed as a function of task complexity within each group. Following the introduction of a second stimulus, older but not younger adults showed a significant decrease in the production of eye movements. Whether this shift in oculomotor behaviour serves a compensatory function to bolster task performance represents an important question for future research.

Learning induces EPO/EPOr expression in memory relevant brain areas, whereas exogenously applied EPO promotes remote memory consolidation

Memory and learning allow animals to appropriate certain properties of nature with which they can navigate in it successfully. Memory is acquired slowly and consists of two major phases, a fragile early phase (short-term memory, <4 h) and a more robust and long-lasting late one (long-term memory, >4 h). Erythropoietin (EPO) prolongs memory from 24 to 72 h when animals are trained for 5 min in a place recognition task but not when training lasted 3 min (short-term memory). It is not known whether it promotes the formation of remote memory (≥21 days). We address whether the systemic administration of EPO can convert a short-term memory into a long-term remote memory, and the neural plasticity mechanisms involved. We evaluated the effect of training duration (3 or 5 min) on the expression of endogenous EPO and its receptor to shed light on the role of EPO in coordinating mechanisms of neural plasticity using a single-trial spatial learning test. We administered EPO 10 min post-training and evaluated memory after 24 h, 96 h, 15 days, or 21 days. We also determined the effect of EPO administered 10 min after training on the expression of arc and bdnf during retrieval at 24 h and 21 days. Data show that learning induces EPO/EPOr expression increase linked to memory extent, exogenous EPO prolongs memory up to 21 days; and prefrontal cortex bdnf expression at 24 h and in the hippocampus at 21 days, whereas arc expression increases at 21 days in the hippocampus and prefrontal cortex.

Attention and schema violations of real world scenes differentially modulate time perception

In the real world, object arrangement follows a number of rules. Some of the rules pertain to the spatial relations between objects and scenes (i.e., syntactic rules) and others about the contextual relations (i.e., semantic rules). Research has shown that violation of semantic rules influences interval timing with the duration of scenes containing such violations to be overestimated as compared to scenes with no violations. However, no study has yet investigated whether both semantic and syntactic violations can affect timing in the same way. Furthermore, it is unclear whether the effect of scene violations on timing is due to attentional or other cognitive accounts. Using an oddball paradigm and real-world scenes with or without semantic and syntactic violations, we conducted two experiments on whether time dilation will be obtained in the presence of any type of scene violation and the role of attention in any such effect. Our results from Experiment 1 showed that time dilation indeed occurred in the presence of syntactic violations, while time compression was observed for semantic violations. In Experiment 2, we further investigated whether these estimations were driven by attentional accounts, by utilizing a contrast manipulation of the target objects. The results showed that an increased contrast led to duration overestimation for both semantic and syntactic oddballs. Together, our results indicate that scene violations differentially affect timing due to violation processing differences and, moreover, their effect on timing seems to be sensitive to attentional manipulations such as target contrast.

Allocentric information represented by self-referenced spatial coding in the primate medial temporal lobe

For living organisms, the ability to acquire information regarding the external space around them is critical for future actions. While the information must be stored in an allocentric frame to facilitate its use in various spatial contexts, each case of use requires the information to be represented in a particular self-referenced frame. Previous studies have explored neural substrates responsible for the linkage between self-referenced and allocentric spatial representations based on findings in rodents. However, the behaviors of rodents are different from those of primates in several aspects; for example, rodents mainly explore their environments through locomotion, while primates use eye movements. In this review, we discuss the brain mechanisms responsible for the linkage in nonhuman primates. Based on recent physiological studies, we propose that two types of neural substrates link the first-person perspective with allocentric coding. The first is the view-center background signal, which represents an image of the background surrounding the current position of fixation on the retina. This perceptual signal is transmitted from the ventral visual pathway to the hippocampus (HPC) via the perirhinal cortex and parahippocampal cortex. Because images that share the same objective-position in the environment tend to appear similar when seen from different self-positions, the view-center background signals are easily associated with one another in the formation of allocentric position coding and storage. The second type of neural substrate is the HPC neurons' dynamic activity that translates the stored location memory to the first-person perspective depending on the current spatial context.

The human orbitofrontal cortex, vmPFC, and anterior cingulate cortex effective connectome: emotion, memory, and action

The human orbitofrontal cortex, ventromedial prefrontal cortex (vmPFC), and anterior cingulate cortex are involved in reward processing and thereby in emotion but are also implicated in episodic memory. To understand these regions better, the effective connectivity between 360 cortical regions and 24 subcortical regions was measured in 172 humans from the Human Connectome Project and complemented with functional connectivity and diffusion tractography. The orbitofrontal cortex has effective connectivity from gustatory, olfactory, and temporal visual, auditory, and pole cortical areas. The orbitofrontal cortex has connectivity to the pregenual anterior and posterior cingulate cortex and hippocampal system and provides for rewards to be used in memory and navigation to goals. The orbitofrontal and pregenual anterior cortex have connectivity to the supracallosal anterior cingulate cortex, which projects to midcingulate and other premotor cortical areas and provides for action-outcome learning including limb withdrawal or flight or fight to aversive and nonreward stimuli. The lateral orbitofrontal cortex has outputs to language systems in the inferior frontal gyrus. The medial orbitofrontal cortex connects to the nucleus basalis of Meynert and the pregenual cingulate to the septum, and damage to these cortical regions may contribute to memory impairments by disrupting cholinergic influences on the neocortex and hippocampus.

Significance of visual scene-based learning in the hippocampal systems across mammalian species

The hippocampus and its associated cortical regions in the medial temporal lobe play essential roles when animals form a cognitive map and use it to achieve their goals. As the nature of map-making involves sampling different local views of the environment and putting them together in a spatially cohesive way, visual scenes are essential ingredients in the formative process of cognitive maps. Visual scenes also serve as important cues during information retrieval from the cognitive map. Research in humans has shown that there are regions in the brain that selectively process scenes and that the hippocampus is involved in scene-based memory tasks. The neurophysiological correlates of scene-based information processing in the hippocampus have been reported as "spatial view cells" in nonhuman primates. Like primates, it is widely accepted that rodents also use visual scenes in their background for spatial navigation and other kinds of problems. However, in rodents, it is not until recently that researchers examined the neural correlates of the hippocampus from the perspective of visual scene-based information processing. With the advent of virtual reality (VR) systems, it has been demonstrated that place cells in the hippocampus exhibit remarkably similar firing correlates in the VR environment compared with that of the real-world environment. Despite some limitations, the new trend of studying hippocampal functions in a visually controlled environment has the potential to allow investigation of the input-output relationships of network functions and experimental testing of traditional computational predictions more rigorously by providing well-defined visual stimuli. As scenes are essential for navigation and episodic memory in humans, further investigation of the rodents' hippocampal systems in scene-based tasks will provide a critical functional link across different mammalian species.

Hippocampal activity during memory and visual perception: The role of representational content

The functional organisation of the medial temporal lobe (MTL) has long been described on the basis of cognitive processes such as recollection or familiarity. However, this view has recently been challenged, and researchers have proposed decomposing cognitive phenomena into representations and operations. According to the representational view, representations, such as scenes for the hippocampus and objects for the perirhinal cortex, are critical in understanding the role of MTL regions in cognition. In the present study, 51 healthy young participants underwent functional magnetic resonance imaging (fMRI) while completing a visual-discrimination task. Subsequently, half of the participants performed a patch-cue recognition procedure in which "Rec" responses are believed to reflect the operation of pattern completion, whereas the other half performed a whole-item remember/know procedure. We replicated the previously-reported demonstration that hippocampal involvement in pattern completion is preferential for scenes as compared with objects. In contrast, the perirhinal cortex was more recruited for object processing than for scene processing. We further extended these results to the operations of strength-signal memory and visual discrimination. Finally, the modulation of hippocampal engagement in pattern completion by representational content was found to be specific to its anterior segment. This observation is consistent with the proposal that this segment would process broad/global representations, whereas the posterior hippocampus would perform sharp/local representations. Taken together, these results favour the representational view of MTL functional organisation, but support that this specialisation differs along the hippocampal long-axis.

The hippocampus, ventromedial prefrontal cortex, and episodic and semantic memory

The human ventromedial prefrontal cortex (vmPFC)/anterior cingulate cortex is implicated in reward and emotion, but also in memory. It is shown how the human orbitofrontal cortex connecting with the vmPFC and anterior cingulate cortex provide a route to the hippocampus for reward and emotional value to be incorporated into episodic memory, enabling memory of where a reward was seen. It is proposed that this value component results in primarily episodic memories with some value component to be repeatedly recalled from the hippocampus so that they are more likely to become incorporated into neocortical semantic and autobiographical memories. The same orbitofrontal and anterior cingulate regions also connect in humans to the septal and basal forebrain cholinergic nuclei, thereby helping to consolidate memory, and helping to account for why damage to the vMPFC impairs memory. The human hippocampus and vmPFC thus contribute in complementary ways to forming episodic and semantic memories.