1
|
Rolls ET, Yan X, Deco G, Zhang Y, Jousmaki V, Feng J. A ventromedial visual cortical 'Where' stream to the human hippocampus for spatial scenes revealed with magnetoencephalography. Commun Biol 2024; 7:1047. [PMID: 39183244 PMCID: PMC11345434 DOI: 10.1038/s42003-024-06719-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2024] [Accepted: 08/12/2024] [Indexed: 08/27/2024] Open
Abstract
The primate including the human hippocampus implicated in episodic memory and navigation represents a spatial view, very different from the place representations in rodents. To understand this system in humans, and the computations performed, the pathway for this spatial view information to reach the hippocampus was analysed in humans. Whole-brain effective connectivity was measured with magnetoencephalography between 30 visual cortical regions and 150 other cortical regions using the HCP-MMP1 atlas in 21 participants while performing a 0-back scene memory task. In a ventromedial visual stream, V1-V4 connect to the ProStriate region where the retrosplenial scene area is located. The ProStriate region has connectivity to ventromedial visual regions VMV1-3 and VVC. These ventromedial regions connect to the medial parahippocampal region PHA1-3, which, with the VMV regions, include the parahippocampal scene area. The medial parahippocampal regions have effective connectivity to the entorhinal cortex, perirhinal cortex, and hippocampus. In contrast, when viewing faces, the effective connectivity was more through a ventrolateral visual cortical stream via the fusiform face cortex to the inferior temporal visual cortex regions TE2p and TE2a. A ventromedial visual cortical 'Where' stream to the hippocampus for spatial scenes was supported by diffusion topography in 171 HCP participants at 7 T.
Collapse
Affiliation(s)
- Edmund T Rolls
- Oxford Centre for Computational Neuroscience, Oxford, UK.
- Department of Computer Science, University of Warwick, Coventry, UK.
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, China.
| | - Xiaoqian Yan
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, China
| | - Gustavo Deco
- Department of Information and Communication Technologies, Center for Brain and Cognition, Computational Neuroscience Group, Universitat Pompeu Fabra, Barcelona, Spain
- Institució Catalana de la Recerca i Estudis Avançats (ICREA), Universitat Pompeu Fabra, Passeig Lluís Companys 23, Barcelona, Spain
| | - Yi Zhang
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, China
| | - Veikko Jousmaki
- Aalto NeuroImaging, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| | - Jianfeng Feng
- Department of Computer Science, University of Warwick, Coventry, UK
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, China
| |
Collapse
|
2
|
Rolls ET, Treves A. A theory of hippocampal function: New developments. Prog Neurobiol 2024; 238:102636. [PMID: 38834132 DOI: 10.1016/j.pneurobio.2024.102636] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2024] [Revised: 04/15/2024] [Accepted: 05/30/2024] [Indexed: 06/06/2024]
Abstract
We develop further here the only quantitative theory of the storage of information in the hippocampal episodic memory system and its recall back to the neocortex. The theory is upgraded to account for a revolution in understanding of spatial representations in the primate, including human, hippocampus, that go beyond the place where the individual is located, to the location being viewed in a scene. This is fundamental to much primate episodic memory and navigation: functions supported in humans by pathways that build 'where' spatial view representations by feature combinations in a ventromedial visual cortical stream, separate from those for 'what' object and face information to the inferior temporal visual cortex, and for reward information from the orbitofrontal cortex. Key new computational developments include the capacity of the CA3 attractor network for storing whole charts of space; how the correlations inherent in self-organizing continuous spatial representations impact the storage capacity; how the CA3 network can combine continuous spatial and discrete object and reward representations; the roles of the rewards that reach the hippocampus in the later consolidation into long-term memory in part via cholinergic pathways from the orbitofrontal cortex; and new ways of analysing neocortical information storage using Potts networks.
Collapse
Affiliation(s)
- Edmund T Rolls
- Oxford Centre for Computational Neuroscience, Oxford, UK; Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK.
| | | |
Collapse
|
3
|
Rolls ET, Feng J, Zhang R. Selective activations and functional connectivities to the sight of faces, scenes, body parts and tools in visual and non-visual cortical regions leading to the human hippocampus. Brain Struct Funct 2024; 229:1471-1493. [PMID: 38839620 PMCID: PMC11176242 DOI: 10.1007/s00429-024-02811-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2024] [Accepted: 05/22/2024] [Indexed: 06/07/2024]
Abstract
Connectivity maps are now available for the 360 cortical regions in the Human Connectome Project Multimodal Parcellation atlas. Here we add function to these maps by measuring selective fMRI activations and functional connectivity increases to stationary visual stimuli of faces, scenes, body parts and tools from 956 HCP participants. Faces activate regions in the ventrolateral visual cortical stream (FFC), in the superior temporal sulcus (STS) visual stream for face and head motion; and inferior parietal visual (PGi) and somatosensory (PF) regions. Scenes activate ventromedial visual stream VMV and PHA regions in the parahippocampal scene area; medial (7m) and lateral parietal (PGp) regions; and the reward-related medial orbitofrontal cortex. Body parts activate the inferior temporal cortex object regions (TE1p, TE2p); but also visual motion regions (MT, MST, FST); and the inferior parietal visual (PGi, PGs) and somatosensory (PF) regions; and the unpleasant-related lateral orbitofrontal cortex. Tools activate an intermediate ventral stream area (VMV3, VVC, PHA3); visual motion regions (FST); somatosensory (1, 2); and auditory (A4, A5) cortical regions. The findings add function to cortical connectivity maps; and show how stationary visual stimuli activate other cortical regions related to their associations, including visual motion, somatosensory, auditory, semantic, and orbitofrontal cortex value-related, regions.
Collapse
Affiliation(s)
- Edmund T Rolls
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK.
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, 200403, China.
- Oxford Centre for Computational Neuroscience, Oxford, UK.
| | - Jianfeng Feng
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, 200403, China
| | - Ruohan Zhang
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK.
| |
Collapse
|
4
|
Rolls ET. The memory systems of the human brain and generative artificial intelligence. Heliyon 2024; 10:e31965. [PMID: 38841455 PMCID: PMC11152951 DOI: 10.1016/j.heliyon.2024.e31965] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Revised: 05/11/2024] [Accepted: 05/24/2024] [Indexed: 06/07/2024] Open
Abstract
Generative Artificial Intelligence foundation models (for example Generative Pre-trained Transformer - GPT - models) can generate the next token given a sequence of tokens. How can this 'generative AI' be compared with the 'real' intelligence of the human brain, when for example a human generates a whole memory in response to an incomplete retrieval cue, and then generates further prospective thoughts? Here these two types of generative intelligence, artificial in machines and real in the human brain are compared, and it is shown how when whole memories are generated by hippocampal recall in response to an incomplete retrieval cue, what the human brain computes, and how it computes it, are very different from generative AI. Key differences are the use of local associative learning rules in the hippocampal memory system, and of non-local backpropagation of error learning in AI. Indeed, it is argued that the whole operation of the human brain is performed computationally very differently to what is implemented in generative AI. Moreover, it is emphasized that the primate including human hippocampal system includes computations about spatial view and where objects and people are in scenes, whereas in rodents the emphasis is on place cells and path integration by movements between places. This comparison with generative memory and processing in the human brain has interesting implications for the further development of generative AI and for neuroscience research.
Collapse
Affiliation(s)
- Edmund T. Rolls
- Oxford Centre for Computational Neuroscience, Oxford, UK
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai, 200403, China
| |
Collapse
|
5
|
Rolls ET. Two what, two where, visual cortical streams in humans. Neurosci Biobehav Rev 2024; 160:105650. [PMID: 38574782 DOI: 10.1016/j.neubiorev.2024.105650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Revised: 03/25/2024] [Accepted: 03/31/2024] [Indexed: 04/06/2024]
Abstract
ROLLS, E. T. Two What, Two Where, Visual Cortical Streams in Humans. NEUROSCI BIOBEHAV REV 2024. Recent cortical connectivity investigations lead to new concepts about 'What' and 'Where' visual cortical streams in humans, and how they connect to other cortical systems. A ventrolateral 'What' visual stream leads to the inferior temporal visual cortex for object and face identity, and provides 'What' information to the hippocampal episodic memory system, the anterior temporal lobe semantic system, and the orbitofrontal cortex emotion system. A superior temporal sulcus (STS) 'What' visual stream utilising connectivity from the temporal and parietal visual cortex responds to moving objects and faces, and face expression, and connects to the orbitofrontal cortex for emotion and social behaviour. A ventromedial 'Where' visual stream builds feature combinations for scenes, and provides 'Where' inputs via the parahippocampal scene area to the hippocampal episodic memory system that are also useful for landmark-based navigation. The dorsal 'Where' visual pathway to the parietal cortex provides for actions in space, but also provides coordinate transforms to provide inputs to the parahippocampal scene area for self-motion update of locations in scenes in the dark or when the view is obscured.
Collapse
Affiliation(s)
- Edmund T Rolls
- Oxford Centre for Computational Neuroscience, Oxford, UK; Department of Computer Science, University of Warwick, Coventry CV4 7AL, UK; Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai 200403, China.
| |
Collapse
|
6
|
Kyzar M, Kamiński J, Brzezicka A, Reed CM, Chung JM, Mamelak AN, Rutishauser U. Dataset of human-single neuron activity during a Sternberg working memory task. Sci Data 2024; 11:89. [PMID: 38238342 PMCID: PMC10796636 DOI: 10.1038/s41597-024-02943-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Accepted: 01/10/2024] [Indexed: 01/22/2024] Open
Abstract
We present a dataset of 1809 single neurons recorded from the human medial temporal lobe (amygdala and hippocampus) and medial frontal lobe (anterior cingulate cortex, pre-supplementary motor area, ventral medial prefrontal cortex) across 41 sessions from 21 patients that underwent seizure monitoring with depth electrodes. Subjects performed a screening task (907 neurons) to identify images for which highly selective cells were present. Subjects then performed a working memory task (902 neurons), in which they were sequentially presented with 1-3 images for which highly selective cells were present and, following a maintenance period, were asked if the probe was identical to one of the maintained images. This Neurodata Without Borders formatted dataset includes spike times, extracellular spike waveforms, stimuli presented, behavior, electrode locations, and subject demographics. As validation, we replicate previous findings on the selectivity of concept cells and their persistent activity during working memory maintenance. This large dataset of rare human single-neuron recordings and behavior enables the investigation of the neural mechanisms of working memory in humans.
Collapse
Affiliation(s)
- Michael Kyzar
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Jan Kamiński
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
- Center of Excellence for Neural Plasticity and Brain Disorders: BRAINCITY, Nencki Institute of Experimental Biology, Polish Academy of Sciences, Warsaw, Poland
| | - Aneta Brzezicka
- Institute of Psychology, SWPS University of Social Sciences and Humanities, Warsaw, Poland
| | - Chrystal M Reed
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Jeffrey M Chung
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Adam N Mamelak
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Ueli Rutishauser
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
- Center for Neural Science and Medicine, Department of Biomedical Sciences, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA.
| |
Collapse
|
7
|
Donoghue T, Maesta-Pereira S, Han CZ, Qasim SE, Jacobs J. spiketools: a Python package for analyzing single-unit neural activity. JOURNAL OF OPEN SOURCE SOFTWARE 2023; 8:5268. [PMID: 39027322 PMCID: PMC11257244 DOI: 10.21105/joss.05268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/20/2024]
Affiliation(s)
- Thomas Donoghue
- Department of Biomedical Engineering, Columbia University, New York, United States of America
| | - Sandra Maesta-Pereira
- Department of Biomedical Engineering, Columbia University, New York, United States of America
| | - Claire Zhixian Han
- Department of Biomedical Engineering, Columbia University, New York, United States of America
| | - Salman Ehtesham Qasim
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, United States of America
| | - Joshua Jacobs
- Department of Biomedical Engineering, Columbia University, New York, United States of America
- Department of Neurological Surgery, Columbia University, New York, United States of America
| |
Collapse
|
8
|
Han CZ, Donoghue T, Cao R, Kunz L, Wang S, Jacobs J. Using multi-task experiments to test principles of hippocampal function. Hippocampus 2023; 33:646-657. [PMID: 37042212 PMCID: PMC10249632 DOI: 10.1002/hipo.23540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 03/30/2023] [Accepted: 03/31/2023] [Indexed: 04/13/2023]
Abstract
Investigations of hippocampal functions have revealed a dizzying array of findings, from lesion-based behavioral deficits, to a diverse range of characterized neural activations, to computational models of putative functionality. Across these findings, there remains an ongoing debate about the core function of the hippocampus and the generality of its representation. Researchers have debated whether the hippocampus's primary role relates to the representation of space, the neural basis of (episodic) memory, or some more general computation that generalizes across various cognitive domains. Within these different perspectives, there is much debate about the nature of feature encodings. Here, we suggest that in order to evaluate hippocampal responses-investigating, for example, whether neuronal representations are narrowly targeted to particular tasks or if they subserve domain-general purposes-a promising research strategy may be the use of multi-task experiments, or more generally switching between multiple task contexts while recording from the same neurons in a given session. We argue that this strategy-when combined with explicitly defined theoretical motivations that guide experiment design-could be a fruitful approach to better understand how hippocampal representations support different behaviors. In doing so, we briefly review key open questions in the field, as exemplified by articles in this special issue, as well as previous work using multi-task experiments, and extrapolate to consider how this strategy could be further applied to probe fundamental questions about hippocampal function.
Collapse
Affiliation(s)
- Claire Z. Han
- Department of Biomedical Engineering, Columbia University
| | | | - Runnan Cao
- Department of Radiology, Washington University in St. Louis
| | - Lukas Kunz
- Department of Epileptology, University of Bonn Medical Center, Bonn, Germany
| | - Shuo Wang
- Department of Radiology, Washington University in St. Louis
| | - Joshua Jacobs
- Department of Biomedical Engineering, Columbia University
- Department of Neurological Surgery, Columbia University
| |
Collapse
|
9
|
Murphy E. ROSE: A Neurocomputational Architecture for Syntax. ARXIV 2023:arXiv:2303.08877v1. [PMID: 36994166 PMCID: PMC10055479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 03/31/2023]
Abstract
A comprehensive model of natural language processing in the brain must accommodate four components: representations, operations, structures and encoding. It further requires a principled account of how these different components mechanistically, and causally, relate to each another. While previous models have isolated regions of interest for structure-building and lexical access, and have utilized specific neural recording measures to expose possible signatures of syntax, many gaps remain with respect to bridging distinct scales of analysis that map onto these four components. By expanding existing accounts of how neural oscillations can index various linguistic processes, this article proposes a neurocomputational architecture for syntax, termed the ROSE model (Representation, Operation, Structure, Encoding). Under ROSE, the basic data structures of syntax are atomic features, types of mental representations (R), and are coded at the single-unit and ensemble level. Elementary computations (O) that transform these units into manipulable objects accessible to subsequent structure-building levels are coded via high frequency broadband γ activity. Low frequency synchronization and cross-frequency coupling code for recursive categorial inferences (S). Distinct forms of low frequency coupling and phase-amplitude coupling (δ-θ coupling via pSTS-IFG; θ-γ coupling via IFG to conceptual hubs in lateral and ventral temporal cortex) then encode these structures onto distinct workspaces (E). Causally connecting R to O is spike-phase/LFP coupling; connecting O to S is phase-amplitude coupling; connecting S to E is a system of frontotemporal traveling oscillations; connecting E back to lower levels is low-frequency phase resetting of spike-LFP coupling. This compositional neural code has important implications for algorithmic accounts, since it makes concrete predictions for the appropriate level of study for psycholinguistic parsing models. ROSE is reliant on neurophysiologically plausible mechanisms, is supported at all four levels by a range of recent empirical research, and provides an anatomically precise and falsifiable grounding for the basic property of natural language syntax: hierarchical, recursive structure-building.
Collapse
Affiliation(s)
- Elliot Murphy
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, UTHealth, Houston, TX, USA
- Texas Institute for Restorative Neurotechnologies, UTHealth, Houston, TX, USA
| |
Collapse
|