51
|
Wise T, Liu Y, Chowdhury F, Dolan RJ. Model-based aversive learning in humans is supported by preferential task state reactivation. SCIENCE ADVANCES 2021; 7:eabf9616. [PMID: 34321205 PMCID: PMC8318377 DOI: 10.1126/sciadv.abf9616] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Accepted: 06/10/2021] [Indexed: 06/13/2023]
Abstract
Harm avoidance is critical for survival, yet little is known regarding the neural mechanisms supporting avoidance in the absence of trial-and-error experience. Flexible avoidance may be supported by a mental model (i.e., model-based), a process for which neural reactivation and sequential replay have emerged as candidate mechanisms. During an aversive learning task, combined with magnetoencephalography, we show prospective and retrospective reactivation during planning and learning, respectively, coupled to evidence for sequential replay. Specifically, when individuals plan in an aversive context, we find preferential reactivation of subsequently chosen goal states. Stronger reactivation is associated with greater hippocampal theta power. At outcome receipt, unchosen goal states are reactivated regardless of outcome valence. Replay of paths leading to goal states was modulated by outcome valence, with aversive outcomes associated with stronger reverse replay than safe outcomes. Our findings are suggestive of avoidance involving simulation of unexperienced states through hippocampally mediated reactivation and replay.
Collapse
Affiliation(s)
- Toby Wise
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK.
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Chinese Institute for Brain Research, Beijing, China
| | - Fatima Chowdhury
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Queen Square MS Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, London, UK
| | - Raymond J Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| |
Collapse
|
52
|
Liu Y, Dolan RJ, Higgins C, Penagos H, Woolrich MW, Ólafsdóttir HF, Barry C, Kurth-Nelson Z, Behrens TE. Temporally delayed linear modelling (TDLM) measures replay in both animals and humans. eLife 2021; 10:e66917. [PMID: 34096501 PMCID: PMC8318595 DOI: 10.7554/elife.66917] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2021] [Accepted: 06/06/2021] [Indexed: 12/25/2022] Open
Abstract
There are rich structures in off-task neural activity which are hypothesized to reflect fundamental computations across a broad spectrum of cognitive functions. Here, we develop an analysis toolkit - temporal delayed linear modelling (TDLM) - for analysing such activity. TDLM is a domain-general method for finding neural sequences that respect a pre-specified transition graph. It combines nonlinear classification and linear temporal modelling to test for statistical regularities in sequences of task-related reactivations. TDLM is developed on the non-invasive neuroimaging data and is designed to take care of confounds and maximize sequence detection ability. Notably, as a linear framework, TDLM can be easily extended, without loss of generality, to capture rodent replay in electrophysiology, including in continuous spaces, as well as addressing second-order inference questions, for example, its temporal and spatial varying pattern. We hope TDLM will advance a deeper understanding of neural computation and promote a richer convergence between animal and human neuroscience.
Collapse
Affiliation(s)
- Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal UniversityBeijingChina
- Chinese Institute for Brain ResearchBeijingChina
- Max Planck University College London Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
| | - Raymond J Dolan
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal UniversityBeijingChina
- Max Planck University College London Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
- Wellcome Centre for Human Neuroimaging, University College LondonLondonUnited Kingdom
| | - Cameron Higgins
- Wellcome Centre for Integrative Neuroimaging, University of OxfordOxfordUnited Kingdom
| | - Hector Penagos
- Center for Brains, Minds and Machines, Picower Institute for Learning and Memory, Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
| | - Mark W Woolrich
- Wellcome Centre for Integrative Neuroimaging, University of OxfordOxfordUnited Kingdom
| | - H Freyja Ólafsdóttir
- Donders Institute for Brain Cognition and Behaviour, Radboud UniversityNijmegenNetherlands
| | - Caswell Barry
- Research Department of Cell and Developmental Biology, University College LondonLondonUnited Kingdom
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
- DeepMindLondonUnited Kingdom
| | - Timothy E Behrens
- Wellcome Centre for Human Neuroimaging, University College LondonLondonUnited Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of OxfordOxfordUnited Kingdom
| |
Collapse
|
53
|
Liu Y, Mattar MG, Behrens TEJ, Daw ND, Dolan RJ. Experience replay is associated with efficient nonlocal learning. Science 2021; 372:372/6544/eabf1357. [PMID: 34016753 DOI: 10.1126/science.abf1357] [Citation(s) in RCA: 48] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2020] [Accepted: 04/15/2021] [Indexed: 01/08/2023]
Abstract
To make effective decisions, people need to consider the relationship between actions and outcomes. These are often separated by time and space. The neural mechanisms by which disjoint actions and outcomes are linked remain unknown. One promising hypothesis involves neural replay of nonlocal experience. Using a task that segregates direct from indirect value learning, combined with magnetoencephalography, we examined the role of neural replay in human nonlocal learning. After receipt of a reward, we found significant backward replay of nonlocal experience, with a 160-millisecond state-to-state time lag, which was linked to efficient learning of action values. Backward replay and behavioral evidence of nonlocal learning were more pronounced for experiences of greater benefit for future behavior. These findings support nonlocal replay as a neural mechanism for solving complex credit assignment problems during learning.
Collapse
Affiliation(s)
- Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China. .,Chinese Institute for Brain Research, Beijing, China.,Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK.,Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Marcelo G Mattar
- Department of Cognitive Science, University of California, San Diego, CA, USA.
| | - Timothy E J Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK. .,Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Nathaniel D Daw
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA.
| | - Raymond J Dolan
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China. .,Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK.,Wellcome Centre for Human Neuroimaging, University College London, London, UK.,Department of Psychiatry, Universitätsmedizin Berlin (Campus Charité Mitte), Berlin, Germany
| |
Collapse
|
54
|
Frölich S, Marković D, Kiebel SJ. Neuronal Sequence Models for Bayesian Online Inference. Front Artif Intell 2021; 4:530937. [PMID: 34095815 PMCID: PMC8176225 DOI: 10.3389/frai.2021.530937] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2021] [Accepted: 04/13/2021] [Indexed: 11/13/2022] Open
Abstract
Various imaging and electrophysiological studies in a number of different species and brain regions have revealed that neuronal dynamics associated with diverse behavioral patterns and cognitive tasks take on a sequence-like structure, even when encoding stationary concepts. These neuronal sequences are characterized by robust and reproducible spatiotemporal activation patterns. This suggests that the role of neuronal sequences may be much more fundamental for brain function than is commonly believed. Furthermore, the idea that the brain is not simply a passive observer but an active predictor of its sensory input, is supported by an enormous amount of evidence in fields as diverse as human ethology and physiology, besides neuroscience. Hence, a central aspect of this review is to illustrate how neuronal sequences can be understood as critical for probabilistic predictive information processing, and what dynamical principles can be used as generators of neuronal sequences. Moreover, since different lines of evidence from neuroscience and computational modeling suggest that the brain is organized in a functional hierarchy of time scales, we will also review how models based on sequence-generating principles can be embedded in such a hierarchy, to form a generative model for recognition and prediction of sensory input. We shortly introduce the Bayesian brain hypothesis as a prominent mathematical description of how online, i.e., fast, recognition, and predictions may be computed by the brain. Finally, we briefly discuss some recent advances in machine learning, where spatiotemporally structured methods (akin to neuronal sequences) and hierarchical networks have independently been developed for a wide range of tasks. We conclude that the investigation of specific dynamical and structural principles of sequential brain activity not only helps us understand how the brain processes information and generates predictions, but also informs us about neuroscientific principles potentially useful for designing more efficient artificial neuronal networks for machine learning tasks.
Collapse
Affiliation(s)
- Sascha Frölich
- Department of Psychology, Technische Universität Dresden, Dresden, Germany
| | | | | |
Collapse
|
55
|
Constantino AC, Sisterson ND, Zaher N, Urban A, Richardson RM, Kokkinos V. Expert-Level Intracranial Electroencephalogram Ictal Pattern Detection by a Deep Learning Neural Network. Front Neurol 2021; 12:603868. [PMID: 34012415 PMCID: PMC8126697 DOI: 10.3389/fneur.2021.603868] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Accepted: 04/08/2021] [Indexed: 11/13/2022] Open
Abstract
Background: Decision-making in epilepsy surgery is strongly connected to the interpretation of the intracranial EEG (iEEG). Although deep learning approaches have demonstrated efficiency in processing extracranial EEG, few studies have addressed iEEG seizure detection, in part due to the small number of seizures per patient typically available from intracranial investigations. This study aims to evaluate the efficiency of deep learning methodology in detecting iEEG seizures using a large dataset of ictal patterns collected from epilepsy patients implanted with a responsive neurostimulation system (RNS). Methods: Five thousand two hundred and twenty-six ictal events were collected from 22 patients implanted with RNS. A convolutional neural network (CNN) architecture was created to provide personalized seizure annotations for each patient. Accuracy of seizure identification was tested in two scenarios: patients with seizures occurring following a period of chronic recording (scenario 1) and patients with seizures occurring immediately following implantation (scenario 2). The accuracy of the CNN in identifying RNS-recorded iEEG ictal patterns was evaluated against human neurophysiology expertise. Statistical performance was assessed via the area-under-precision-recall curve (AUPRC). Results: In scenario 1, the CNN achieved a maximum mean binary classification AUPRC of 0.84 ± 0.19 (95%CI, 0.72-0.93) and mean regression accuracy of 6.3 ± 1.0 s (95%CI, 4.3-8.5 s) at 30 seed samples. In scenario 2, maximum mean AUPRC was 0.80 ± 0.19 (95%CI, 0.68-0.91) and mean regression accuracy was 6.3 ± 0.9 s (95%CI, 4.8-8.3 s) at 20 seed samples. We obtained near-maximum accuracies at seed size of 10 in both scenarios. CNN classification failures can be explained by ictal electro-decrements, brief seizures, single-channel ictal patterns, highly concentrated interictal activity, changes in the sleep-wake cycle, and progressive modulation of electrographic ictal features. Conclusions: We developed a deep learning neural network that performs personalized detection of RNS-derived ictal patterns with expert-level accuracy. These results suggest the potential for automated techniques to significantly improve the management of closed-loop brain stimulation, including during the initial period of recording when the device is otherwise naïve to a given patient's seizures.
Collapse
Affiliation(s)
- Alexander C Constantino
- Brain Modulation Lab, Department of Neurological Surgery, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States
| | - Nathaniel D Sisterson
- Department of Neurosurgery, Massachusetts General Hospital, Boston, MA, United States
| | - Naoir Zaher
- Department of Neurology, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States.,University of Pittsburgh Comprehensive Epilepsy Center, Pittsburgh, PA, United States
| | - Alexandra Urban
- Department of Neurology, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States.,University of Pittsburgh Comprehensive Epilepsy Center, Pittsburgh, PA, United States
| | - R Mark Richardson
- Department of Neurosurgery, Massachusetts General Hospital, Boston, MA, United States.,Harvard Medical School, Boston, MA, United States
| | - Vasileios Kokkinos
- Department of Neurosurgery, Massachusetts General Hospital, Boston, MA, United States.,Harvard Medical School, Boston, MA, United States
| |
Collapse
|
56
|
Parish G, Michelmann S, Hanslmayr S, Bowman H. The Sync-Fire/deSync model: Modelling the reactivation of dynamic memories from cortical alpha oscillations. Neuropsychologia 2021; 158:107867. [PMID: 33905757 DOI: 10.1016/j.neuropsychologia.2021.107867] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2020] [Revised: 04/19/2021] [Accepted: 04/20/2021] [Indexed: 11/29/2022]
Abstract
We propose a neural network model to explore how humans can learn and accurately retrieve temporal sequences, such as melodies, movies, or other dynamic content. We identify target memories by their neural oscillatory signatures, as shown in recent human episodic memory paradigms. Our model comprises three plausible components for the binding of temporal content, where each component imposes unique limitations on the encoding and representation of that content. A cortical component actively represents sequences through the disruption of an intrinsically generated alpha rhythm, where a desynchronisation marks information-rich operations as the literature predicts. A binding component converts each event into a discrete index, enabling repetitions through a sparse encoding of events. A timing component - consisting of an oscillatory "ticking clock" made up of hierarchical synfire chains - discretely indexes a moment in time. By encoding the absolute timing between discretised events, we show how one can use cortical desynchronisations to dynamically detect unique temporal signatures as they are reactivated in the brain. We validate this model by simulating a series of events where sequences are uniquely identifiable by analysing phasic information, as several recent EEG/MEG studies have shown. As such, we show how one can encode and retrieve complete episodic memories where the quality of such memories is modulated by the following: alpha gate keepers to content representation; binding limitations that induce a blink in temporal perception; and nested oscillations that provide preferential learning phases in order to temporally sequence events.
Collapse
Affiliation(s)
- George Parish
- School of Psychology and Centre for Human Brain Health, University of Birmingham, UK.
| | | | - Simon Hanslmayr
- Institute of Neuroscience and Psychology & Centre for Cognitive Neuroimaging, University of Glasgow, UK
| | - Howard Bowman
- School of Psychology and Centre for Human Brain Health, University of Birmingham, UK; School of Computing, University of Kent, UK
| |
Collapse
|
57
|
|
58
|
|
59
|
Dynamics of fMRI patterns reflect sub-second activation sequences and reveal replay in human visual cortex. Nat Commun 2021; 12:1795. [PMID: 33741933 PMCID: PMC7979874 DOI: 10.1038/s41467-021-21970-2] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2020] [Accepted: 02/16/2021] [Indexed: 01/31/2023] Open
Abstract
Neural computations are often fast and anatomically localized. Yet, investigating such computations in humans is challenging because non-invasive methods have either high temporal or spatial resolution, but not both. Of particular relevance, fast neural replay is known to occur throughout the brain in a coordinated fashion about which little is known. We develop a multivariate analysis method for functional magnetic resonance imaging that makes it possible to study sequentially activated neural patterns separated by less than 100 ms with precise spatial resolution. Human participants viewed five images individually and sequentially with speeds up to 32 ms between items. Probabilistic pattern classifiers were trained on activation patterns in visual and ventrotemporal cortex during individual image trials. Applied to sequence trials, probabilistic classifier time courses allow the detection of neural representations and their order. Order detection remains possible at speeds up to 32 ms between items (plus 100 ms per item). The frequency spectrum of the sequentiality metric distinguishes between sub- versus supra-second sequences. Importantly, applied to resting-state data our method reveals fast replay of task-related stimuli in visual cortex. This indicates that non-hippocampal replay occurs even after tasks without memory requirements and shows that our method can be used to detect such spontaneously occurring replay.
Collapse
|
60
|
Patai EZ, Spiers HJ. The Versatile Wayfinder: Prefrontal Contributions to Spatial Navigation. Trends Cogn Sci 2021; 25:520-533. [PMID: 33752958 DOI: 10.1016/j.tics.2021.02.010] [Citation(s) in RCA: 48] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2020] [Revised: 02/22/2021] [Accepted: 02/23/2021] [Indexed: 12/15/2022]
Abstract
The prefrontal cortex (PFC) supports decision-making, goal tracking, and planning. Spatial navigation is a behavior that taxes these cognitive processes, yet the role of the PFC in models of navigation has been largely overlooked. In humans, activity in dorsolateral PFC (dlPFC) and ventrolateral PFC (vlPFC) during detours, reveal a role in inhibition and replanning. Dorsal anterior cingulate cortex (dACC) is implicated in planning and spontaneous internally-generated changes of route. Orbitofrontal cortex (OFC) integrates representations of the environment with the value of actions, providing a 'map' of possible decisions. In rodents, medial frontal areas interact with hippocampus during spatial decisions and switching between navigation strategies. In reviewing these advances, we provide a framework for how different prefrontal regions may contribute to different stages of navigation.
Collapse
Affiliation(s)
- Eva Zita Patai
- Centre for Neuroimaging Sciences, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, UK; Institute of Behavioural Neuroscience, Department of Experimental Psychology, Division of Psychology and Language sciences, University College London, UK.
| | - Hugo J Spiers
- Institute of Behavioural Neuroscience, Department of Experimental Psychology, Division of Psychology and Language sciences, University College London, UK.
| |
Collapse
|
61
|
Mantziara M, Ivanov T, Houghton G, Kornysheva K. Competitive state of movements during planning predicts sequence performance. J Neurophysiol 2021; 125:1251-1268. [PMID: 33656932 DOI: 10.1152/jn.00645.2020] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Humans can learn and produce skilled movement sequences from memory, yet the nature of sequence planning is not well understood. Previous computational and neurophysiological work suggests that movements in a sequence are planned as parallel graded activations and selected for output through competition. However, the relevance of this planning pattern to sequence production fluency and accuracy, as opposed to the temporal structure of sequences, is unclear. To resolve this question, we assessed the relative availability of constituent movements behaviorally during the preparation of motor sequences from memory. In three separate multisession experiments, healthy participants were trained to retrieve and produce four-element finger press sequences with particular timing according to an abstract sequence cue. We evaluated reaction time (RT) and error rate as markers of movement availability to constituent movement probes. Our results demonstrate that longer preparation time produces more pronounced differences in availability between adjacent sequence elements, whereas no effect was found for sequence speed or temporal grouping. Further, participants with larger position-dependent differences in movement availability tended to initiate correct sequences faster and with a higher temporal accuracy. Our results suggest that competitive preactivation is established gradually during sequence planning and predicts sequence skill, rather than the temporal structure of the motor sequence.NEW & NOTEWORTHY Sequence planning is an integral part of motor sequence control. Here, we demonstrate that the competitive state of sequential movements during sequence planning can be read out behaviorally through movement probes. We show that position-dependent differences in movement availability during planning reflect sequence preparedness and skill but not the timing of the planned sequence. Behavioral access to the preparatory state of movements may serve as a marker of sequence planning capacity.
Collapse
Affiliation(s)
- Myrto Mantziara
- School of Psychology, Bangor University, Bangor, Wales, United Kingdom.,Bangor Imaging Unit, Bangor University, Bangor, Wales, United Kingdom
| | - Tsvetoslav Ivanov
- School of Psychology, Bangor University, Bangor, Wales, United Kingdom
| | - George Houghton
- School of Psychology, Bangor University, Bangor, Wales, United Kingdom
| | - Katja Kornysheva
- School of Psychology, Bangor University, Bangor, Wales, United Kingdom.,Bangor Imaging Unit, Bangor University, Bangor, Wales, United Kingdom
| |
Collapse
|
62
|
Findlay G, Tononi G, Cirelli C. The evolving view of replay and its functions in wake and sleep. ACTA ACUST UNITED AC 2021; 1:zpab002. [PMID: 33644760 PMCID: PMC7898724 DOI: 10.1093/sleepadvances/zpab002] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Revised: 01/20/2021] [Indexed: 12/28/2022]
Abstract
The term hippocampal replay originally referred to the temporally compressed reinstantiation, during rest, of sequential neural activity observed during prior active wake. Since its description in the 1990s, hippocampal replay has often been viewed as the key mechanism by which a memory trace is repeatedly rehearsed at high speeds during sleep and gradually transferred to neocortical circuits. However, the methods used to measure the occurrence of replay remain debated, and it is now clear that the underlying neural events are considerably more complicated than the traditional narratives had suggested. “Replay-like” activity happens during wake, can play out in reverse order, may represent trajectories never taken by the animal, and may have additional functions beyond memory consolidation, from learning values and solving the problem of credit assignment to decision-making and planning. Still, we know little about the role of replay in cognition, and to what extent it differs between wake and sleep. This may soon change, however, because decades-long efforts to explain replay in terms of reinforcement learning (RL) have started to yield testable predictions and possible explanations for a diverse set of observations. Here, we (1) survey the diverse features of replay, focusing especially on the latest findings; (2) discuss recent attempts at unifying disparate experimental results and putatively different cognitive functions under the banner of RL; (3) discuss methodological issues and theoretical biases that impede progress or may warrant a partial revaluation of the current literature, and finally; (4) highlight areas of considerable uncertainty and promising avenues of inquiry.
Collapse
Affiliation(s)
- Graham Findlay
- Department of Psychiatry, University of Wisconsin-Madison, Madison, WI, USA.,Neuroscience Training Program, University of Wisconsin-Madison, Madison, WI, USA
| | - Giulio Tononi
- Department of Psychiatry, University of Wisconsin-Madison, Madison, WI, USA
| | - Chiara Cirelli
- Department of Psychiatry, University of Wisconsin-Madison, Madison, WI, USA
| |
Collapse
|
63
|
Schechtman E, Antony JW, Lampe A, Wilson BJ, Norman KA, Paller KA. Multiple memories can be simultaneously reactivated during sleep as effectively as a single memory. Commun Biol 2021; 4:25. [PMID: 33398075 PMCID: PMC7782847 DOI: 10.1038/s42003-020-01512-0] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2020] [Accepted: 11/20/2020] [Indexed: 01/27/2023] Open
Abstract
Memory consolidation involves the reactivation of memory traces during sleep. If different memories are reactivated each night, how much do they interfere with one another? We examined whether reactivating multiple memories incurs a cost to sleep-related benefits by contrasting reactivation of multiple memories versus single memories during sleep. First, participants learned the on-screen location of different objects. Each object was part of a semantically coherent group comprised of either one, two, or six items (e.g., six different cats). During sleep, sounds were unobtrusively presented to reactivate memories for half of the groups (e.g., "meow"). Memory benefits for cued versus non-cued items were independent of the number of items in the group, suggesting that reactivation occurs in a simultaneous and promiscuous manner. Intriguingly, sleep spindles and delta-theta power modulations were sensitive to group size, reflecting the extent of previous learning. Our results demonstrate that multiple memories may be consolidated in parallel without compromising each memory's sleep-related benefit. These findings highlight alternative models for parallel consolidation that should be considered in future studies.
Collapse
Affiliation(s)
- Eitan Schechtman
- Department of Psychology, Northwestern University, Evanston, IL, 60208, USA.
| | - James W Antony
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, 08544, USA
| | - Anna Lampe
- Department of Psychology, Northwestern University, Evanston, IL, 60208, USA
| | - Brianna J Wilson
- Department of Psychology, Northwestern University, Evanston, IL, 60208, USA
| | - Kenneth A Norman
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, 08544, USA
| | - Ken A Paller
- Department of Psychology, Northwestern University, Evanston, IL, 60208, USA
| |
Collapse
|
64
|
Barron HC, Mars RB, Dupret D, Lerch JP, Sampaio-Baptista C. Cross-species neuroscience: closing the explanatory gap. Philos Trans R Soc Lond B Biol Sci 2021; 376:20190633. [PMID: 33190601 PMCID: PMC7116399 DOI: 10.1098/rstb.2019.0633] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/20/2020] [Indexed: 12/17/2022] Open
Abstract
Neuroscience has seen substantial development in non-invasive methods available for investigating the living human brain. However, these tools are limited to coarse macroscopic measures of neural activity that aggregate the diverse responses of thousands of cells. To access neural activity at the cellular and circuit level, researchers instead rely on invasive recordings in animals. Recent advances in invasive methods now permit large-scale recording and circuit-level manipulations with exquisite spatio-temporal precision. Yet, there has been limited progress in relating these microcircuit measures to complex cognition and behaviour observed in humans. Contemporary neuroscience thus faces an explanatory gap between macroscopic descriptions of the human brain and microscopic descriptions in animal models. To close the explanatory gap, we propose adopting a cross-species approach. Despite dramatic differences in the size of mammalian brains, this approach is broadly justified by preserved homology. Here, we outline a three-armed approach for effective cross-species investigation that highlights the need to translate different measures of neural activity into a common space. We discuss how a cross-species approach has the potential to transform basic neuroscience while also benefiting neuropsychiatric drug development where clinical translation has, to date, seen minimal success. This article is part of the theme issue 'Key relationships between non-invasive functional neuroimaging and the underlying neuronal activity'.
Collapse
Affiliation(s)
- Helen C. Barron
- Medical Research Council Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Mansfield Road, Oxford OX1 3TH, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, FMRIB, John Radcliffe Hospital, Oxford OX3 9DU, UK
| | - Rogier B. Mars
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, FMRIB, John Radcliffe Hospital, Oxford OX3 9DU, UK
- Donders Institute for Brain, Cognition and Behavior, Radboud University, 6525 AJ Nijmegen, The Netherlands
| | - David Dupret
- Medical Research Council Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Mansfield Road, Oxford OX1 3TH, UK
| | - Jason P. Lerch
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, FMRIB, John Radcliffe Hospital, Oxford OX3 9DU, UK
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, CanadaM5G 1L7
| | - Cassandra Sampaio-Baptista
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, FMRIB, John Radcliffe Hospital, Oxford OX3 9DU, UK
- Institute of Neuroscience and Psychology, University of Glasgow, Glasgow G12 8QB, UK
| |
Collapse
|
65
|
Higgins C, Liu Y, Vidaurre D, Kurth-Nelson Z, Dolan R, Behrens T, Woolrich M. Replay bursts in humans coincide with activation of the default mode and parietal alpha networks. Neuron 2020; 109:882-893.e7. [PMID: 33357412 PMCID: PMC7927915 DOI: 10.1016/j.neuron.2020.12.007] [Citation(s) in RCA: 58] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2020] [Revised: 10/15/2020] [Accepted: 12/04/2020] [Indexed: 11/04/2022]
Abstract
Our brains at rest spontaneously replay recently acquired information, but how this process is orchestrated to avoid interference with ongoing cognition is an open question. Here we investigated whether replay coincided with spontaneous patterns of whole-brain activity. We found, in two separate datasets, that replay sequences were packaged into transient bursts occurring selectively during activation of the default mode network (DMN) and parietal alpha networks. These networks are believed to support inwardly oriented attention and inhibit bottom-up sensory processing and were characterized by widespread synchronized oscillations coupled to increases in high frequency power, mechanisms thought to coordinate information flow between disparate cortical areas. Our data reveal a tight correspondence between two widely studied phenomena in neural physiology and suggest that the DMN may coordinate replay bursts in a manner that minimizes interference with ongoing cognition. Replay in humans coincides with activity in specific resting brain networks Clusters of heightened default mode and alpha activity are linked to replay bursts These networks are characterized by highly synchronized brain-wide oscillations High-frequency power bursts are uniquely linked to default mode network activation
Collapse
Affiliation(s)
- Cameron Higgins
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| | - Yunzhe Liu
- Wellcome Trust Centre for Neuroimaging, University College London, London, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| | - Diego Vidaurre
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK; Center of Functionally Integrative Neuroscience, Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Deepmind, London, UK
| | - Ray Dolan
- Wellcome Trust Centre for Neuroimaging, University College London, London, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| | - Timothy Behrens
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK; Wellcome Trust Centre for Neuroimaging, University College London, London, UK
| | - Mark Woolrich
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| |
Collapse
|
66
|
Liashenko A, Dizaji AS, Melloni L, Schwiedrzik CM. Memory guidance of value-based decision making at an abstract level of representation. Sci Rep 2020; 10:21496. [PMID: 33299077 PMCID: PMC7726557 DOI: 10.1038/s41598-020-78460-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2020] [Accepted: 11/24/2020] [Indexed: 01/12/2023] Open
Abstract
Value-based decisions about alternatives we have never experienced can be guided by associations between current choice options and memories of prior reward. A critical question is how similar memories need to be to the current situation to effectively guide decisions. We address this question in the context of associative learning of faces using a sensory preconditioning paradigm. We find that memories of reward spread along established associations between faces to guide decision making. While memory guidance is specific for associated facial identities, it does not only occur for the specific images that were originally encountered. Instead, memory guidance generalizes across different images of the associated identities. This suggests that memory guidance does not rely on a pictorial format of representation but on a higher, view-invariant level of abstraction. Thus, memory guidance operates on a level of representation that neither over- nor underspecifies associative relationships in the context of obtaining reward.
Collapse
Affiliation(s)
- Anna Liashenko
- Neural Circuits and Cognition Lab, European Neuroscience Institute Göttingen - A Joint Initiative of the University Medical Center Göttingen and the Max Planck Society, Grisebachstraße 5, 37077, Göttingen, Germany
- International Max Planck Research School Neurosciences at the Georg August University Göttingen, Grisebachstraße 5, 37077, Göttingen, Germany
| | - Aslan S Dizaji
- Neural Circuits and Cognition Lab, European Neuroscience Institute Göttingen - A Joint Initiative of the University Medical Center Göttingen and the Max Planck Society, Grisebachstraße 5, 37077, Göttingen, Germany
| | - Lucia Melloni
- Department of Neuroscience, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, 60322, Frankfurt am Main, Germany
- Department of Neurology, New York University School of Medicine, 223 East 34th Street, New York, NY, 10016, USA
| | - Caspar M Schwiedrzik
- Neural Circuits and Cognition Lab, European Neuroscience Institute Göttingen - A Joint Initiative of the University Medical Center Göttingen and the Max Planck Society, Grisebachstraße 5, 37077, Göttingen, Germany.
- Perception and Plasticity Group, German Primate Center - Leibniz Institute for Primate Research, Kellnerweg 4, 37077, Göttingen, Germany.
| |
Collapse
|
67
|
Saxe A, Nelli S, Summerfield C. If deep learning is the answer, what is the question? Nat Rev Neurosci 2020; 22:55-67. [PMID: 33199854 DOI: 10.1038/s41583-020-00395-8] [Citation(s) in RCA: 112] [Impact Index Per Article: 28.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/02/2020] [Indexed: 11/09/2022]
Abstract
Neuroscience research is undergoing a minor revolution. Recent advances in machine learning and artificial intelligence research have opened up new ways of thinking about neural computation. Many researchers are excited by the possibility that deep neural networks may offer theories of perception, cognition and action for biological brains. This approach has the potential to radically reshape our approach to understanding neural systems, because the computations performed by deep networks are learned from experience, and not endowed by the researcher. If so, how can neuroscientists use deep networks to model and understand biological brains? What is the outlook for neuroscientists who seek to characterize computations or neural codes, or who wish to understand perception, attention, memory and executive functions? In this Perspective, our goal is to offer a road map for systems neuroscience research in the age of deep learning. We discuss the conceptual and methodological challenges of comparing behaviour, learning dynamics and neural representations in artificial and biological systems, and we highlight new research questions that have emerged for neuroscience as a direct consequence of recent advances in machine learning.
Collapse
Affiliation(s)
- Andrew Saxe
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Stephanie Nelli
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | | |
Collapse
|
68
|
Park SA, Miller DS, Nili H, Ranganath C, Boorman ED. Map Making: Constructing, Combining, and Inferring on Abstract Cognitive Maps. Neuron 2020; 107:1226-1238.e8. [PMID: 32702288 PMCID: PMC7529977 DOI: 10.1016/j.neuron.2020.06.030] [Citation(s) in RCA: 90] [Impact Index Per Article: 22.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2019] [Revised: 05/29/2020] [Accepted: 06/24/2020] [Indexed: 10/23/2022]
Abstract
Cognitive maps enable efficient inferences from limited experience that can guide novel decisions. We tested whether the hippocampus (HC), entorhinal cortex (EC), and ventromedial prefrontal cortex (vmPFC)/medial orbitofrontal cortex (mOFC) organize abstract and discrete relational information into a cognitive map to guide novel inferences. Subjects learned the status of people in two unseen 2D social hierarchies, with each dimension learned on a separate day. Although one dimension was behaviorally relevant, multivariate activity patterns in HC, EC, and vmPFC/mOFC were linearly related to the Euclidean distance between people in the mentally reconstructed 2D space. Hubs created unique comparisons between the hierarchies, enabling inferences between novel pairs. We found that both behavior and neural activity in EC and vmPFC/mOFC reflected the Euclidean distance to the retrieved hub, which was reinstated in HC. These findings reveal how abstract and discrete relational structures are represented, are combined, and enable novel inferences in the human brain.
Collapse
Affiliation(s)
- Seongmin A Park
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA; Center for Neuroscience, University of California, Davis, Davis, CA, USA.
| | - Douglas S Miller
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA; Center for Neuroscience, University of California, Davis, Davis, CA, USA
| | - Hamed Nili
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Charan Ranganath
- Center for Neuroscience, University of California, Davis, Davis, CA, USA; Department of Psychology, University of California, Davis, Davis, CA, USA
| | - Erie D Boorman
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA; Department of Psychology, University of California, Davis, Davis, CA, USA.
| |
Collapse
|
69
|
Hebart MN, Schuck NW. Current topics in Computational Cognitive Neuroscience. Neuropsychologia 2020; 147:107621. [PMID: 32898518 DOI: 10.1016/j.neuropsychologia.2020.107621] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Affiliation(s)
- Martin N Hebart
- Vision and Computational Cognition Group, Max Planck Institute for Human Cognitive and Brain Sciences, 04103, Leipzig, Germany.
| | - Nicolas W Schuck
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, 14195, Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, 14195, Berlin, Germany.
| |
Collapse
|
70
|
Wimmer GE, Liu Y, Vehar N, Behrens TEJ, Dolan RJ. Episodic memory retrieval success is associated with rapid replay of episode content. Nat Neurosci 2020; 23:1025-1033. [PMID: 32514135 PMCID: PMC7610376 DOI: 10.1038/s41593-020-0649-z] [Citation(s) in RCA: 33] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2019] [Accepted: 04/28/2020] [Indexed: 12/18/2022]
Abstract
Retrieval of everyday experiences is fundamental for informing our future decisions. The fine-grained neurophysiological mechanisms that support such memory retrieval are largely unknown. We studied participants who first experienced, without repetition, unique multicomponent 40-80-s episodes. One day later, they engaged in cued retrieval of these episodes while undergoing magnetoencephalography. By decoding individual episode elements, we found that trial-by-trial successful retrieval was supported by the sequential replay of episode elements, with a temporal compression factor of >60. The direction of replay supporting retrieval, either backward or forward, depended on whether the task goal was to retrieve elements of an episode that followed or preceded, respectively, a retrieval cue. This sequential replay was weaker in very-high-performing participants, in whom instead we found evidence for simultaneous clustered reactivation. Our results demonstrate that memory-mediated decisions are supported by a rapid replay mechanism that can flexibly shift in direction in response to task goals.
Collapse
Affiliation(s)
- G Elliott Wimmer
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK.
- Wellcome Centre for Human Neuroimaging, University College London, London, UK.
| | - Yunzhe Liu
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Neža Vehar
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Timothy E J Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Wellcome Centre for Integrative Neuroimaging, Centre for Functional Magnetic Resonance Imaging of the Brain, University of Oxford, Oxford, UK
| | - Raymond J Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| |
Collapse
|
71
|
Eldar E, Lièvre G, Dayan P, Dolan RJ. The roles of online and offline replay in planning. eLife 2020; 9:e56911. [PMID: 32553110 PMCID: PMC7299337 DOI: 10.7554/elife.56911] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2020] [Accepted: 05/13/2020] [Indexed: 11/24/2022] Open
Abstract
Animals and humans replay neural patterns encoding trajectories through their environment, both whilst they solve decision-making tasks and during rest. Both on-task and off-task replay are believed to contribute to flexible decision making, though how their relative contributions differ remains unclear. We investigated this question by using magnetoencephalography (MEG) to study human subjects while they performed a decision-making task that was designed to reveal the decision algorithms employed. We characterised subjects in terms of how flexibly each adjusted their choices to changes in temporal, spatial and reward structure. The more flexible a subject, the more they replayed trajectories during task performance, and this replay was coupled with re-planning of the encoded trajectories. The less flexible a subject, the more they replayed previously preferred trajectories during rest periods between task epochs. The data suggest that online and offline replay both participate in planning but support distinct decision strategies.
Collapse
Affiliation(s)
- Eran Eldar
- Departments of Psychology and Cognitive Sciences, Hebrew University of JerusalemJerusalemIsrael
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College LondonLondonUnited Kingdom
- Wellcome Centre for Human Neuroimaging, University College LondonLondonUnited Kingdom
| | - Gaëlle Lièvre
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College LondonLondonUnited Kingdom
- Wellcome Centre for Human Neuroimaging, University College LondonLondonUnited Kingdom
| | - Peter Dayan
- Max Planck Institute for Biological CyberneticsTübingenGermany
- University of TübingenTübingenGermany
| | - Raymond J Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College LondonLondonUnited Kingdom
- Wellcome Centre for Human Neuroimaging, University College LondonLondonUnited Kingdom
| |
Collapse
|
72
|
Bellmund JLS, Polti I, Doeller CF. Sequence Memory in the Hippocampal-Entorhinal Region. J Cogn Neurosci 2020; 32:2056-2070. [PMID: 32530378 DOI: 10.1162/jocn_a_01592] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Episodic memories are constructed from sequences of events. When recalling such a memory, we not only recall individual events, but we also retrieve information about how the sequence of events unfolded. Here, we focus on the role of the hippocampal-entorhinal region in processing and remembering sequences of events, which are thought to be stored in relational networks. We summarize evidence that temporal relations are a central organizational principle for memories in the hippocampus. Importantly, we incorporate novel insights from recent studies about the role of the adjacent entorhinal cortex in sequence memory. In rodents, the lateral entorhinal subregion carries temporal information during ongoing behavior. The human homologue is recruited during memory recall where its representations reflect the temporal relationships between events encountered in a sequence. We further introduce the idea that the hippocampal-entorhinal region might enable temporal scaling of sequence representations. Flexible changes of sequence progression speed could underlie the traversal of episodic memories and mental simulations at different paces. In conclusion, we describe how the entorhinal cortex and hippocampus contribute to remembering event sequences-a core component of episodic memory.
Collapse
Affiliation(s)
- Jacob L S Bellmund
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Ignacio Polti
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Norwegian University of Science and Technology, Trondheim, Norway
| | - Christian F Doeller
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.,Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Norwegian University of Science and Technology, Trondheim, Norway
| |
Collapse
|
73
|
Castegnetti G, Tzovara A, Khemka S, Melinščak F, Barnes GR, Dolan RJ, Bach DR. Representation of probabilistic outcomes during risky decision-making. Nat Commun 2020; 11:2419. [PMID: 32415145 PMCID: PMC7229012 DOI: 10.1038/s41467-020-16202-y] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2019] [Accepted: 04/21/2020] [Indexed: 12/19/2022] Open
Abstract
Goal-directed behaviour requires prospectively retrieving and evaluating multiple possible action outcomes. While a plethora of studies suggested sequential retrieval for deterministic choice outcomes, it remains unclear whether this is also the case when integrating multiple probabilistic outcomes of the same action. We address this question by capitalising on magnetoencephalography (MEG) in humans who made choices in a risky foraging task. We train classifiers to distinguish MEG field patterns during presentation of two probabilistic outcomes (reward, loss), and then apply these to decode such patterns during deliberation. First, decoded outcome representations have a temporal structure, suggesting alternating retrieval of the outcomes. Moreover, the probability that one or the other outcome is being represented depends on loss magnitude, but not on loss probability, and it predicts the chosen action. In summary, we demonstrate decodable outcome representations during probabilistic decision-making, which are sequentially structured, depend on task features, and predict subsequent action.
Collapse
Affiliation(s)
- Giuseppe Castegnetti
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, University of Zurich, Zurich, Switzerland.
- Neuroscience Centre Zurich, University of Zurich, Zurich, Switzerland.
- Institute of Cognitive Neuroscience, University College London, London, UK.
| | - Athina Tzovara
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, University of Zurich, Zurich, Switzerland
- Neuroscience Centre Zurich, University of Zurich, Zurich, Switzerland
- Department of Computer Science & Faculty of Medicine, University of Bern, Bern, Switzerland
- Helen Wills Neuroscience Institute, University of California, Berkeley, USA
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Saurabh Khemka
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, University of Zurich, Zurich, Switzerland
- Neuroscience Centre Zurich, University of Zurich, Zurich, Switzerland
| | - Filip Melinščak
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, University of Zurich, Zurich, Switzerland
- Neuroscience Centre Zurich, University of Zurich, Zurich, Switzerland
| | - Gareth R Barnes
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Raymond J Dolan
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Max Planck UCL Centre for Computational Psychiatry and Ageing, University College London, London, UK
| | - Dominik R Bach
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, University of Zurich, Zurich, Switzerland
- Neuroscience Centre Zurich, University of Zurich, Zurich, Switzerland
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Max Planck UCL Centre for Computational Psychiatry and Ageing, University College London, London, UK
| |
Collapse
|
74
|
Schreiner T, Staudigl T. Electrophysiological signatures of memory reactivation in humans. Philos Trans R Soc Lond B Biol Sci 2020; 375:20190293. [PMID: 32248789 PMCID: PMC7209925 DOI: 10.1098/rstb.2019.0293] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022] Open
Abstract
The reactivation of neural activity that was present during the encoding of an event is assumed to be essential for human episodic memory retrieval and the consolidation of memories during sleep. Pioneering animal work has already established a crucial role of memory reactivation to prepare and guide behaviour. Research in humans is now delineating the neural processes involved in memory reactivation during both wakefulness and sleep as well as their functional significance. Focusing on the electrophysiological signatures of memory reactivation in humans during both memory retrieval and sleep-related consolidation, this review provides an overview of the state of the art in the field. We outline recent advances, methodological developments and open questions and specifically highlight commonalities and differences in the neuronal signatures of memory reactivation during the states of wakefulness and sleep. This article is part of the Theo Murphy meeting issue ‘Memory reactivation: replaying events past, present and future’.
Collapse
Affiliation(s)
- Thomas Schreiner
- School of Psychology and Centre for Human Brain Health, University of Birmingham, Birmingham, UK.,Department of Psychology, Ludwig-Maximilians-University Munich, Munich, Germany
| | - Tobias Staudigl
- Department of Psychology, Ludwig-Maximilians-University Munich, Munich, Germany
| |
Collapse
|
75
|
van der Meer MAA, Kemere C, Diba K. Progress and issues in second-order analysis of hippocampal replay. Philos Trans R Soc Lond B Biol Sci 2020; 375:20190238. [PMID: 32248780 DOI: 10.1098/rstb.2019.0238] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023] Open
Abstract
Patterns of neural activity that occur spontaneously during sharp-wave ripple (SWR) events in the hippocampus are thought to play an important role in memory formation, consolidation and retrieval. Typical studies examining the content of SWRs seek to determine whether the identity and/or temporal order of cell firing is different from chance. Such 'first-order' analyses are focused on a single time point and template (map), and have been used to show, for instance, the existence of preplay. The major methodological challenge in first-order analyses is the construction and interpretation of different chance distributions. By contrast, 'second-order' analyses involve a comparison of SWR content between different time points, and/or between different templates. Typical second-order questions include tests of experience-dependence (replay) that compare SWR content before and after experience, and comparisons or replay between different arms of a maze. Such questions entail additional methodological challenges that can lead to biases in results and associated interpretations. We provide an inventory of analysis challenges for second-order questions about SWR content, and suggest ways of preventing, identifying and addressing possible analysis biases. Given evolving interest in understanding SWR content in more complex experimental scenarios and across different time scales, we expect these issues to become increasingly pervasive. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.
Collapse
Affiliation(s)
| | - Caleb Kemere
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, USA
| | - Kamran Diba
- Neuroscience Graduate Program, University of Michigan Medical School, Ann Arbor, MI 48109, USA.,Department of Anesthesiology, University of Michigan Medical School, Ann Arbor, MI 48109, USA
| |
Collapse
|
76
|
O C Jordan H, Navarro DM, Stringer SM. The formation and use of hierarchical cognitive maps in the brain: A neural network model. NETWORK (BRISTOL, ENGLAND) 2020; 31:37-141. [PMID: 32746663 DOI: 10.1080/0954898x.2020.1798531] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/18/2020] [Revised: 06/21/2020] [Accepted: 07/16/2020] [Indexed: 06/11/2023]
Abstract
Many researchers have tried to model how environmental knowledge is learned by the brain and used in the form of cognitive maps. However, previous work was limited in various important ways: there was little consensus on how these cognitive maps were formed and represented, the planning mechanism was inherently limited to performing relatively simple tasks, and there was little consideration of how these mechanisms would scale up. This paper makes several significant advances. Firstly, the planning mechanism used by the majority of previous work propagates a decaying signal through the network to create a gradient that points towards the goal. However, this decaying signal limited the scale and complexity of tasks that can be solved in this manner. Here we propose several ways in which a network can can self-organize a novel planning mechanism that does not require decaying activity. We also extend this model with a hierarchical planning mechanism: a layer of cells that identify frequently-used sequences of actions and reuse them to significantly increase the efficiency of planning. We speculate that our results may explain the apparent ability of humans and animals to perform model-based planning on both small and large scales without a noticeable loss of efficiency.
Collapse
|
77
|
Pezzulo G, Donnarumma F, Maisto D, Stoianov I. Planning at decision time and in the background during spatial navigation. Curr Opin Behav Sci 2019. [DOI: 10.1016/j.cobeha.2019.04.009] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
|
78
|
Tambini A, Davachi L. Awake Reactivation of Prior Experiences Consolidates Memories and Biases Cognition. Trends Cogn Sci 2019; 23:876-890. [PMID: 31445780 DOI: 10.1016/j.tics.2019.07.008] [Citation(s) in RCA: 79] [Impact Index Per Article: 15.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2019] [Revised: 07/22/2019] [Accepted: 07/22/2019] [Indexed: 01/06/2023]
Abstract
After experiences are encoded into memory, post-encoding reactivation mechanisms have been proposed to mediate long-term memory stabilization and transformation. Spontaneous reactivation of hippocampal representations, together with hippocampal-cortical interactions, are leading candidate mechanisms for promoting systems-level memory strengthening and reorganization. While the replay of spatial representations has been extensively studied in rodents, here we review recent fMRI work that provides evidence for spontaneous reactivation of nonspatial, episodic event representations in the human hippocampus and cortex, as well as for experience-dependent alterations in systems-level hippocampal connectivity. We focus on reactivation during awake post-encoding periods, relationships between reactivation and subsequent behavior, how reactivation is modulated by factors that influence consolidation, and the implications of persistent reactivation for biasing ongoing perception and cognition.
Collapse
Affiliation(s)
- Arielle Tambini
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA
| | - Lila Davachi
- Department of Psychology, Columbia University, New York, NY, USA; Nathan Kline Institute, Orangeburg, NY, USA.
| |
Collapse
|
79
|
Liu Y, Dolan RJ, Kurth-Nelson Z, Behrens TEJ. Human Replay Spontaneously Reorganizes Experience. Cell 2019; 178:640-652.e14. [PMID: 31280961 PMCID: PMC6657653 DOI: 10.1016/j.cell.2019.06.012] [Citation(s) in RCA: 188] [Impact Index Per Article: 37.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2018] [Revised: 03/29/2019] [Accepted: 06/05/2019] [Indexed: 12/29/2022]
Abstract
Knowledge abstracted from previous experiences can be transferred to aid new learning. Here, we asked whether such abstract knowledge immediately guides the replay of new experiences. We first trained participants on a rule defining an ordering of objects and then presented a novel set of objects in a scrambled order. Across two studies, we observed that representations of these novel objects were reactivated during a subsequent rest. As in rodents, human "replay" events occurred in sequences accelerated in time, compared to actual experience, and reversed their direction after a reward. Notably, replay did not simply recapitulate visual experience, but followed instead a sequence implied by learned abstract knowledge. Furthermore, each replay contained more than sensory representations of the relevant objects. A sensory code of object representations was preceded 50 ms by a code factorized into sequence position and sequence identity. We argue that this factorized representation facilitates the generalization of a previously learned structure to new objects.
Collapse
Affiliation(s)
- Yunzhe Liu
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, UK.
| | - Raymond J Dolan
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, UK
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, UK; DeepMind, London, UK
| | - Timothy E J Behrens
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, Centre for Functional Magnetic Resonance Imaging of the Brain, University of Oxford, Oxford OX3 9DU, UK
| |
Collapse
|
80
|
Schuck NW, Niv Y. Sequential replay of nonspatial task states in the human hippocampus. Science 2019; 364:eaaw5181. [PMID: 31249030 PMCID: PMC7241311 DOI: 10.1126/science.aaw5181] [Citation(s) in RCA: 117] [Impact Index Per Article: 23.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2018] [Accepted: 04/26/2019] [Indexed: 12/25/2022]
Abstract
Sequential neural activity patterns related to spatial experiences are "replayed" in the hippocampus of rodents during rest. We investigated whether replay of nonspatial sequences can be detected noninvasively in the human hippocampus. Participants underwent functional magnetic resonance imaging (fMRI) while resting after performing a decision-making task with sequential structure. Hippocampal fMRI patterns recorded at rest reflected sequentiality of previously experienced task states, with consecutive patterns corresponding to nearby states. Hippocampal sequentiality correlated with the fidelity of task representations recorded in the orbitofrontal cortex during decision-making, which were themselves related to better task performance. Our findings suggest that hippocampal replay may be important for building representations of complex, abstract tasks elsewhere in the brain and establish feasibility of investigating fast replay signals with fMRI.
Collapse
Affiliation(s)
- Nicolas W Schuck
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Lentzeallee 94, 14195 Berlin, Germany.
- Max Planck University College London (UCL) Centre for Computational Psychiatry and Ageing Research, Berlin, Germany, and London, UK
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Washington Road, Princeton, NJ 08544, USA
| | - Yael Niv
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Washington Road, Princeton, NJ 08544, USA.
| |
Collapse
|
81
|
Elliott Wimmer G, Büchel C. Learning of distant state predictions by the orbitofrontal cortex in humans. Nat Commun 2019; 10:2554. [PMID: 31186425 PMCID: PMC6560030 DOI: 10.1038/s41467-019-10597-z] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2018] [Accepted: 05/21/2019] [Indexed: 01/06/2023] Open
Abstract
Representations of our future environment are essential for planning and decision making. Previous research in humans has demonstrated that the hippocampus is a critical region for forming and retrieving associations, while the medial orbitofrontal cortex (OFC) is an important region for representing information about recent states. However, it is not clear how the brain acquires predictive representations during goal-directed learning. Here, we show using fMRI that while participants learned to find rewards in multiple different Y-maze environments, hippocampal activity was highest during initial exposure and then decayed across the remaining repetitions of each maze, consistent with a role in rapid encoding. Importantly, multivariate patterns in the OFC-VPFC came to represent predictive information about upcoming states approximately 30 s in the future. Our findings provide a mechanism by which the brain can build models of the world that span long-timescales to make predictions.
Collapse
Affiliation(s)
- G Elliott Wimmer
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, WC1B 5EH, UK.
- Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, UK.
| | - Christian Büchel
- Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, 20246, Germany
| |
Collapse
|
82
|
Bellmund JLS, Gärdenfors P, Moser EI, Doeller CF. Navigating cognition: Spatial codes for human thinking. Science 2019; 362:362/6415/eaat6766. [PMID: 30409861 DOI: 10.1126/science.aat6766] [Citation(s) in RCA: 234] [Impact Index Per Article: 46.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
The hippocampal formation has long been suggested to underlie both memory formation and spatial navigation. We discuss how neural mechanisms identified in spatial navigation research operate across information domains to support a wide spectrum of cognitive functions. In our framework, place and grid cell population codes provide a representational format to map variable dimensions of cognitive spaces. This highly dynamic mapping system enables rapid reorganization of codes through remapping between orthogonal representations across behavioral contexts, yielding a multitude of stable cognitive spaces at different resolutions and hierarchical levels. Action sequences result in trajectories through cognitive space, which can be simulated via sequential coding in the hippocampus. In this way, the spatial representational format of the hippocampal formation has the capacity to support flexible cognition and behavior.
Collapse
Affiliation(s)
- Jacob L S Bellmund
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, NTNU, Norwegian University of Science and Technology, Trondheim, Norway. .,Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands.,Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Peter Gärdenfors
- Department of Philosophy and Cognitive Science, Lund University, Lund, Sweden.,Centre for Artificial Intelligence, University of Technology Sydney, Sydney, Australia
| | - Edvard I Moser
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, NTNU, Norwegian University of Science and Technology, Trondheim, Norway
| | - Christian F Doeller
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, NTNU, Norwegian University of Science and Technology, Trondheim, Norway. .,Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| |
Collapse
|
83
|
Abstract
The mammalian hippocampus is important for normal memory function, particularly memory for places and events. Place cells, neurons within the hippocampus that have spatial receptive fields, represent information about an animal’s position. During periods of rest, but also during active task engagement, place cells spontaneously recapitulate past trajectories. Such ‘replay’ has been proposed as a mechanism necessary for a range of neurobiological functions, including systems memory consolidation, recall and spatial working memory, navigational planning, and reinforcement learning. Focusing mainly, but not exclusively, on work conducted in rodents, we describe the methodologies used to analyse replay and review evidence for its putative roles. We identify outstanding questions as well as apparent inconsistencies in existing data, making suggestions as to how these might be resolved. In particular, we find support for the involvement of replay in disparate processes, including the maintenance of hippocampal memories and decision making. We propose that the function of replay changes dynamically according to task demands placed on an organism and its current level of arousal.
Collapse
Affiliation(s)
- H Freyja Ólafsdóttir
- Research Department of Cell and Developmental Biology, UCL, Gower Street, London, WC1E 6BT, UK.
| | - Daniel Bush
- UCL Institute of Cognitive Neuroscience and UCL Institute of Neurology, 17 Queen Square, London, WC1N 3AZ, UK
| | - Caswell Barry
- Research Department of Cell and Developmental Biology, UCL, Gower Street, London, WC1E 6BT, UK.
| |
Collapse
|
84
|
Hauser TU, Will GJ, Dubois M, Dolan RJ. Annual Research Review: Developmental computational psychiatry. J Child Psychol Psychiatry 2019; 60:412-426. [PMID: 30252127 DOI: 10.1111/jcpp.12964] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 07/03/2018] [Indexed: 11/29/2022]
Abstract
Most psychiatric disorders emerge during childhood and adolescence. This is also a period that coincides with the brain undergoing substantial growth and reorganisation. However, it remains unclear how a heightened vulnerability to psychiatric disorder relates to this brain maturation. Here, we propose 'developmental computational psychiatry' as a framework for linking brain maturation to cognitive development. We argue that through modelling some of the brain's fundamental cognitive computations, and relating them to brain development, we can bridge the gap between brain and cognitive development. This in turn can lead to a richer understanding of the ontogeny of psychiatric disorders. We illustrate this perspective with examples from reinforcement learning and dopamine function. Specifically, we show how computational modelling deepens an understanding of how cognitive processes, such as reward learning, effort learning, and social learning might go awry in psychiatric disorders. Finally, we sketch the promises and limitations of a developmental computational psychiatry.
Collapse
Affiliation(s)
- Tobias U Hauser
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK.,Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Geert-Jan Will
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK.,Institute of Psychology, Leiden University, Leiden, The Netherlands
| | - Magda Dubois
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Raymond J Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK.,Wellcome Centre for Human Neuroimaging, University College London, London, UK
| |
Collapse
|
85
|
Ruzich E, Crespo‐García M, Dalal SS, Schneiderman JF. Characterizing hippocampal dynamics with MEG: A systematic review and evidence-based guidelines. Hum Brain Mapp 2019; 40:1353-1375. [PMID: 30378210 PMCID: PMC6456020 DOI: 10.1002/hbm.24445] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2018] [Revised: 10/12/2018] [Accepted: 10/16/2018] [Indexed: 12/12/2022] Open
Abstract
The hippocampus, a hub of activity for a variety of important cognitive processes, is a target of increasing interest for researchers and clinicians. Magnetoencephalography (MEG) is an attractive technique for imaging spectro-temporal aspects of function, for example, neural oscillations and network timing, especially in shallow cortical structures. However, the decrease in MEG signal-to-noise ratio as a function of source depth implies that the utility of MEG for investigations of deeper brain structures, including the hippocampus, is less clear. To determine whether MEG can be used to detect and localize activity from the hippocampus, we executed a systematic review of the existing literature and found successful detection of oscillatory neural activity originating in the hippocampus with MEG. Prerequisites are the use of established experimental paradigms, adequate coregistration, forward modeling, analysis methods, optimization of signal-to-noise ratios, and protocol trial designs that maximize contrast for hippocampal activity while minimizing those from other brain regions. While localizing activity to specific sub-structures within the hippocampus has not been achieved, we provide recommendations for improving the reliability of such endeavors.
Collapse
Affiliation(s)
- Emily Ruzich
- Department of Clinical Neurophysiology and MedTech West, Institute of Neuroscience and PhysiologySahlgrenska Academy & the University of GothenburgGothenburgSweden
| | | | - Sarang S. Dalal
- Center of Functionally Integrative NeuroscienceAarhus UniversityAarhus CDenmark
| | - Justin F. Schneiderman
- Department of Clinical Neurophysiology and MedTech West, Institute of Neuroscience and PhysiologySahlgrenska Academy & the University of GothenburgGothenburgSweden
| |
Collapse
|
86
|
Clewett D, DuBrow S, Davachi L. Transcending time in the brain: How event memories are constructed from experience. Hippocampus 2019; 29:162-183. [PMID: 30734391 PMCID: PMC6629464 DOI: 10.1002/hipo.23074] [Citation(s) in RCA: 87] [Impact Index Per Article: 17.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2018] [Revised: 01/07/2019] [Accepted: 01/09/2019] [Indexed: 11/06/2022]
Abstract
Our daily lives unfold continuously, yet when we reflect on the past, we remember those experiences as distinct and cohesive events. To understand this phenomenon, early investigations focused on how and when individuals perceive natural breakpoints, or boundaries, in ongoing experience. More recent research has examined how these boundaries modulate brain mechanisms that support long-term episodic memory. This work has revealed that a complex interplay between hippocampus and prefrontal cortex promotes the integration and separation of sequential information to help organize our experiences into mnemonic events. Here, we discuss how both temporal stability and change in one's thoughts, goals, and surroundings may provide scaffolding for these neural processes to link and separate memories across time. When learning novel or familiar sequences of information, dynamic hippocampal processes may work both independently from and in concert with other brain regions to bind sequential representations together in memory. The formation and storage of discrete episodic memories may occur both proactively as an experience unfolds. They may also occur retroactively, either during a context shift or when reactivation mechanisms bring the past into the present to allow integration. We also describe conditions and factors that shape the construction and integration of event memories across different timescales. Together these findings shed new light on how the brain transcends time to transform everyday experiences into meaningful memory representations.
Collapse
Affiliation(s)
| | - Sarah DuBrow
- Neuroscience Institute, Princeton University, USA
| | - Lila Davachi
- Department of Psychology, Columbia University, USA
- Nathan Kline Institute, Orangeburg, New York, USA
| |
Collapse
|
87
|
Muessig L, Lasek M, Varsavsky I, Cacucci F, Wills TJ. Coordinated Emergence of Hippocampal Replay and Theta Sequences during Post-natal Development. Curr Biol 2019; 29:834-840.e4. [PMID: 30773370 PMCID: PMC6408330 DOI: 10.1016/j.cub.2019.01.005] [Citation(s) in RCA: 43] [Impact Index Per Article: 8.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2018] [Revised: 12/30/2018] [Accepted: 01/02/2019] [Indexed: 12/15/2022]
Abstract
Hippocampal place cells encode an animal’s current position in space during exploration [1]. During sleep, hippocampal network activity recapitulates patterns observed during recent experience: place cells with overlapping spatial fields show a greater tendency to co-fire (“reactivation”) [2], and temporally ordered and compressed sequences of place cell firing observed during wakefulness are reinstated (“replay”) [3, 4, 5]. Reactivation and replay may underlie memory consolidation [6, 7, 8, 9, 10]. Compressed sequences of place cell firing also occur during exploration: during each cycle of the theta oscillation, the set of active place cells shifts from those signaling positions behind to those signaling positions ahead of an animal’s current location [11, 12]. These “theta sequences” have been linked to spatial planning [13]. Here, we demonstrate that, before weaning (post-natal day [P]21), offline place cell activity associated with sharp-wave ripples (SWRs) reflects predominantly stationary locations in recently visited environments. By contrast, sequential place cell firing, describing extended trajectories through space during exploration (theta sequences) and subsequent rest (replay), emerge gradually after weaning in a coordinated fashion, possibly due to a progressive decrease in the threshold for experience-driven plasticity. Hippocampus-dependent learning and memory emerge late in altricial mammals [14, 15, 16, 17], appearing around weaning in rats and slowly maturing thereafter [14, 15]. In contrast, spatially localized firing is observed 1 week earlier (with reduced spatial tuning and stability) [18, 19, 20, 21]. By examining the development of hippocampal reactivation, replay, and theta sequences, we show that the coordinated maturation of offline consolidation and online sequence generation parallels the late emergence of hippocampal memory in the rat. Hippocampal activity encoding single places is reactivated during sleep in young rats The threshold for plasticity-driven reactivation is higher during early development Sequential firing linking contiguous places emerges gradually during development Maturation of online and offline sequential activity and memory are coordinated
Collapse
Affiliation(s)
- Laurenz Muessig
- Cell and Developmental Biology, University College London, Gower Street, London WC1E 6BT, UK
| | - Michal Lasek
- Neuroscience, Physiology and Pharmacology, University College London, Gower Street, London WC1E 6BT, UK
| | - Isabella Varsavsky
- Cell and Developmental Biology, University College London, Gower Street, London WC1E 6BT, UK
| | - Francesca Cacucci
- Neuroscience, Physiology and Pharmacology, University College London, Gower Street, London WC1E 6BT, UK.
| | - Thomas Joseph Wills
- Cell and Developmental Biology, University College London, Gower Street, London WC1E 6BT, UK.
| |
Collapse
|
88
|
Momennejad I, Otto AR, Daw ND, Norman KA. Offline replay supports planning in human reinforcement learning. eLife 2018; 7:32548. [PMID: 30547886 PMCID: PMC6303108 DOI: 10.7554/elife.32548] [Citation(s) in RCA: 62] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2017] [Accepted: 12/04/2018] [Indexed: 11/13/2022] Open
Abstract
Making decisions in sequentially structured tasks requires integrating distally acquired information. The extensive computational cost of such integration challenges planning methods that integrate online, at decision time. Furthermore, it remains unclear whether 'offline' integration during replay supports planning, and if so which memories should be replayed. Inspired by machine learning, we propose that (a) offline replay of trajectories facilitates integrating representations that guide decisions, and (b) unsigned prediction errors (uncertainty) trigger such integrative replay. We designed a 2-step revaluation task for fMRI, whereby participants needed to integrate changes in rewards with past knowledge to optimally replan decisions. As predicted, we found that (a) multi-voxel pattern evidence for off-task replay predicts subsequent replanning; (b) neural sensitivity to uncertainty predicts subsequent replay and replanning; (c) off-task hippocampus and anterior cingulate activity increase when revaluation is required. These findings elucidate how the brain leverages offline mechanisms in planning and goal-directed behavior under uncertainty.
Collapse
Affiliation(s)
- Ida Momennejad
- Princeton Neuroscience Institute, Princeton University, New Jersey, United States
| | - A Ross Otto
- Department of Psychology, McGill University, Montreal, Canada
| | - Nathaniel D Daw
- Princeton Neuroscience Institute, Princeton University, New Jersey, United States
| | - Kenneth A Norman
- Princeton Neuroscience Institute, Princeton University, New Jersey, United States
| |
Collapse
|
89
|
Huang Q, Jia J, Han Q, Luo H. Fast-backward replay of sequentially memorized items in humans. eLife 2018; 7:35164. [PMID: 30334735 PMCID: PMC6231774 DOI: 10.7554/elife.35164] [Citation(s) in RCA: 26] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2018] [Accepted: 10/18/2018] [Indexed: 11/24/2022] Open
Abstract
Storing temporal sequences of events (i.e., sequence memory) is fundamental to many cognitive functions. However, it is unknown how the sequence order information is maintained and represented in working memory and its behavioral significance, particularly in human subjects. We recorded electroencephalography (EEG) in combination with a temporal response function (TRF) method to dissociate item-specific neuronal reactivations. We demonstrate that serially remembered items are successively reactivated during memory retention. The sequential replay displays two interesting properties compared to the actual sequence. First, the item-by-item reactivation is compressed within a 200 – 400 ms window, suggesting that external events are associated within a plasticity-relevant window to facilitate memory consolidation. Second, the replay is in a temporally reversed order and is strongly related to the recency effect in behavior. This fast-backward replay, previously revealed in rat hippocampus and demonstrated here in human cortical activities, might constitute a general neural mechanism for sequence memory and learning. Have you ever played the ‘Memory Maze Challenge’ game, or its predecessor from the 1980s, ‘Simon’? Players must memorize a sequence of colored lights, and then reproduce the sequence by tapping the colors on a pad. The sequence becomes longer with each trial, making the task more and more difficult. One wrong response and the game is over. Storing and retrieving sequences is key to many cognitive processes, from following speech to hitting a tennis ball to recalling what you did last week. Such tasks require memorizing the order in which items occur as well as the items themselves. But how do we hold this information in memory? Huang et al. reveal the answer by using scalp electrodes to record the brain activity of healthy volunteers as they memorize and then recall a sequence. Memorizing, or encoding, each of the items in the sequence triggered a distinct pattern of brain activity. As the volunteers held the sequence in memory, their brains replayed these activity patterns one after the other. But this replay showed two non-intuitive features. First, it was speeded up relative to the original encoding. In fact, the brain compressed the entire sequence into about 200 to 400 milliseconds. Second, the brain replayed the sequence backwards. The activity pattern corresponding to the last item was replayed first, while that corresponding to the first item was replayed last. This ‘fast-backward’ replay may explain why we tend to recall items at the end of a list better than those in the middle, a phenomenon known as the recency effect. The results of Huang et al. suggest that when we hold a list of items in memory, the brain does not replay the list in its original form, like an echo. Instead, the brain restructures and reorganizes the list, compressing and reversing it. This process, which is also seen in rodents, helps the brain to incorporate the list of items into existing neuronal networks for memory storage.
Collapse
Affiliation(s)
- Qiaoli Huang
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China.,PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China.,Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| | - Jianrong Jia
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China.,PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China.,Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China.,Peking-Tsinghua Center for Life Sciences, Peking University, Beijing, China
| | - Qiming Han
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China.,PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China.,Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China.,Peking-Tsinghua Center for Life Sciences, Peking University, Beijing, China
| | - Huan Luo
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China.,PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China.,Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| |
Collapse
|
90
|
Schapiro AC, McDevitt EA, Rogers TT, Mednick SC, Norman KA. Human hippocampal replay during rest prioritizes weakly learned information and predicts memory performance. Nat Commun 2018; 9:3920. [PMID: 30254219 PMCID: PMC6156217 DOI: 10.1038/s41467-018-06213-1] [Citation(s) in RCA: 117] [Impact Index Per Article: 19.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2018] [Accepted: 08/20/2018] [Indexed: 12/20/2022] Open
Abstract
The hippocampus replays experiences during quiet rest periods, and this replay benefits subsequent memory. A critical open question is how memories are prioritized for this replay. We used functional magnetic resonance imaging (fMRI) pattern analysis to track item-level replay in the hippocampus during an awake rest period after participants studied 15 objects and completed a memory test. Objects that were remembered less well were replayed more during the subsequent rest period, suggesting a prioritization process in which weaker memories—memories most vulnerable to forgetting—are selected for replay. In a second session 12 hours later, more replay of an object during a rest period predicted better subsequent memory for that object. Replay predicted memory improvement across sessions only for participants who slept during that interval. Our results provide evidence that replay in the human hippocampus prioritizes weakly learned information, predicts subsequent memory performance, and relates to memory improvement across a delay with sleep. The hippocampus is known to 'replay' experiences and memories during rest periods, but it is unclear how particular memories are prioritized for replay. Here, the authors show that information that is remembered less well is replayed more often, suggesting that weaker memories are selected for replay.
Collapse
Affiliation(s)
- Anna C Schapiro
- Department of Psychiatry, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, MA, 02215, USA.
| | - Elizabeth A McDevitt
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, 08544, USA
| | - Timothy T Rogers
- Department of Psychology, University of Wisconsin-Madison, Madison, WI, 53706, USA
| | - Sara C Mednick
- Department of Cognitive Sciences, University of California-Irvine, Irvine, CA, 92617, USA
| | - Kenneth A Norman
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, 08544, USA
| |
Collapse
|
91
|
Eldar E, Bae GJ, Kurth-Nelson Z, Dayan P, Dolan RJ. Magnetoencephalography decoding reveals structural differences within integrative decision processes. Nat Hum Behav 2018; 2:670-681. [PMID: 31346283 DOI: 10.1038/s41562-018-0423-3] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2017] [Accepted: 07/26/2018] [Indexed: 11/09/2022]
Abstract
When confronted with complex inputs consisting of multiple elements, humans use various strategies to integrate the elements quickly and accurately. For instance, accuracy may be improved by processing elements one at a time1-4 or over extended periods5-8; speed can increase if the internal representation of elements is accelerated9,10. However, little is known about how humans actually approach these challenges because behavioural findings can be accounted for by multiple alternative process models11 and neuroimaging investigations typically rely on haemodynamic signals that change too slowly. Consequently, to uncover the fast neural dynamics that support information integration, we decoded magnetoencephalographic signals that were recorded as human subjects performed a complex decision task. Our findings reveal three sources of individual differences in the temporal structure of the integration process-sequential representation, partial reinstatement and early computation-each having a dissociable effect on how subjects handled problem complexity and temporal constraints. Our findings shed new light on the structure and influence of self-determined neural integration processes.
Collapse
Affiliation(s)
- Eran Eldar
- Wellcome Trust Centre for Neuroimaging, University College London, London, UK. .,Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK.
| | - Gyung Jin Bae
- Wellcome Trust Centre for Neuroimaging, University College London, London, UK.,Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Zeb Kurth-Nelson
- Wellcome Trust Centre for Neuroimaging, University College London, London, UK.,Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Peter Dayan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK.,Gatsby Computational Neuroscience Unit, University College London, London, UK
| | - Raymond J Dolan
- Wellcome Trust Centre for Neuroimaging, University College London, London, UK.,Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK
| |
Collapse
|
92
|
Pu Y, Cornwell BR, Cheyne D, Johnson BW. High-gamma activity in the human hippocampus and parahippocampus during inter-trial rest periods of a virtual navigation task. Neuroimage 2018; 178:92-103. [DOI: 10.1016/j.neuroimage.2018.05.029] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2017] [Revised: 05/02/2018] [Accepted: 05/10/2018] [Indexed: 12/14/2022] Open
|
93
|
Replay of Episodic Memories in the Rat. Curr Biol 2018; 28:1628-1634.e7. [PMID: 29754898 DOI: 10.1016/j.cub.2018.04.006] [Citation(s) in RCA: 47] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2017] [Revised: 03/05/2018] [Accepted: 04/03/2018] [Indexed: 01/01/2023]
Abstract
Vivid episodic memories in people have been characterized as the replay of multiple unique events in sequential order [1-3]. The hippocampus plays a critical role in episodic memories in both people and rodents [2, 4-6]. Although rats remember multiple unique episodes [7, 8], it is currently unknown if animals "replay" episodic memories. Therefore, we developed an animal model of episodic memory replay. Here, we show that rats can remember a trial-unique stream of multiple episodes and the order in which these events occurred by engaging hippocampal-dependent episodic memory replay. We document that rats rely on episodic memory replay to remember the order of events rather than relying on non-episodic memories. Replay of episodic memories survives a long retention-interval challenge and interference from the memory of other events, which documents that replay is part of long-term episodic memory. The chemogenetic activating drug clozapine N-oxide (CNO), but not vehicle, reversibly impairs episodic memory replay in rats previously injected bilaterally in the hippocampus with a recombinant viral vector containing an inhibitory designer receptor exclusively activated by a designer drug (DREADD; AAV8-hSyn-hM4Di-mCherry). By contrast, two non-episodic memory assessments are unaffected by CNO, showing selectivity of this hippocampal-dependent impairment. Our approach provides an animal model of episodic memory replay, a process by which the rat searches its representations in episodic memory in sequential order to find information. Our findings using rats suggest that the ability to replay a stream of episodic memories is quite old in the evolutionary timescale.
Collapse
|
94
|
Afsardeir A, Keramati M. Behavioural signatures of backward planning in animals. Eur J Neurosci 2018; 47:479-487. [DOI: 10.1111/ejn.13851] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2017] [Revised: 01/15/2018] [Accepted: 01/21/2018] [Indexed: 11/30/2022]
Affiliation(s)
- Arsham Afsardeir
- Control and Intelligence Processing Center of Excellence; School of ECE; College of Engineering; University of Tehran; Tehran Iran
| | - Mehdi Keramati
- Gatsby Computational Neuroscience Unit; Sainsbury Wellcome Centre; University College London; 25 Howland Street London W1T 4JG UK
- Max Planck Centre for Computational Psychiatry and Ageing Research; University College London; London UK
| |
Collapse
|
95
|
Schirrmeister RT, Springenberg JT, Fiederer LDJ, Glasstetter M, Eggensperger K, Tangermann M, Hutter F, Burgard W, Ball T. Deep learning with convolutional neural networks for EEG decoding and visualization. Hum Brain Mapp 2017; 38:5391-5420. [PMID: 28782865 PMCID: PMC5655781 DOI: 10.1002/hbm.23730] [Citation(s) in RCA: 824] [Impact Index Per Article: 117.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2017] [Revised: 05/31/2017] [Accepted: 07/05/2017] [Indexed: 02/06/2023] Open
Abstract
Deep learning with convolutional neural networks (deep ConvNets) has revolutionized computer vision through end-to-end learning, that is, learning from the raw data. There is increasing interest in using deep ConvNets for end-to-end EEG analysis, but a better understanding of how to design and train ConvNets for end-to-end EEG decoding and how to visualize the informative EEG features the ConvNets learn is still needed. Here, we studied deep ConvNets with a range of different architectures, designed for decoding imagined or executed tasks from raw EEG. Our results show that recent advances from the machine learning field, including batch normalization and exponential linear units, together with a cropped training strategy, boosted the deep ConvNets decoding performance, reaching at least as good performance as the widely used filter bank common spatial patterns (FBCSP) algorithm (mean decoding accuracies 82.1% FBCSP, 84.0% deep ConvNets). While FBCSP is designed to use spectral power modulations, the features used by ConvNets are not fixed a priori. Our novel methods for visualizing the learned features demonstrated that ConvNets indeed learned to use spectral power modulations in the alpha, beta, and high gamma frequencies, and proved useful for spatially mapping the learned features by revealing the topography of the causal contributions of features in different frequency bands to the decoding decision. Our study thus shows how to design and train ConvNets to decode task-related information from the raw EEG without handcrafted features and highlights the potential of deep ConvNets combined with advanced visualization techniques for EEG-based brain mapping. Hum Brain Mapp 38:5391-5420, 2017. © 2017 Wiley Periodicals, Inc.
Collapse
Affiliation(s)
- Robin Tibor Schirrmeister
- Translational Neurotechnology Lab, Epilepsy Center, Medical Center – University of Freiburg, Engelberger Str. 21Freiburg79106Germany
- BrainLinks‐BrainTools Cluster of Excellence, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
| | - Jost Tobias Springenberg
- BrainLinks‐BrainTools Cluster of Excellence, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
- Machine Learning LabComputer Science Dept, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
| | - Lukas Dominique Josef Fiederer
- Translational Neurotechnology Lab, Epilepsy Center, Medical Center – University of Freiburg, Engelberger Str. 21Freiburg79106Germany
- BrainLinks‐BrainTools Cluster of Excellence, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
- Neurobiology and BiophysicsFaculty of Biology, University of Freiburg, Hansastr. 9aFreiburg79104Germany
| | - Martin Glasstetter
- Translational Neurotechnology Lab, Epilepsy Center, Medical Center – University of Freiburg, Engelberger Str. 21Freiburg79106Germany
- BrainLinks‐BrainTools Cluster of Excellence, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
| | - Katharina Eggensperger
- BrainLinks‐BrainTools Cluster of Excellence, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
- Machine Learning for Automated Algorithm Design LabComputer Science Dept, University of Freiburg, Georges‐Köhler‐Allee 52Freiburg im Breisgau79110Germany
| | - Michael Tangermann
- BrainLinks‐BrainTools Cluster of Excellence, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
- Brain State Decoding LabComputer Science Dept, University of Freiburg, Albertstr. 23Freiburg79104Germany
| | - Frank Hutter
- BrainLinks‐BrainTools Cluster of Excellence, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
- Machine Learning for Automated Algorithm Design LabComputer Science Dept, University of Freiburg, Georges‐Köhler‐Allee 52Freiburg im Breisgau79110Germany
| | - Wolfram Burgard
- BrainLinks‐BrainTools Cluster of Excellence, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
- Autonomous Intelligent Systems LabComputer Science Dept, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
| | - Tonio Ball
- Translational Neurotechnology Lab, Epilepsy Center, Medical Center – University of Freiburg, Engelberger Str. 21Freiburg79106Germany
- BrainLinks‐BrainTools Cluster of Excellence, University of Freiburg, Georges‐Köhler‐Allee 79Freiburg79110Germany
| |
Collapse
|
96
|
Manohar SG, Pertzov Y, Husain M. Short-term memory for spatial, sequential and duration information. Curr Opin Behav Sci 2017; 17:20-26. [PMID: 29167809 PMCID: PMC5678495 DOI: 10.1016/j.cobeha.2017.05.023] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
Abstract
Analog report methods provide novel insights on STM for space and time. Space and time may be used to bind features in STM. The hippocampus is involved in object-location binding in STM.
Space and time appear to play key roles in the way that information is organized in short-term memory (STM). Some argue that they are crucial contexts within which other stored features are embedded, allowing binding of information that belongs together within STM. Here we review recent behavioral, neurophysiological and imaging studies that have sought to investigate the nature of spatial, sequential and duration representations in STM, and how these might break down in disease. Findings from these studies point to an important role of the hippocampus and other medial temporal lobe structures in aspects of STM, challenging conventional accounts of involvement of these regions in only long-term memory.
Collapse
Affiliation(s)
- Sanjay G Manohar
- Dept Experimental Psychology and Nuffield Dept of Clinical Neuroscience, University of Oxford, United Kingdom
| | - Yoni Pertzov
- Dept of Psychology, The Hebrew University of Jerusalem, Israel
| | - Masud Husain
- Dept Experimental Psychology and Nuffield Dept of Clinical Neuroscience, University of Oxford, United Kingdom
| |
Collapse
|
97
|
Hassabis D, Kumaran D, Summerfield C, Botvinick M. Neuroscience-Inspired Artificial Intelligence. Neuron 2017; 95:245-258. [PMID: 28728020 DOI: 10.1016/j.neuron.2017.06.011] [Citation(s) in RCA: 454] [Impact Index Per Article: 64.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2017] [Revised: 06/03/2017] [Accepted: 06/06/2017] [Indexed: 01/29/2023]
Abstract
The fields of neuroscience and artificial intelligence (AI) have a long and intertwined history. In more recent times, however, communication and collaboration between the two fields has become less commonplace. In this article, we argue that better understanding biological brains could play a vital role in building intelligent machines. We survey historical interactions between the AI and neuroscience fields and emphasize current advances in AI that have been inspired by the study of neural computation in humans and other animals. We conclude by highlighting shared themes that may be key for advancing future research in both fields.
Collapse
Affiliation(s)
- Demis Hassabis
- DeepMind, 5 New Street Square, London, UK; Gatsby Computational Neuroscience Unit, 25 Howland Street, London, UK.
| | - Dharshan Kumaran
- DeepMind, 5 New Street Square, London, UK; Institute of Cognitive Neuroscience, University College London, 17 Queen Square, London, UK
| | - Christopher Summerfield
- DeepMind, 5 New Street Square, London, UK; Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Matthew Botvinick
- DeepMind, 5 New Street Square, London, UK; Gatsby Computational Neuroscience Unit, 25 Howland Street, London, UK
| |
Collapse
|
98
|
Pezzulo G, Kemere C, van der Meer MAA. Internally generated hippocampal sequences as a vantage point to probe future-oriented cognition. Ann N Y Acad Sci 2017; 1396:144-165. [PMID: 28548460 DOI: 10.1111/nyas.13329] [Citation(s) in RCA: 43] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2016] [Revised: 01/31/2017] [Accepted: 02/07/2017] [Indexed: 12/22/2022]
Abstract
Information processing in the rodent hippocampus is fundamentally shaped by internally generated sequences (IGSs), expressed during two different network states: theta sequences, which repeat and reset at the ∼8 Hz theta rhythm associated with active behavior, and punctate sharp wave-ripple (SWR) sequences associated with wakeful rest or slow-wave sleep. A potpourri of diverse functional roles has been proposed for these IGSs, resulting in a fragmented conceptual landscape. Here, we advance a unitary view of IGSs, proposing that they reflect an inferential process that samples a policy from the animal's generative model, supported by hippocampus-specific priors. The same inference affords different cognitive functions when the animal is in distinct dynamical modes, associated with specific functional networks. Theta sequences arise when inference is coupled to the animal's action-perception cycle, supporting online spatial decisions, predictive processing, and episode encoding. SWR sequences arise when the animal is decoupled from the action-perception cycle and may support offline cognitive processing, such as memory consolidation, the prospective simulation of spatial trajectories, and imagination. We discuss the empirical bases of this proposal in relation to rodent studies and highlight how the proposed computational principles can shed light on the mechanisms of future-oriented cognition in humans.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Caleb Kemere
- Electrical and Computer Engineering, Rice University, Houston, Texas
| | | |
Collapse
|
99
|
van der Meer MAA, Carey AA, Tanaka Y. Optimizing for generalization in the decoding of internally generated activity in the hippocampus. Hippocampus 2017; 27:580-595. [PMID: 28177571 DOI: 10.1002/hipo.22714] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2016] [Revised: 01/23/2017] [Accepted: 01/24/2017] [Indexed: 12/27/2022]
Abstract
The decoding of a sensory or motor variable from neural activity benefits from a known ground truth against which decoding performance can be compared. In contrast, the decoding of covert, cognitive neural activity, such as occurs in memory recall or planning, typically cannot be compared to a known ground truth. As a result, it is unclear how decoders of such internally generated activity should be configured in practice. We suggest that if the true code for covert activity is unknown, decoders should be optimized for generalization performance using cross-validation. Using ensemble recording data from hippocampal place cells, we show that this cross-validation approach results in different decoding error, different optimal decoding parameters, and different distributions of error across the decoded variable space. In addition, we show that a minor modification to the commonly used Bayesian decoding procedure, which enables the use of spike density functions, results in substantially lower decoding errors. These results have implications for the interpretation of covert neural activity, and suggest easy-to-implement changes to commonly used procedures across domains, with applications to hippocampal place cells in particular. © 2017 Wiley Periodicals, Inc.
Collapse
Affiliation(s)
| | - Alyssa A Carey
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, North Hampshire
| | - Youki Tanaka
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, North Hampshire
| |
Collapse
|