1
|
Huang Q, Luo H. Shared structure facilitates working memory of multiple sequences. eLife 2024; 12:RP93158. [PMID: 39046319 PMCID: PMC11268885 DOI: 10.7554/elife.93158] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/25/2024] Open
Abstract
Daily experiences often involve the processing of multiple sequences, yet storing them challenges the limited capacity of working memory (WM). To achieve efficient memory storage, relational structures shared by sequences would be leveraged to reorganize and compress information. Here, participants memorized a sequence of items with different colors and spatial locations and later reproduced the full color and location sequences one after another. Crucially, we manipulated the consistency between location and color sequence trajectories. First, sequences with consistent trajectories demonstrate improved memory performance and a trajectory correlation between reproduced color and location sequences. Second, sequences with consistent trajectories show neural reactivation of common trajectories, and display spontaneous replay of color sequences when recalling locations. Finally, neural reactivation correlates with WM behavior. Our findings suggest that a shared common structure is leveraged for the storage of multiple sequences through compressed encoding and neural replay, together facilitating efficient information organization in WM.
Collapse
Affiliation(s)
- Qiaoli Huang
- School of Psychological and Cognitive Sciences, Peking UniversityBeijingChina
- PKU-IDG/McGovern Institute for Brain Research, Peking UniversityBeijingChina
- Beijing Key Laboratory of Behavior and Mental Health, Peking UniversityBeijingChina
- Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
| | - Huan Luo
- School of Psychological and Cognitive Sciences, Peking UniversityBeijingChina
- PKU-IDG/McGovern Institute for Brain Research, Peking UniversityBeijingChina
- Beijing Key Laboratory of Behavior and Mental Health, Peking UniversityBeijingChina
| |
Collapse
|
2
|
Sheridan CL, Panoz-Brown D, Shiffrin RM, Crystal JD. Validation of a rodent model of episodic memory replay. Learn Behav 2024:10.3758/s13420-024-00632-5. [PMID: 39020162 DOI: 10.3758/s13420-024-00632-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/12/2024] [Indexed: 07/19/2024]
Abstract
Vivid episodic memories in humans have been described as the replay of the flow of past events in sequential order. Recently, Panoz-Brown et al. Current Biology, 28, 1628-1634, (2018) developed an olfactory memory task in which rats were presented with a list of trial-unique odors in an encoding context; next, in a distinctive memory assessment context, the rats were rewarded for choosing the second to last item from the list while avoiding other items from the list. In a different memory assessment context, the fourth to last item was rewarded. According to the episodic memory replay hypothesis, the rat remembers the list items and searches these items to find the item at the targeted locations in the list. However, events presented sequentially differ in memory trace strength, allowing a rat to use the relative familiarity of the memory traces, instead of episodic memory replay, to solve the task. Here, we directly manipulated memory trace strength by manipulating the odor intensity of target odors in both the list presentation and memory assessment. The rats relied on episodic memory replay to solve the memory assessment in conditions in which reliance on memory trace strength is ruled out. We conclude that rats are able to replay episodic memories.
Collapse
Affiliation(s)
- Cassandra L Sheridan
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10TH St., Bloomington, IN, 47405, USA
| | - Danielle Panoz-Brown
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10TH St., Bloomington, IN, 47405, USA
| | - Richard M Shiffrin
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10TH St., Bloomington, IN, 47405, USA
| | - Jonathon D Crystal
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10TH St., Bloomington, IN, 47405, USA.
| |
Collapse
|
3
|
Jensen KT, Hennequin G, Mattar MG. A recurrent network model of planning explains hippocampal replay and human behavior. Nat Neurosci 2024; 27:1340-1348. [PMID: 38849521 PMCID: PMC11239510 DOI: 10.1038/s41593-024-01675-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2023] [Accepted: 05/07/2024] [Indexed: 06/09/2024]
Abstract
When faced with a novel situation, people often spend substantial periods of time contemplating possible futures. For such planning to be rational, the benefits to behavior must compensate for the time spent thinking. Here, we capture these features of behavior by developing a neural network model where planning itself is controlled by the prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences from its own policy, which we call 'rollouts'. In a spatial navigation task, the agent learns to plan when it is beneficial, which provides a normative explanation for empirical variability in human thinking times. Additionally, the patterns of policy rollouts used by the artificial agent closely resemble patterns of rodent hippocampal replays. Our work provides a theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by-and adaptively affect-prefrontal dynamics.
Collapse
Affiliation(s)
- Kristopher T Jensen
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK.
- Sainsbury Wellcome Centre, University College London, London, UK.
| | - Guillaume Hennequin
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK
| | - Marcelo G Mattar
- Department of Cognitive Science, University of California, San Diego, CA, USA
- Department of Psychology, New York University, New York, NY, USA
| |
Collapse
|
4
|
Sheridan CL, Bonner L, Crystal JD. Replay of incidentally encoded novel odors in the rat. Anim Cogn 2024; 27:43. [PMID: 38874623 PMCID: PMC11178560 DOI: 10.1007/s10071-024-01880-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2024] [Revised: 05/28/2024] [Accepted: 05/29/2024] [Indexed: 06/15/2024]
Abstract
Although events are not always known to be important when they occur, people can remember details about such incidentally encoded information using episodic memory. Sheridan et al. (2024) argued that rats replayed episodic memories of incidentally encoded information in an unexpected assessment of memory. In one task, rats reported the third-last item in an explicitly encoded list of trial-unique odors. In a second task, rats foraged in a radial maze in the absence of odors. On a critical test, rats foraged in the maze, but scented lids covered the food. Next, memory of the third-last odor was assessed. The rats correctly answered the unexpected question. Because the odors used in the critical test were the same as those used during training, automatically encoding odors for the purpose of taking an upcoming test of memory (stimulus generalization) may have been encouraged. Here, we provided an opportunity for incidental encoding of novel odors. Previously trained rats foraged in the radial maze with entirely novel odors covering the food. Next, memory of the third-last odor was assessed. The rats correctly answered the unexpected question. High accuracy when confronted with novel odors provides evidence that the rats did not automatically encode odors for the purpose of taking an upcoming test, ruling out stimulus generalization. We conclude that rats encode multiple pieces of putatively unimportant information, and later replayed a stream of novel episodic memories when that information was needed to solve an unexpected problem.
Collapse
Affiliation(s)
- Cassandra L Sheridan
- Department of Psychological & Brain Sciences, Indiana University, Bloomington, IN, USA
| | - Lauren Bonner
- Department of Psychological & Brain Sciences, Indiana University, Bloomington, IN, USA
| | - Jonathon D Crystal
- Department of Psychological & Brain Sciences, Indiana University, Bloomington, IN, USA.
| |
Collapse
|
5
|
Kern S, Nagel J, Gerchen MF, Gürsoy Ç, Meyer-Lindenberg A, Kirsch P, Dolan RJ, Gais S, Feld GB. Reactivation strength during cued recall is modulated by graph distance within cognitive maps. eLife 2024; 12:RP93357. [PMID: 38810249 PMCID: PMC11136493 DOI: 10.7554/elife.93357] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/31/2024] Open
Abstract
Declarative memory retrieval is thought to involve reinstatement of neuronal activity patterns elicited and encoded during a prior learning episode. Furthermore, it is suggested that two mechanisms operate during reinstatement, dependent on task demands: individual memory items can be reactivated simultaneously as a clustered occurrence or, alternatively, replayed sequentially as temporally separate instances. In the current study, participants learned associations between images that were embedded in a directed graph network and retained this information over a brief 8 min consolidation period. During a subsequent cued recall session, participants retrieved the learned information while undergoing magnetoencephalographic recording. Using a trained stimulus decoder, we found evidence for clustered reactivation of learned material. Reactivation strength of individual items during clustered reactivation decreased as a function of increasing graph distance, an ordering present solely for successful retrieval but not for retrieval failure. In line with previous research, we found evidence that sequential replay was dependent on retrieval performance and was most evident in low performers. The results provide evidence for distinct performance-dependent retrieval mechanisms, with graded clustered reactivation emerging as a plausible mechanism to search within abstract cognitive maps.
Collapse
Affiliation(s)
- Simon Kern
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Addiction Behavior and Addiction Medicine, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
| | - Juliane Nagel
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Addiction Behavior and Addiction Medicine, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
| | - Martin F Gerchen
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Department of Psychology, Ruprecht Karl University of HeidelbergHeidelbergGermany
- Bernstein Center for Computational Neuroscience Heidelberg/MannheimMannheimGermany
| | - Çağatay Gürsoy
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Addiction Behavior and Addiction Medicine, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
| | - Andreas Meyer-Lindenberg
- Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Bernstein Center for Computational Neuroscience Heidelberg/MannheimMannheimGermany
| | - Peter Kirsch
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Department of Psychology, Ruprecht Karl University of HeidelbergHeidelbergGermany
- Bernstein Center for Computational Neuroscience Heidelberg/MannheimMannheimGermany
| | - Raymond J Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
- Wellcome Centre for Human Neuroimaging, University College LondonLondonUnited Kingdom
| | - Steffen Gais
- Institute of Medical Psychology and Behavioral Neurobiology, Eberhard-Karls-University TübingenTübingenGermany
| | - Gordon B Feld
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Addiction Behavior and Addiction Medicine, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Department of Psychology, Ruprecht Karl University of HeidelbergHeidelbergGermany
| |
Collapse
|
6
|
Guo Z, Wang J, Jing T, Fu L. Investigating the interpretability of schizophrenia EEG mechanism through a 3DCNN-based hidden layer features aggregation framework. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 247:108105. [PMID: 38447316 DOI: 10.1016/j.cmpb.2024.108105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/25/2023] [Revised: 02/07/2024] [Accepted: 02/26/2024] [Indexed: 03/08/2024]
Abstract
BACKGROUND AND OBJECTIVE Electroencephalogram (EEG) signals record brain activity, with growing interest in quantifying neural activity through complexity analysis as a potential biological marker for schizophrenia. Presently, EEG complexity analysis primarily relies on manual feature extraction, which is subjective and yields varied findings in studies involving schizophrenia and healthy controls. METHODS This study aims to leverage deep learning methods for enhanced EEG complexity exploration, aiding early schizophrenia screening and diagnosis. Our proposed approach utilizes a three-dimensional Convolutional Neural Network (3DCNN) to extract enhanced data features for early schizophrenia identification and subsequent complexity analysis. Leveraging the spatiotemporal capabilities of 3DCNN, we extract advanced latent features and employ knowledge distillation to reintegrate these features into the original channels, creating feature-enhanced data. RESULTS We employ a 10-fold cross-validation strategy, achieving the average accuracies of 99.46% and 98.06% in subject-dependent experiments on Dataset 1(14SZ and 14HC) and Dataset 2 (45SZ and 39HC). The average accuracy for subject-independent is 96.04% and 92.67% on both datasets. Feature extraction and classification are conducted on both the re-aggregated data and the original data. Our results demonstrate that re-aggregated data exhibit superior classification performance and a more stable training process after feature extraction. In the complexity analysis of re-aggregated data, we observe lower entropy features in schizophrenic patients compared to healthy controls, with more pronounced differences in the temporal and frontal lobes. Analyzing Katz's Fractal Dimension (KFD) across three sub-bands of lobe channels reveals the lowest α band KFD value in schizophrenia patients. CONCLUSIONS This emphasizes the ability of our method to enhance the discrimination and interpretability in schizophrenia detection and analysis. Our approach enhances the potential for EEG-based schizophrenia diagnosis by leveraging deep learning, offering superior discrimination capabilities and richer interpretive insights.
Collapse
Affiliation(s)
- Zhifen Guo
- College of Information Science and Engineering, Northeastern University, Shenyang, China.
| | - Jiao Wang
- College of Information Science and Engineering, Northeastern University, Shenyang, China.
| | - Tianyu Jing
- College of Information Science and Engineering, Northeastern University, Shenyang, China.
| | - Longyue Fu
- College of Information Science and Engineering, Northeastern University, Shenyang, China.
| |
Collapse
|
7
|
Abstract
A fundamental question in the development of animal models of episodic memory concerns the role of temporal processes in episodic memory. Gallistel, (1990) developed a framework in which animals remember specific features about an event, including the time of occurrence of the event and its location in space. Gallistel proposed that timing is based on a series of biological oscillators, spanning a wide range of periods. Accordingly, a snapshot of the phases of multiple oscillators provides a representation of the time of occurrence of the event. I review research on basic timing mechanisms that may support memory for times of occurrence. These studies suggest that animals use biological oscillators to represent time. Next, I describe recently developed animal models of episodic memory that highlight the importance of temporal representations in memory. One line of research suggests that an oscillator representation of time supports episodic memory. A second line of research highlights the flow of events in time in episodic memory. Investigations that integrate time and memory may advance the development of animal models of episodic memory.
Collapse
Affiliation(s)
- Jonathon D Crystal
- Department of Psychological & Brain Science, Indiana University, 1101 E 10TH ST, Bloomington, IN, 47405, USA.
| |
Collapse
|
8
|
Sheridan CL, Lang S, Knappenberger M, Albers C, Loper R, Tillett B, Sanchez J, Wilcox A, Harrison T, Panoz-Brown D, Crystal JD. Replay of incidentally encoded episodic memories in the rat. Curr Biol 2024; 34:641-647.e5. [PMID: 38218186 DOI: 10.1016/j.cub.2023.12.043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 04/03/2023] [Accepted: 12/13/2023] [Indexed: 01/15/2024]
Abstract
Although events are not always known to be important when they occur, people can remember details about such incidentally encoded information using episodic memory. Importantly, when information is explicitly encoded for use in an expected test of retention (as in most assessments in animals), it is possible that it is used to generate a planned action1,2,3; thus, the remembered action can occur without remembering the earlier episode. By contrast, when a test is unexpected, transforming information into an action plan is unlikely because the importance of the information and the nature of the test are not yet known. Thus, accurate performance in an unexpected test after incidental encoding documents episodic memory.1,2,3,4,5,6,7,8 Here, we present evidence that rats replay episodic memories of incidentally encoded information in an unexpected assessment of memory. In one task,9 rats reported the third-last item in an explicitly encoded list of trial-unique odors. In a second task,10 rats foraged in a radial maze in the absence of odors. On a critical test, rats foraged in the radial maze, but scented lids covered the food. Next, memory of the third-last odor was assessed. All participating rats correctly answered the unexpected question. These results suggest that rats encoded multiple pieces of putatively unimportant information, and later they replayed a stream of episodic memories when that information was needed to solve an unexpected problem. We propose that rats replay episodic memories of incidentally encoded information, which documents a critical aspect of human episodic memory in a non-human animal.
Collapse
Affiliation(s)
- Cassandra L Sheridan
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA
| | - Stephen Lang
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA
| | - Mya Knappenberger
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA
| | - Cami Albers
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA
| | - Ryleigh Loper
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA
| | - Baily Tillett
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA
| | - Jonah Sanchez
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA
| | - Alyssa Wilcox
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA
| | - Tess Harrison
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA
| | - Danielle Panoz-Brown
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA
| | - Jonathon D Crystal
- Department of Psychological and Brain Sciences, Indiana University, 1101 E 10(TH) ST, Bloomington, IN 47405-7007, USA.
| |
Collapse
|
9
|
Muhle-Karbe PS, Sheahan H, Pezzulo G, Spiers HJ, Chien S, Schuck NW, Summerfield C. Goal-seeking compresses neural codes for space in the human hippocampus and orbitofrontal cortex. Neuron 2023; 111:3885-3899.e6. [PMID: 37725981 DOI: 10.1016/j.neuron.2023.08.021] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Revised: 05/10/2023] [Accepted: 08/18/2023] [Indexed: 09/21/2023]
Abstract
Humans can navigate flexibly to meet their goals. Here, we asked how the neural representation of allocentric space is distorted by goal-directed behavior. Participants navigated an agent to two successive goal locations in a grid world environment comprising four interlinked rooms, with a contextual cue indicating the conditional dependence of one goal location on another. Examining the neural geometry by which room and context were encoded in fMRI signals, we found that map-like representations of the environment emerged in both hippocampus and neocortex. Cognitive maps in hippocampus and orbitofrontal cortices were compressed so that locations cued as goals were coded together in neural state space, and these distortions predicted successful learning. This effect was captured by a computational model in which current and prospective locations are jointly encoded in a place code, providing a theory of how goals warp the neural representation of space in macroscopic neural signals.
Collapse
Affiliation(s)
- Paul S Muhle-Karbe
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, UK; School of Psychology, University of Birmingham, Birmingham B15 2SA, UK; Centre for Human Brain Health, University of Birmingham, Birmingham B15 2SA, UK.
| | - Hannah Sheahan
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, UK; Google DeepMind, London EC4A 3TW, UK
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, 00185 Rome, Italy
| | - Hugo J Spiers
- Department of Experimental Psychology, University College London, London WC1E 6BT, UK
| | - Samson Chien
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, 14195 Berlin, Germany
| | - Nicolas W Schuck
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, 14195 Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Aging Research, 14195 Berlin, Germany; Institute of Psychology, Universität Hamburg, 20146 Hamburg, Germany
| | - Christopher Summerfield
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, UK; Centre for Human Brain Health, University of Birmingham, Birmingham B15 2SA, UK.
| |
Collapse
|
10
|
Csaky R, van Es MWJ, Jones OP, Woolrich M. Interpretable many-class decoding for MEG. Neuroimage 2023; 282:120396. [PMID: 37805019 PMCID: PMC10938061 DOI: 10.1016/j.neuroimage.2023.120396] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 09/11/2023] [Accepted: 09/27/2023] [Indexed: 10/09/2023] Open
Abstract
Multivariate pattern analysis (MVPA) of Magnetoencephalography (MEG) and Electroencephalography (EEG) data is a valuable tool for understanding how the brain represents and discriminates between different stimuli. Identifying the spatial and temporal signatures of stimuli is typically a crucial output of these analyses. Such analyses are mainly performed using linear, pairwise, sliding window decoding models. These allow for relative ease of interpretation, e.g. by estimating a time-course of decoding accuracy, but have limited decoding performance. On the other hand, full epoch multiclass decoding models, commonly used for brain-computer interface (BCI) applications, can provide better decoding performance. However interpretation methods for such models have been designed with a low number of classes in mind. In this paper, we propose an approach that combines a multiclass, full epoch decoding model with supervised dimensionality reduction, while still being able to reveal the contributions of spatiotemporal and spectral features using permutation feature importance. Crucially, we introduce a way of doing supervised dimensionality reduction of input features within a neural network optimised for the classification task, improving performance substantially. We demonstrate the approach on 3 different many-class task-MEG datasets using image presentations. Our results demonstrate that this approach consistently achieves higher accuracy than the peak accuracy of a sliding window decoder while estimating the relevant spatiotemporal features in the MEG signal.
Collapse
Affiliation(s)
- Richard Csaky
- Oxford Centre for Human Brain Activity, Department of Psychiatry, University of Oxford, OX3 7JX, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, OX3 9DU, Oxford, UK; Christ Church, OX1 1DP, Oxford, UK.
| | - Mats W J van Es
- Oxford Centre for Human Brain Activity, Department of Psychiatry, University of Oxford, OX3 7JX, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, OX3 9DU, Oxford, UK.
| | - Oiwi Parker Jones
- Wellcome Centre for Integrative Neuroimaging, OX3 9DU, Oxford, UK; Department of Engineering Science, University of Oxford, OX1 3PJ, Oxford, UK; Jesus College, OX1 3DW, Oxford, UK.
| | - Mark Woolrich
- Oxford Centre for Human Brain Activity, Department of Psychiatry, University of Oxford, OX3 7JX, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, OX3 9DU, Oxford, UK.
| |
Collapse
|
11
|
Finn ES, Poldrack RA, Shine JM. Functional neuroimaging as a catalyst for integrated neuroscience. Nature 2023; 623:263-273. [PMID: 37938706 DOI: 10.1038/s41586-023-06670-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 09/22/2023] [Indexed: 11/09/2023]
Abstract
Functional magnetic resonance imaging (fMRI) enables non-invasive access to the awake, behaving human brain. By tracking whole-brain signals across a diverse range of cognitive and behavioural states or mapping differences associated with specific traits or clinical conditions, fMRI has advanced our understanding of brain function and its links to both normal and atypical behaviour. Despite this headway, progress in human cognitive neuroscience that uses fMRI has been relatively isolated from rapid advances in other subdomains of neuroscience, which themselves are also somewhat siloed from one another. In this Perspective, we argue that fMRI is well-placed to integrate the diverse subfields of systems, cognitive, computational and clinical neuroscience. We first summarize the strengths and weaknesses of fMRI as an imaging tool, then highlight examples of studies that have successfully used fMRI in each subdomain of neuroscience. We then provide a roadmap for the future advances that will be needed to realize this integrative vision. In this way, we hope to demonstrate how fMRI can help usher in a new era of interdisciplinary coherence in neuroscience.
Collapse
Affiliation(s)
- Emily S Finn
- Department of Psychological and Brain Sciences, Dartmouth College, Dartmouth, NH, USA.
| | | | - James M Shine
- School of Medical Sciences, University of Sydney, Sydney, New South Wales, Australia.
| |
Collapse
|
12
|
Schwartenbeck P, Baram A, Liu Y, Mark S, Muller T, Dolan R, Botvinick M, Kurth-Nelson Z, Behrens T. Generative replay underlies compositional inference in the hippocampal-prefrontal circuit. Cell 2023; 186:4885-4897.e14. [PMID: 37804832 PMCID: PMC10914680 DOI: 10.1016/j.cell.2023.09.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 01/23/2023] [Accepted: 09/06/2023] [Indexed: 10/09/2023]
Abstract
Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.
Collapse
Affiliation(s)
- Philipp Schwartenbeck
- University of Tübingen, Tübingen, Germany; Max Planck Institute for Biological Cybernetics, Tübingen, Baden-Württemberg, Germany; Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK.
| | - Alon Baram
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Shirley Mark
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK
| | - Timothy Muller
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Institute of Neurology, University College London, London WC1N 3BG, UK
| | - Raymond Dolan
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Department of Psychiatry, Universitätsmedizin Berlin (Campus Charité Mitte), Berlin, Germany
| | - Matthew Botvinick
- Google DeepMind, London, UK; Gatsby Computational Neuroscience Unit, University College London, London, UK
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Google DeepMind, London, UK
| | - Timothy Behrens
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Sainsbury Wellcome Centre for Neural Circuits and Behaviour, UCL, London W1T 4JG, UK
| |
Collapse
|
13
|
Brændholt M, Kluger DS, Varga S, Heck DH, Gross J, Allen MG. Breathing in waves: Understanding respiratory-brain coupling as a gradient of predictive oscillations. Neurosci Biobehav Rev 2023; 152:105262. [PMID: 37271298 DOI: 10.1016/j.neubiorev.2023.105262] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2022] [Revised: 05/03/2023] [Accepted: 05/24/2023] [Indexed: 06/06/2023]
Abstract
Breathing plays a crucial role in shaping perceptual and cognitive processes by regulating the strength and synchronisation of neural oscillations. Numerous studies have demonstrated that respiratory rhythms govern a wide range of behavioural effects across cognitive, affective, and perceptual domains. Additionally, respiratory-modulated brain oscillations have been observed in various mammalian models and across diverse frequency spectra. However, a comprehensive framework to elucidate these disparate phenomena remains elusive. In this review, we synthesise existing findings to propose a neural gradient of respiratory-modulated brain oscillations and examine recent computational models of neural oscillations to map this gradient onto a hierarchical cascade of precision-weighted prediction errors. By deciphering the computational mechanisms underlying respiratory control of these processes, we can potentially uncover new pathways for understanding the link between respiratory-brain coupling and psychiatric disorders.
Collapse
Affiliation(s)
- Malthe Brændholt
- Center of Functionally Integrative Neuroscience, Aarhus University, Denmark
| | - Daniel S Kluger
- Institute for Biomagnetism and Biosignal Analysis, University of Münster, Germany.
| | - Somogy Varga
- School of Culture and Society, Aarhus University, Denmark; The Centre for Philosophy of Epidemiology, Medicine and Public Health, University of Johannesburg, South Africa
| | - Detlef H Heck
- Department of Biomedical Sciences, University of Minnesota Medical School, Duluth, MN
| | - Joachim Gross
- Institute for Biomagnetism and Biosignal Analysis, University of Münster, Germany; Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Germany
| | - Micah G Allen
- Center of Functionally Integrative Neuroscience, Aarhus University, Denmark; Cambridge Psychiatry, University of Cambridge, UK
| |
Collapse
|
14
|
Tanrıverdi B, Cowan ET, Metoki A, Jobson KR, Murty VP, Chein J, Olson IR. Awake Hippocampal-Cortical Co-reactivation Is Associated with Forgetting. J Cogn Neurosci 2023; 35:1446-1462. [PMID: 37348130 PMCID: PMC10759317 DOI: 10.1162/jocn_a_02021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/24/2023]
Abstract
Systems consolidation theories posit that consolidation occurs primarily through a coordinated communication between hippocampus and neocortex [Moscovitch, M., & Gilboa, A. Systems consolidation, transformation and reorganization: Multiple trace theory, trace transformation theory and their competitors. PsyArXiv, 2021; Kumaran, D., Hassabis, D., & McClelland, J. L. What learning systems do intelligent agents need? Complementary learning systems theory updated. Trends in Cognitive Sciences, 20, 512-534, 2016; McClelland, J. L., & O'Reilly, R. C. Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102, 419-457, 1995]. Recent sleep studies in rodents have shown that hippocampus and visual cortex replay the same information at temporal proximity ("co-replay"; Lansink, C. S., Goltstein, P. M., Lankelma, J. V., McNaughton, B. L., & Pennartz, C. M. A. Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biology, 7, e1000173, 2009; Peyrache, A., Khamassi, M., Benchenane, K., Wiener, S. I., & Battaglia, F. P. Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nature Neuroscience, 12, 919-926, 2009; Wierzynski, C. M., Lubenov, E. V., Gu, M., & Siapas, A. G. State-dependent spike-timing relationships between hippocampal and prefrontal circuits during sleep. Neuron, 61, 587-596, 2009; Ji, D., & Wilson, M. A. Coordinated memory replay in the visual cortex and hippocampus during sleep. Nature Neuroscience, 10, 100-107, 2007). We developed a novel repetition time (TR)-based co-reactivation analysis method to study hippocampal-cortical co-replays in humans using fMRI. Thirty-six young adults completed an image (face or scene) and location paired associate encoding task in the scanner, which were preceded and followed by resting state scans. We identified post-encoding rest TRs (± 1) that showed neural reactivation of each image-location trials in both hippocampus (HPC) and category-selective cortex (fusiform face area [FFA]). This allowed us to characterize temporally proximal coordinated reactivations ("co-reactivations") between HPC and FFA. Moreover, we found that increased HPC-FFA co-reactivations were associated with incorrectly recognized trials after a 1-week delay (p = .004). Finally, we found that these HPC-FFA co-reactivations were also associated with trials that were initially correctly recognized immediately after encoding but were later forgotten in 1-day (p = .043) and 1-week delay period (p = .031). We discuss these results from a trace transformation perspective [Sekeres, M. J., Winocur, G., & Moscovitch, M. The hippocampus and related neocortical structures in memory transformation. Neuroscience Letters, 680, 39-53, 2018; Winocur, G., & Moscovitch, M. Memory transformation and systems consolidation. Journal of the International Neuropsychological Society, 17, 766-780, 2011] and speculate that HPC-FFA co-reactivations may be integrating related events, at the expense of disrupting event-specific details, hence leading to forgetting.
Collapse
|
15
|
Wise T, Charpentier CJ, Dayan P, Mobbs D. Interactive cognitive maps support flexible behavior under threat. Cell Rep 2023; 42:113008. [PMID: 37610871 PMCID: PMC10658881 DOI: 10.1016/j.celrep.2023.113008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 07/11/2023] [Accepted: 08/03/2023] [Indexed: 08/25/2023] Open
Abstract
In social environments, survival can depend upon inferring and adapting to other agents' goal-directed behavior. However, it remains unclear how humans achieve this, despite the fact that many decisions must account for complex, dynamic agents acting according to their own goals. Here, we use a predator-prey task (total n = 510) to demonstrate that humans exploit an interactive cognitive map of the social environment to infer other agents' preferences and simulate their future behavior, providing for flexible, generalizable responses. A model-based inverse reinforcement learning model explained participants' inferences about threatening agents' preferences, with participants using this inferred knowledge to enact generalizable, model-based behavioral responses. Using tree-search planning models, we then found that behavior was best explained by a planning algorithm that incorporated simulations of the threat's goal-directed behavior. Our results indicate that humans use a cognitive map to determine other agents' preferences, facilitating generalized predictions of their behavior and effective responses.
Collapse
Affiliation(s)
- Toby Wise
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK; Department of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA.
| | - Caroline J Charpentier
- Department of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA; Department of Psychology, University of Maryland, College Park, MD, USA; Brain and Behavior Institute, University of Maryland, College Park, MD, USA
| | - Peter Dayan
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany; University of Tübingen, Tübingen, Germany
| | - Dean Mobbs
- Department of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA; Computation and Neural Systems Program, California Institute of Technology, Pasadena, CA, USA
| |
Collapse
|
16
|
Nelli S, Braun L, Dumbalska T, Saxe A, Summerfield C. Neural knowledge assembly in humans and neural networks. Neuron 2023; 111:1504-1516.e9. [PMID: 36898375 PMCID: PMC10618408 DOI: 10.1016/j.neuron.2023.02.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Revised: 12/21/2022] [Accepted: 02/09/2023] [Indexed: 03/11/2023]
Abstract
Human understanding of the world can change rapidly when new information comes to light, such as when a plot twist occurs in a work of fiction. This flexible "knowledge assembly" requires few-shot reorganization of neural codes for relations among objects and events. However, existing computational theories are largely silent about how this could occur. Here, participants learned a transitive ordering among novel objects within two distinct contexts before exposure to new knowledge that revealed how they were linked. Blood-oxygen-level-dependent (BOLD) signals in dorsal frontoparietal cortical areas revealed that objects were rapidly and dramatically rearranged on the neural manifold after minimal exposure to linking information. We then adapt online stochastic gradient descent to permit similar rapid knowledge assembly in a neural network model.
Collapse
Affiliation(s)
- Stephanie Nelli
- Department of Cognitive Science, Occidental College, Los Angeles, CA 90041, USA; Department of Experimental Psychology, University of Oxford, Oxford OX2 6GC, UK.
| | - Lukas Braun
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GC, UK
| | | | - Andrew Saxe
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GC, UK; Gatsby Unit & Sainsbury Wellcome Centre, University College London, London W1T 4JG, UK; CIFAR Azrieli Global Scholars Program, CIFAR, Toronto, ON M5G 1M1, Canada
| | | |
Collapse
|
17
|
Wu X, Packard PA, García-Arch J, Bunzeck N, Fuentemilla L. Contextual incongruency triggers memory reinstatement and the disruption of neural stability. Neuroimage 2023; 273:120114. [PMID: 37080120 DOI: 10.1016/j.neuroimage.2023.120114] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Revised: 03/13/2023] [Accepted: 04/13/2023] [Indexed: 04/22/2023] Open
Abstract
Schemas, or internal representation models of the environment, are thought to be central in organising our everyday life behaviour by giving stability and predictiveness to the structure of the world. However, when an element from an unfolding event mismatches the schema-derived expectations, the coherent narrative is interrupted and an update to the current event model representation is required. Here, we asked whether the perceived incongruence of an item from an unfolding event and its impact on memory relied on the disruption of neural stability patterns preceded by the neural reactivation of the memory representations of the just-encoded event. Our study includes data from two different experiments whereby human participants (N = 33, 26 females and N = 18, 16 females, respectively) encoded images of objects preceded by trial-unique sequences of events depicting daily routine. We found that neural stability patterns gradually increased throughout the ongoing exposure to a schema-consistent episode, which was corroborated by the re-analysis of data from two other experiments, and that the brain stability pattern was interrupted when the encoding of an object of the event was incongruent with the ongoing schema. We found that the decrease in neural stability for low-congruence items was seen at ∼1000 ms from object encoding onset and that it was preceded by an enhanced N400 ERP and an increased degree of neural reactivation of the just-encoded episode. Current results offer new insights into the neural mechanisms and their temporal orchestration that are engaged during online encoding of schema-consistent episodic narratives and the detection of incongruencies.
Collapse
Affiliation(s)
- Xiongbo Wu
- Cognition and Brain Plasticity Group, Bellvitge Institute for Biomedical Research, Hospitalet de Llobregat 08907, Spain; Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona 08035, Spain; Institute of Neurosciences, University of Barcelona, Barcelona 08035, Spain.
| | - Pau A Packard
- Multisensory Research Group, Center for Brain and Cognition, Pompeu Fabra University, Barcelona, Spain
| | - Josué García-Arch
- Cognition and Brain Plasticity Group, Bellvitge Institute for Biomedical Research, Hospitalet de Llobregat 08907, Spain; Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona 08035, Spain; Institute of Neurosciences, University of Barcelona, Barcelona 08035, Spain
| | - Nico Bunzeck
- Department of Psychology, University of Lübeck, Lübeck 23562, Germany; Center of Brain, Behavior and Metabolism (CBBM), University of Lübeck, Lübeck 23562, Germany
| | - Lluís Fuentemilla
- Cognition and Brain Plasticity Group, Bellvitge Institute for Biomedical Research, Hospitalet de Llobregat 08907, Spain; Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona 08035, Spain; Institute of Neurosciences, University of Barcelona, Barcelona 08035, Spain
| |
Collapse
|
18
|
McFadyen J, Dolan RJ. Spatiotemporal Precision of Neuroimaging in Psychiatry. Biol Psychiatry 2023; 93:671-680. [PMID: 36376110 DOI: 10.1016/j.biopsych.2022.08.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Revised: 07/20/2022] [Accepted: 08/12/2022] [Indexed: 12/23/2022]
Abstract
Aberrant patterns of cognition, perception, and behavior seen in psychiatric disorders are thought to be driven by a complex interplay of neural processes that evolve at a rapid temporal scale. Understanding these dynamic processes in vivo in humans has been hampered by a trade-off between spatial and temporal resolutions inherent to current neuroimaging technology. A recent trend in psychiatric research has been the use of high temporal resolution imaging, particularly magnetoencephalography, often in conjunction with sophisticated machine learning decoding techniques. Developments here promise novel insights into the spatiotemporal dynamics of cognitive phenomena, including domains relevant to psychiatric illnesses such as reward and avoidance learning, memory, and planning. This review considers recent advances afforded by exploiting this increased spatiotemporal precision, with specific reference to applications that seek to drive a mechanistic understanding of psychopathology and the realization of preclinical translation.
Collapse
Affiliation(s)
- Jessica McFadyen
- UCL Max Planck Centre for Computational Psychiatry and Ageing Research and Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.
| | - Raymond J Dolan
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| |
Collapse
|
19
|
McFadyen J, Liu Y, Dolan RJ. Differential replay of reward and punishment paths predicts approach and avoidance. Nat Neurosci 2023; 26:627-637. [PMID: 37020116 DOI: 10.1038/s41593-023-01287-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 02/16/2023] [Indexed: 04/07/2023]
Abstract
Neural replay is implicated in planning, where states relevant to a task goal are rapidly reactivated in sequence. It remains unclear whether, during planning, replay relates to an actual prospective choice. Here, using magnetoencephalography (MEG), we studied replay in human participants while they planned to either approach or avoid an uncertain environment containing paths leading to reward or punishment. We find evidence for forward sequential replay during planning, with rapid state-to-state transitions from 20 to 90 ms. Replay of rewarding paths was boosted, relative to aversive paths, before a decision to avoid and attenuated before a decision to approach. A trial-by-trial bias toward replaying prospective punishing paths predicted irrational decisions to approach riskier environments, an effect more pronounced in participants with higher trait anxiety. The findings indicate a coupling of replay with planned behavior, where replay prioritizes an online representation of a worst-case scenario for approaching or avoiding.
Collapse
Affiliation(s)
- Jessica McFadyen
- The UCL Max Planck Centre for Computational Psychiatry and Ageing Research, University College London, London, UK.
- Wellcome Centre for Human Neuroimaging, University College London, London, UK.
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Chinese Institute for Brain Research, Beijing, China
| | - Raymond J Dolan
- The UCL Max Planck Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| |
Collapse
|
20
|
Bono J, Zannone S, Pedrosa V, Clopath C. Learning predictive cognitive maps with spiking neurons during behavior and replays. eLife 2023; 12:e80671. [PMID: 36927625 PMCID: PMC10019888 DOI: 10.7554/elife.80671] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 01/12/2023] [Indexed: 03/18/2023] Open
Abstract
The hippocampus has been proposed to encode environments using a representation that contains predictive information about likely future states, called the successor representation. However, it is not clear how such a representation could be learned in the hippocampal circuit. Here, we propose a plasticity rule that can learn this predictive map of the environment using a spiking neural network. We connect this biologically plausible plasticity rule to reinforcement learning, mathematically and numerically showing that it implements the TD-lambda algorithm. By spanning these different levels, we show how our framework naturally encompasses behavioral activity and replays, smoothly moving from rate to temporal coding, and allows learning over behavioral timescales with a plasticity rule acting on a timescale of milliseconds. We discuss how biological parameters such as dwelling times at states, neuronal firing rates and neuromodulation relate to the delay discounting parameter of the TD algorithm, and how they influence the learned representation. We also find that, in agreement with psychological studies and contrary to reinforcement learning theory, the discount factor decreases hyperbolically with time. Finally, our framework suggests a role for replays, in both aiding learning in novel environments and finding shortcut trajectories that were not experienced during behavior, in agreement with experimental data.
Collapse
Affiliation(s)
- Jacopo Bono
- Department of Bioengineering, Imperial College LondonLondonUnited Kingdom
| | - Sara Zannone
- Department of Bioengineering, Imperial College LondonLondonUnited Kingdom
| | - Victor Pedrosa
- Department of Bioengineering, Imperial College LondonLondonUnited Kingdom
| | - Claudia Clopath
- Department of Bioengineering, Imperial College LondonLondonUnited Kingdom
| |
Collapse
|
21
|
Chen ZS, Wilson MA. How our understanding of memory replay evolves. J Neurophysiol 2023; 129:552-580. [PMID: 36752404 PMCID: PMC9988534 DOI: 10.1152/jn.00454.2022] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 01/20/2023] [Accepted: 01/20/2023] [Indexed: 02/09/2023] Open
Abstract
Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.
Collapse
Affiliation(s)
- Zhe Sage Chen
- Department of Psychiatry, New York University Grossman School of Medicine, New York, New York, United States
- Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, New York, United States
- Neuroscience Institute, New York University Grossman School of Medicine, New York, New York, United States
- Department of Biomedical Engineering, New York University Tandon School of Engineering, Brooklyn, New York, United States
| | - Matthew A Wilson
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
| |
Collapse
|
22
|
Kurth-Nelson Z, Behrens T, Wayne G, Miller K, Luettgau L, Dolan R, Liu Y, Schwartenbeck P. Replay and compositional computation. Neuron 2023; 111:454-469. [PMID: 36640765 DOI: 10.1016/j.neuron.2022.12.028] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/11/2022] [Accepted: 12/18/2022] [Indexed: 01/15/2023]
Abstract
Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form of compositional computation where entities are assembled into relationally bound structures to derive qualitatively new knowledge. This idea builds on recent advances in neuroscience, which indicate that the hippocampus flexibly binds objects to generalizable roles and that replay strings these role-bound objects into compound statements. We suggest experiments to test our hypothesis, and we end by noting the implications for AI systems which lack the human ability to radically generalize past experience to solve new problems.
Collapse
Affiliation(s)
- Zeb Kurth-Nelson
- DeepMind, London, UK; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK.
| | - Timothy Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | | | - Kevin Miller
- DeepMind, London, UK; Institute of Ophthalmology, University College London, London, UK
| | - Lennart Luettgau
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Ray Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK; Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Philipp Schwartenbeck
- Max Planck Institute for Biological Cybernetics, Tubingen, Germany; University of Tubingen, Tubingen, Germany
| |
Collapse
|
23
|
Wimmer GE, Liu Y, McNamee DC, Dolan RJ. Distinct replay signatures for prospective decision-making and memory preservation. Proc Natl Acad Sci U S A 2023; 120:e2205211120. [PMID: 36719914 PMCID: PMC9963918 DOI: 10.1073/pnas.2205211120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 12/05/2022] [Indexed: 02/01/2023] Open
Abstract
Theories of neural replay propose that it supports a range of functions, most prominently planning and memory consolidation. Here, we test the hypothesis that distinct signatures of replay in the same task are related to model-based decision-making ("planning") and memory preservation. We designed a reward learning task wherein participants utilized structure knowledge for model-based evaluation, while at the same time had to maintain knowledge of two independent and randomly alternating task environments. Using magnetoencephalography and multivariate analysis, we first identified temporally compressed sequential reactivation, or replay, both prior to choice and following reward feedback. Before choice, prospective replay strength was enhanced for the current task-relevant environment when a model-based planning strategy was beneficial. Following reward receipt, and consistent with a memory preservation role, replay for the alternative distal task environment was enhanced as a function of decreasing recency of experience with that environment. Critically, these planning and memory preservation relationships were selective to pre-choice and post-feedback periods, respectively. Our results provide support for key theoretical proposals regarding the functional role of replay and demonstrate that the relative strength of planning and memory-related signals are modulated by ongoing computational and task demands.
Collapse
Affiliation(s)
- G. Elliott Wimmer
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
- Chinese Institute for Brain Research, Beijing100875, China
| | - Daniel C. McNamee
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
- Neuroscience Programme, Champalimaud Research, Lisbon1400-038, Portugal
| | - Raymond J. Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| |
Collapse
|
24
|
Ekman M, Kusch S, de Lange FP. Successor-like representation guides the prediction of future events in human visual cortex and hippocampus. eLife 2023; 12:78904. [PMID: 36729024 PMCID: PMC9894584 DOI: 10.7554/elife.78904] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2022] [Accepted: 01/13/2023] [Indexed: 02/03/2023] Open
Abstract
Human agents build models of their environment, which enable them to anticipate and plan upcoming events. However, little is known about the properties of such predictive models. Recently, it has been proposed that hippocampal representations take the form of a predictive map-like structure, the so-called successor representation (SR). Here, we used human functional magnetic resonance imaging to probe whether activity in the early visual cortex (V1) and hippocampus adhere to the postulated properties of the SR after visual sequence learning. Participants were exposed to an arbitrary spatiotemporal sequence consisting of four items (A-B-C-D). We found that after repeated exposure to the sequence, merely presenting single sequence items (e.g., - B - -) resulted in V1 activation at the successor locations of the full sequence (e.g., C-D), but not at the predecessor locations (e.g., A). This highlights that visual representations are skewed toward future states, in line with the SR. Similar results were also found in the hippocampus. Moreover, the hippocampus developed a coactivation profile that showed sensitivity to the temporal distance in sequence space, with fading representations for sequence events in the more distant past and future. V1, in contrast, showed a coactivation profile that was only sensitive to spatial distance in stimulus space. Taken together, these results provide empirical evidence for the proposition that both visual and hippocampal cortex represent a predictive map of the visual world akin to the SR.
Collapse
Affiliation(s)
- Matthias Ekman
- Radboud University Nijmegen, Donders Institute for Brain, Cognition and BehaviourNijmegenNetherlands
| | - Sarah Kusch
- Radboud University Nijmegen, Donders Institute for Brain, Cognition and BehaviourNijmegenNetherlands
| | - Floris P de Lange
- Radboud University Nijmegen, Donders Institute for Brain, Cognition and BehaviourNijmegenNetherlands
| |
Collapse
|
25
|
Palenciano AF, Senoussi M, Formica S, González-García C. Canonical template tracking: Measuring the activation state of specific neural representations. FRONTIERS IN NEUROIMAGING 2023; 1:974927. [PMID: 37555182 PMCID: PMC10406196 DOI: 10.3389/fnimg.2022.974927] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 12/13/2022] [Indexed: 08/10/2023]
Abstract
Multivariate analyses of neural data have become increasingly influential in cognitive neuroscience since they allow to address questions about the representational signatures of neurocognitive phenomena. Here, we describe Canonical Template Tracking: a multivariate approach that employs independent localizer tasks to assess the activation state of specific representations during the execution of cognitive paradigms. We illustrate the benefits of this methodology in characterizing the particular content and format of task-induced representations, comparing it with standard (cross-)decoding and representational similarity analyses. Then, we discuss relevant design decisions for experiments using this analysis approach, focusing on the nature of the localizer tasks from which the canonical templates are derived. We further provide a step-by-step tutorial of this method, stressing the relevant analysis choices for functional magnetic resonance imaging and magneto/electroencephalography data. Importantly, we point out the potential pitfalls linked to canonical template tracking implementation and interpretation of the results, together with recommendations to mitigate them. To conclude, we provide some examples from previous literature that highlight the potential of this analysis to address relevant theoretical questions in cognitive neuroscience.
Collapse
Affiliation(s)
- Ana F. Palenciano
- Mind, Brain, and Behavior Research Center, University of Granada, Granada, Spain
| | - Mehdi Senoussi
- CLLE Lab, CNRS UMR 5263, University of Toulouse, Toulouse, France
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Silvia Formica
- Department of Psychology, Berlin School of Mind and Brain, Humboldt Universität zu Berlin, Berlin, Germany
| | | |
Collapse
|
26
|
Spontaneous mind wandering impairs model-based decision making. PLoS One 2023; 18:e0279532. [PMID: 36701316 PMCID: PMC9879536 DOI: 10.1371/journal.pone.0279532] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Accepted: 12/08/2022] [Indexed: 01/27/2023] Open
Abstract
BACKGROUND If our attention wanders to other thoughts while making a decision, then the decision might not be directed towards future goals, reflecting a lack of model-based decision making, but may instead be driven by habits, reflecting model-free decision making. Here we aimed to investigate if and how model-based versus model-free decision making is reduced by trait spontaneous mind wandering. METHODS AND FINDINGS We used a sequential two-step Markov decision task and a self-report questionnaire assessing trait spontaneous and deliberate mind wandering propensity, to investigate how trait mind wandering relates to model-free as well as model-based decisions. We estimated parameters of a computational neurocognitive dual-control model of decision making. Analyzing estimated model parameters, we found that trait spontaneous mind wandering was related to impaired model-based decisions, while model-free choice stayed unaffected. CONCLUSIONS Our findings suggest trait spontaneous mind wandering is associated with impaired model-based decision making, and it may reflect model-based offline replay for other tasks (e.g., real-life goals) outside the current lab situation.
Collapse
|
27
|
Pietras B, Schmutz V, Schwalger T. Mesoscopic description of hippocampal replay and metastability in spiking neural networks with short-term plasticity. PLoS Comput Biol 2022; 18:e1010809. [PMID: 36548392 PMCID: PMC9822116 DOI: 10.1371/journal.pcbi.1010809] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 01/06/2023] [Accepted: 12/11/2022] [Indexed: 12/24/2022] Open
Abstract
Bottom-up models of functionally relevant patterns of neural activity provide an explicit link between neuronal dynamics and computation. A prime example of functional activity patterns are propagating bursts of place-cell activities called hippocampal replay, which is critical for memory consolidation. The sudden and repeated occurrences of these burst states during ongoing neural activity suggest metastable neural circuit dynamics. As metastability has been attributed to noise and/or slow fatigue mechanisms, we propose a concise mesoscopic model which accounts for both. Crucially, our model is bottom-up: it is analytically derived from the dynamics of finite-size networks of Linear-Nonlinear Poisson neurons with short-term synaptic depression. As such, noise is explicitly linked to stochastic spiking and network size, and fatigue is explicitly linked to synaptic dynamics. To derive the mesoscopic model, we first consider a homogeneous spiking neural network and follow the temporal coarse-graining approach of Gillespie to obtain a "chemical Langevin equation", which can be naturally interpreted as a stochastic neural mass model. The Langevin equation is computationally inexpensive to simulate and enables a thorough study of metastable dynamics in classical setups (population spikes and Up-Down-states dynamics) by means of phase-plane analysis. An extension of the Langevin equation for small network sizes is also presented. The stochastic neural mass model constitutes the basic component of our mesoscopic model for replay. We show that the mesoscopic model faithfully captures the statistical structure of individual replayed trajectories in microscopic simulations and in previously reported experimental data. Moreover, compared to the deterministic Romani-Tsodyks model of place-cell dynamics, it exhibits a higher level of variability regarding order, direction and timing of replayed trajectories, which seems biologically more plausible and could be functionally desirable. This variability is the product of a new dynamical regime where metastability emerges from a complex interplay between finite-size fluctuations and local fatigue.
Collapse
Affiliation(s)
- Bastian Pietras
- Institute for Mathematics, Technische Universität Berlin, Berlin, Germany
- Bernstein Center for Computational Neuroscience, Berlin, Germany
- Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain
| | - Valentin Schmutz
- Brain Mind Institute, School of Computer and Communication Sciences and School of Life Sciences, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Tilo Schwalger
- Institute for Mathematics, Technische Universität Berlin, Berlin, Germany
- Bernstein Center for Computational Neuroscience, Berlin, Germany
- * E-mail:
| |
Collapse
|
28
|
Comrie AE, Frank LM, Kay K. Imagination as a fundamental function of the hippocampus. Philos Trans R Soc Lond B Biol Sci 2022; 377:20210336. [PMID: 36314152 PMCID: PMC9620759 DOI: 10.1098/rstb.2021.0336] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Accepted: 04/20/2022] [Indexed: 08/25/2023] Open
Abstract
Imagination is a biological function that is vital to human experience and advanced cognition. Despite this importance, it remains unknown how imagination is realized in the brain. Substantial research focusing on the hippocampus, a brain structure traditionally linked to memory, indicates that firing patterns in spatially tuned neurons can represent previous and upcoming paths in space. This work has generally been interpreted under standard views that the hippocampus implements cognitive abilities primarily related to actual experience, whether in the past (e.g. recollection, consolidation), present (e.g. spatial mapping) or future (e.g. planning). However, relatively recent findings in rodents identify robust patterns of hippocampal firing corresponding to a variety of alternatives to actual experience, in many cases without overt reference to the past, present or future. Given these findings, and others on hippocampal contributions to human imagination, we suggest that a fundamental function of the hippocampus is to generate a wealth of hypothetical experiences and thoughts. Under this view, traditional accounts of hippocampal function in episodic memory and spatial navigation can be understood as particular applications of a more general system for imagination. This view also suggests that the hippocampus contributes to a wider range of cognitive abilities than previously thought. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.
Collapse
Affiliation(s)
- Alison E. Comrie
- Neuroscience Graduate Program, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Kavli Institute for Fundamental Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Center for Integrative Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Departments of Physiology and Psychiatry, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
| | - Loren M. Frank
- Kavli Institute for Fundamental Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Center for Integrative Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Departments of Physiology and Psychiatry, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Howard Hughes Medical Institute, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
| | - Kenneth Kay
- Zuckerman Institute, Center for Theoretical Neuroscience, Columbia University, 3227 Broadway, New York, NY 10027, USA
| |
Collapse
|
29
|
Colas JT, Dundon NM, Gerraty RT, Saragosa‐Harris NM, Szymula KP, Tanwisuth K, Tyszka JM, van Geen C, Ju H, Toga AW, Gold JI, Bassett DS, Hartley CA, Shohamy D, Grafton ST, O'Doherty JP. Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T. Hum Brain Mapp 2022; 43:4750-4790. [PMID: 35860954 PMCID: PMC9491297 DOI: 10.1002/hbm.25988] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Revised: 05/20/2022] [Accepted: 06/10/2022] [Indexed: 11/12/2022] Open
Abstract
The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.
Collapse
Affiliation(s)
- Jaron T. Colas
- Department of Psychological and Brain SciencesUniversity of CaliforniaSanta BarbaraCaliforniaUSA
- Division of the Humanities and Social SciencesCalifornia Institute of TechnologyPasadenaCaliforniaUSA
- Computation and Neural Systems Program, California Institute of TechnologyPasadenaCaliforniaUSA
| | - Neil M. Dundon
- Department of Psychological and Brain SciencesUniversity of CaliforniaSanta BarbaraCaliforniaUSA
- Department of Child and Adolescent Psychiatry, Psychotherapy, and PsychosomaticsUniversity of FreiburgFreiburg im BreisgauGermany
| | - Raphael T. Gerraty
- Department of PsychologyColumbia UniversityNew YorkNew YorkUSA
- Zuckerman Mind Brain Behavior Institute, Columbia UniversityNew YorkNew YorkUSA
- Center for Science and SocietyColumbia UniversityNew YorkNew YorkUSA
| | - Natalie M. Saragosa‐Harris
- Department of PsychologyNew York UniversityNew YorkNew YorkUSA
- Department of PsychologyUniversity of CaliforniaLos AngelesCaliforniaUSA
| | - Karol P. Szymula
- Department of BioengineeringUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Koranis Tanwisuth
- Division of the Humanities and Social SciencesCalifornia Institute of TechnologyPasadenaCaliforniaUSA
- Department of PsychologyUniversity of CaliforniaBerkeleyCaliforniaUSA
| | - J. Michael Tyszka
- Division of the Humanities and Social SciencesCalifornia Institute of TechnologyPasadenaCaliforniaUSA
| | - Camilla van Geen
- Zuckerman Mind Brain Behavior Institute, Columbia UniversityNew YorkNew YorkUSA
- Department of PsychologyUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Harang Ju
- Neuroscience Graduate GroupUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Arthur W. Toga
- Laboratory of Neuro ImagingUSC Stevens Neuroimaging and Informatics Institute, Keck School of Medicine of USC, University of Southern CaliforniaLos AngelesCaliforniaUSA
| | - Joshua I. Gold
- Department of NeuroscienceUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
| | - Dani S. Bassett
- Department of BioengineeringUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Department of Electrical and Systems EngineeringUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Department of NeurologyUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Department of PsychiatryUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Department of Physics and AstronomyUniversity of PennsylvaniaPhiladelphiaPennsylvaniaUSA
- Santa Fe InstituteSanta FeNew MexicoUSA
| | - Catherine A. Hartley
- Department of PsychologyNew York UniversityNew YorkNew YorkUSA
- Center for Neural ScienceNew York UniversityNew YorkNew YorkUSA
| | - Daphna Shohamy
- Department of PsychologyColumbia UniversityNew YorkNew YorkUSA
- Zuckerman Mind Brain Behavior Institute, Columbia UniversityNew YorkNew YorkUSA
- Kavli Institute for Brain ScienceColumbia UniversityNew YorkNew YorkUSA
| | - Scott T. Grafton
- Department of Psychological and Brain SciencesUniversity of CaliforniaSanta BarbaraCaliforniaUSA
| | - John P. O'Doherty
- Division of the Humanities and Social SciencesCalifornia Institute of TechnologyPasadenaCaliforniaUSA
- Computation and Neural Systems Program, California Institute of TechnologyPasadenaCaliforniaUSA
| |
Collapse
|
30
|
Folvik L, Sneve MH, Ness HT, Vidal-Piñeiro D, Raud L, Geier OM, Walhovd KB, Fjell AM. Sustained upregulation of widespread hippocampal-neocortical coupling following memory encoding. Cereb Cortex 2022; 33:4844-4858. [PMID: 36190442 PMCID: PMC10110434 DOI: 10.1093/cercor/bhac384] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Revised: 08/29/2022] [Accepted: 08/30/2022] [Indexed: 11/14/2022] Open
Abstract
Systems consolidation of new experiences into lasting episodic memories involves hippocampal-neocortical interactions. Evidence of this process is already observed during early post-encoding rest periods, both as increased hippocampal coupling with task-relevant perceptual regions and reactivation of stimulus-specific patterns following intensive encoding tasks. We investigate the spatial and temporal characteristics of these hippocampally anchored post-encoding neocortical modulations. Eighty-nine adults participated in an experiment consisting of interleaved memory task- and resting-state periods. We observed increased post-encoding functional connectivity between hippocampus and individually localized neocortical regions responsive to stimuli encountered during memory encoding. Post-encoding modulations were manifested as a nearly system-wide upregulation in hippocampal coupling with all major functional networks. The configuration of these extensive modulations resembled hippocampal-neocortical interaction patterns estimated from active encoding operations, suggesting hippocampal post-encoding involvement exceeds perceptual aspects. Reinstatement of encoding patterns was not observed in resting-state scans collected 12 h later, nor when using other candidate seed regions. The similarity in hippocampal functional coupling between online memory encoding and offline post-encoding rest suggests reactivation in humans involves a spectrum of cognitive processes engaged during the experience of an event. There were no age effects, suggesting that upregulation of hippocampal-neocortical connectivity represents a general phenomenon seen across the adult lifespan.
Collapse
Affiliation(s)
- Line Folvik
- Department of Psychology, Center for Lifespan Changes in Brain and Cognition, University of Oslo, Forskningsveien 3A, 0373 Oslo, Norway
| | - Markus H Sneve
- Department of Psychology, Center for Lifespan Changes in Brain and Cognition, University of Oslo, Forskningsveien 3A, 0373 Oslo, Norway
| | - Hedda T Ness
- Department of Psychology, Center for Lifespan Changes in Brain and Cognition, University of Oslo, Forskningsveien 3A, 0373 Oslo, Norway
| | - Didac Vidal-Piñeiro
- Department of Psychology, Center for Lifespan Changes in Brain and Cognition, University of Oslo, Forskningsveien 3A, 0373 Oslo, Norway
| | - Liisa Raud
- Department of Psychology, Center for Lifespan Changes in Brain and Cognition, University of Oslo, Forskningsveien 3A, 0373 Oslo, Norway
| | - Oliver M Geier
- Department of Diagnostic Physics, Oslo University Hospital, Postbox 4950 Nydalen, OUS, Rikshospitalet, 0424 Oslo, Norway
| | - Kristine B Walhovd
- Department of Psychology, Center for Lifespan Changes in Brain and Cognition, University of Oslo, Forskningsveien 3A, 0373 Oslo, Norway.,Division of Radiology and Nuclear Medicine, Oslo University Hospital, Postbox 4950 Nydalen, OUS, Rikshospitalet, 0424 Oslo, Norway
| | - Anders M Fjell
- Department of Psychology, Center for Lifespan Changes in Brain and Cognition, University of Oslo, Forskningsveien 3A, 0373 Oslo, Norway.,Division of Radiology and Nuclear Medicine, Oslo University Hospital, Postbox 4950 Nydalen, OUS, Rikshospitalet, 0424 Oslo, Norway
| |
Collapse
|
31
|
Crystal JD. Memory: Dolphins remember incidental events. Curr Biol 2022; 32:R929-R931. [PMID: 36099900 DOI: 10.1016/j.cub.2022.08.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Abstract
A fundamental problem in the evolution of cognition is the search for complex memory systems given the longstanding belief that complex cognition is unique to humans. Along these lines, new research suggests that bottlenose dolphins can answer unexpected questions after encoding information that was seemingly unimportant when it was encountered.
Collapse
Affiliation(s)
- Jonathon D Crystal
- Department of Psychological & Brain Sciences, Indiana University, 1101 E 10th Street. Bloomington, IN 47405-7007, USA.
| |
Collapse
|
32
|
Loriette C, Amengual JL, Ben Hamed S. Beyond the brain-computer interface: Decoding brain activity as a tool to understand neuronal mechanisms subtending cognition and behavior. Front Neurosci 2022; 16:811736. [PMID: 36161174 PMCID: PMC9492914 DOI: 10.3389/fnins.2022.811736] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Accepted: 08/23/2022] [Indexed: 11/13/2022] Open
Abstract
One of the major challenges in system neurosciences consists in developing techniques for estimating the cognitive information content in brain activity. This has an enormous potential in different domains spanning from clinical applications, cognitive enhancement to a better understanding of the neural bases of cognition. In this context, the inclusion of machine learning techniques to decode different aspects of human cognition and behavior and its use to develop brain–computer interfaces for applications in neuroprosthetics has supported a genuine revolution in the field. However, while these approaches have been shown quite successful for the study of the motor and sensory functions, success is still far from being reached when it comes to covert cognitive functions such as attention, motivation and decision making. While improvement in this field of BCIs is growing fast, a new research focus has emerged from the development of strategies for decoding neural activity. In this review, we aim at exploring how the advanced in decoding of brain activity is becoming a major neuroscience tool moving forward our understanding of brain functions, providing a robust theoretical framework to test predictions on the relationship between brain activity and cognition and behavior.
Collapse
|
33
|
Nour MM, Liu Y, Dolan RJ. Functional neuroimaging in psychiatry and the case for failing better. Neuron 2022; 110:2524-2544. [PMID: 35981525 DOI: 10.1016/j.neuron.2022.07.005] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Revised: 06/06/2022] [Accepted: 07/08/2022] [Indexed: 12/27/2022]
Abstract
Psychiatric disorders encompass complex aberrations of cognition and affect and are among the most debilitating and poorly understood of any medical condition. Current treatments rely primarily on interventions that target brain function (drugs) or learning processes (psychotherapy). A mechanistic understanding of how these interventions mediate their therapeutic effects remains elusive. From the early 1990s, non-invasive functional neuroimaging, coupled with parallel developments in the cognitive neurosciences, seemed to signal a new era of neurobiologically grounded diagnosis and treatment in psychiatry. Yet, despite three decades of intense neuroimaging research, we still lack a neurobiological account for any psychiatric condition. Likewise, functional neuroimaging plays no role in clinical decision making. Here, we offer a critical commentary on this impasse and suggest how the field might fare better and deliver impactful neurobiological insights.
Collapse
Affiliation(s)
- Matthew M Nour
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Wellcome Trust Centre for Human Neuroimaging, University College London, London WC1N 3AR, UK; Department of Psychiatry, University of Oxford, Oxford OX3 7JX, UK.
| | - Yunzhe Liu
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China; Chinese Institute for Brain Research, Beijing 102206, China
| | - Raymond J Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Wellcome Trust Centre for Human Neuroimaging, University College London, London WC1N 3AR, UK; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China.
| |
Collapse
|
34
|
Chen ZS, Zhang X, Long X, Zhang SJ. Are Grid-Like Representations a Component of All Perception and Cognition? Front Neural Circuits 2022; 16:924016. [PMID: 35911570 PMCID: PMC9329517 DOI: 10.3389/fncir.2022.924016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Accepted: 06/14/2022] [Indexed: 11/24/2022] Open
Abstract
Grid cells or grid-like responses have been reported in the rodent, bat and human brains during various spatial and non-spatial tasks. However, the functions of grid-like representations beyond the classical hippocampal formation remain elusive. Based on accumulating evidence from recent rodent recordings and human fMRI data, we make speculative accounts regarding the mechanisms and functional significance of the sensory cortical grid cells and further make theory-driven predictions. We argue and reason the rationale why grid responses may be universal in the brain for a wide range of perceptual and cognitive tasks that involve locomotion and mental navigation. Computational modeling may provide an alternative and complementary means to investigate the grid code or grid-like map. We hope that the new discussion will lead to experimentally testable hypotheses and drive future experimental data collection.
Collapse
Affiliation(s)
- Zhe Sage Chen
- Department of Psychiatry, Department of Neuroscience and Physiology, Neuroscience Institute, New York University School of Medicine, New York, NY, United States
- *Correspondence: Zhe Sage Chen
| | - Xiaohan Zhang
- Department of Psychiatry, Department of Neuroscience and Physiology, Neuroscience Institute, New York University School of Medicine, New York, NY, United States
| | - Xiaoyang Long
- Department of Neurosurgery, Xinqiao Hospital, Army Medical University, Chongqing, China
| | - Sheng-Jia Zhang
- Department of Neurosurgery, Xinqiao Hospital, Army Medical University, Chongqing, China
- Sheng-Jia Zhang
| |
Collapse
|
35
|
Zhu S, Lakshminarasimhan KJ, Arfaei N, Angelaki DE. Eye movements reveal spatiotemporal dynamics of visually-informed planning in navigation. eLife 2022; 11:73097. [PMID: 35503099 PMCID: PMC9135400 DOI: 10.7554/elife.73097] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 05/01/2022] [Indexed: 11/28/2022] Open
Abstract
Goal-oriented navigation is widely understood to depend upon internal maps. Although this may be the case in many settings, humans tend to rely on vision in complex, unfamiliar environments. To study the nature of gaze during visually-guided navigation, we tasked humans to navigate to transiently visible goals in virtual mazes of varying levels of difficulty, observing that they took near-optimal trajectories in all arenas. By analyzing participants’ eye movements, we gained insights into how they performed visually-informed planning. The spatial distribution of gaze revealed that environmental complexity mediated a striking trade-off in the extent to which attention was directed towards two complimentary aspects of the world model: the reward location and task-relevant transitions. The temporal evolution of gaze revealed rapid, sequential prospection of the future path, evocative of neural replay. These findings suggest that the spatiotemporal characteristics of gaze during navigation are significantly shaped by the unique cognitive computations underlying real-world, sequential decision making.
Collapse
Affiliation(s)
- Seren Zhu
- Center for Neural Science, New York University, New York, United States
| | | | - Nastaran Arfaei
- Department of Psychology, New York University, New York, United States
| | - Dora E Angelaki
- Center for Neural Science, New York University, New York, United States
| |
Collapse
|
36
|
Ott F, Legler E, Kiebel SJ. Forward planning driven by context-dependant conflict processing in anterior cingulate cortex. Neuroimage 2022; 256:119222. [PMID: 35447352 DOI: 10.1016/j.neuroimage.2022.119222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Revised: 03/08/2022] [Accepted: 04/16/2022] [Indexed: 11/17/2022] Open
Abstract
Cognitive control and forward planning in particular is costly, and therefore must be regulated such that the amount of cognitive resources invested is adequate to the current situation. However, knowing in advance how beneficial forward planning will be in a given situation is hard. A way to know the exact value of planning would be to actually do it, which would ab initio defeat the purpose of regulating planning, i.e. the reduction of computational and time costs. One possible solution to this dilemma is that planning is regulated by learned associations between stimuli and the expected demand for planning. Such learning might be based on generalisation processes that cluster together stimulus states with similar control relevant properties into more general control contexts. In this way, the brain could infer the demand for planning, based on previous experience with situations that share some structural properties with the current situation. Here, we used a novel sequential task to test the hypothesis that people use control contexts to efficiently regulate their forward planning, using behavioural and functional magnetic resonance imaging data. Consistent with our hypothesis, reaction times increased with trial-by-trial conflict, where this increase was more pronounced in a context with a learned high demand for planning. Similarly, we found that fMRI activity in the dorsal anterior cingulate cortex (dACC) increased with conflict, and this increase was more pronounced in a context with generally high demand for planning. Taken together, the results indicate that the dACC integrates representations of planning demand at different levels of abstraction to regulate planning in an efficient and situation-appropriate way.
Collapse
Affiliation(s)
- Florian Ott
- Department of Psychology, Technische Universität Dresden, Dresden, Germany.
| | - Eric Legler
- Department of Psychology, Technische Universität Dresden, Dresden, Germany
| | - Stefan J Kiebel
- Department of Psychology, Technische Universität Dresden, Dresden, Germany; Centre for Tactile Internet with Human-in-the-Loop (CeTI), Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
37
|
Robertson EM. Memory leaks: information shared across memory systems. Trends Cogn Sci 2022; 26:544-554. [DOI: 10.1016/j.tics.2022.03.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Revised: 03/25/2022] [Accepted: 03/25/2022] [Indexed: 10/18/2022]
|
38
|
Abstract
In human neuroscience, studies of cognition are rarely grounded in non-task-evoked, 'spontaneous' neural activity. Indeed, studies of spontaneous activity tend to focus predominantly on intrinsic neural patterns (for example, resting-state networks). Taking a 'representation-rich' approach bridges the gap between cognition and resting-state communities: this approach relies on decoding task-related representations from spontaneous neural activity, allowing quantification of the representational content and rich dynamics of such activity. For example, if we know the neural representation of an episodic memory, we can decode its subsequent replay during rest. We argue that such an approach advances cognitive research beyond a focus on immediate task demand and provides insight into the functional relevance of the intrinsic neural pattern (for example, the default mode network). This in turn enables a greater integration between human and animal neuroscience, facilitating experimental testing of theoretical accounts of intrinsic activity, and opening new avenues of research in psychiatry.
Collapse
|
39
|
Optimism and pessimism in optimised replay. PLoS Comput Biol 2022; 18:e1009634. [PMID: 35020718 PMCID: PMC8809607 DOI: 10.1371/journal.pcbi.1009634] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Revised: 02/02/2022] [Accepted: 11/12/2021] [Indexed: 11/24/2022] Open
Abstract
The replay of task-relevant trajectories is known to contribute to memory consolidation and improved task performance. A wide variety of experimental data show that the content of replayed sequences is highly specific and can be modulated by reward as well as other prominent task variables. However, the rules governing the choice of sequences to be replayed still remain poorly understood. One recent theoretical suggestion is that the prioritization of replay experiences in decision-making problems is based on their effect on the choice of action. We show that this implies that subjects should replay sub-optimal actions that they dysfunctionally choose rather than optimal ones, when, by being forgetful, they experience large amounts of uncertainty in their internal models of the world. We use this to account for recent experimental data demonstrating exactly pessimal replay, fitting model parameters to the individual subjects’ choices. When animals are asleep or restfully awake, populations of neurons in their brains recapitulate activity associated with extended behaviourally-relevant experiences. This process is called replay, and it has been established for a long time in rodents, and very recently in humans, to be important for good performance in decision-making tasks. The specific experiences which are replayed during those epochs follow highly ordered patterns, but the mechanisms which establish their priority are still not fully understood. One promising theoretical suggestion is that each replay experience is chosen in such a way that the learning that ensues is most helpful for the subsequent performance of the animal. A very recent study reported a surprising result that humans who achieved high performance in a planning task tended to replay actions they found to be sub-optimal, and that this was associated with a useful deprecation of those actions in subsequent performance. In this study, we examine the nature of this pessimized form of replay and show that it is exactly appropriate for forgetful agents. We analyse the role of forgetting for replay choices of our model, and verify our predictions using human subject data.
Collapse
|
40
|
Cao L, Varga V, Chen ZS. Uncovering spatial representations from spatiotemporal patterns of rodent hippocampal field potentials. CELL REPORTS METHODS 2021; 1:100101. [PMID: 34888543 PMCID: PMC8654278 DOI: 10.1016/j.crmeth.2021.100101] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/09/2021] [Revised: 07/27/2021] [Accepted: 09/28/2021] [Indexed: 12/23/2022]
Abstract
Spatiotemporal patterns of large-scale spiking and field potentials of the rodent hippocampus encode spatial representations during maze runs, immobility, and sleep. Here, we show that multisite hippocampal field potential amplitude at ultra-high-frequency band (FPAuhf), a generalized form of multiunit activity, provides not only a fast and reliable reconstruction of the rodent's position when awake, but also a readout of replay content during sharp-wave ripples. This FPAuhf feature may serve as a robust real-time decoding strategy from large-scale recordings in closed-loop experiments. Furthermore, we develop unsupervised learning approaches to extract low-dimensional spatiotemporal FPAuhf features during run and ripple periods and to infer latent dynamical structures from lower-rank FPAuhf features. We also develop an optical flow-based method to identify propagating spatiotemporal LFP patterns from multisite array recordings, which can be used as a decoding application. Finally, we develop a prospective decoding strategy to predict an animal's future decision in goal-directed navigation.
Collapse
Affiliation(s)
- Liang Cao
- The Neuroscience Institute, New York University Grossman School of Medicine, New York, NY 10016, USA
- Department of Physics, East China Normal University, Shanghai 200241, China
| | - Viktor Varga
- The Neuroscience Institute, New York University Grossman School of Medicine, New York, NY 10016, USA
- Institute of Experimental Medicine, 43 Szigony Street, 1083 Budapest, Hungary
| | - Zhe S. Chen
- The Neuroscience Institute, New York University Grossman School of Medicine, New York, NY 10016, USA
- Department of Psychiatry, Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, NY 10016, USA
| |
Collapse
|
41
|
Basu R, Gebauer R, Herfurth T, Kolb S, Golipour Z, Tchumatchenko T, Ito HT. The orbitofrontal cortex maps future navigational goals. Nature 2021; 599:449-452. [PMID: 34707289 PMCID: PMC8599015 DOI: 10.1038/s41586-021-04042-9] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Accepted: 09/20/2021] [Indexed: 11/09/2022]
Abstract
Accurate navigation to a desired goal requires consecutive estimates of spatial relationships between the current position and future destination throughout the journey. Although neurons in the hippocampal formation can represent the position of an animal as well as its nearby trajectories1-7, their role in determining the destination of the animal has been questioned8,9. It is, thus, unclear whether the brain can possess a precise estimate of target location during active environmental exploration. Here we describe neurons in the rat orbitofrontal cortex (OFC) that form spatial representations persistently pointing to the subsequent goal destination of an animal throughout navigation. This destination coding emerges before the onset of navigation, without direct sensory access to a distal goal, and even predicts the incorrect destination of an animal at the beginning of an error trial. Goal representations in the OFC are maintained by destination-specific neural ensemble dynamics, and their brief perturbation at the onset of a journey led to a navigational error. These findings suggest that the OFC is part of the internal goal map of the brain, enabling animals to navigate precisely to a chosen destination that is beyond the range of sensory perception.
Collapse
Affiliation(s)
- Raunak Basu
- Max Planck Institute for Brain Research, Frankfurt am Main, Germany.
| | - Robert Gebauer
- Max Planck Institute for Brain Research, Frankfurt am Main, Germany
| | - Tim Herfurth
- Max Planck Institute for Brain Research, Frankfurt am Main, Germany
| | - Simon Kolb
- Max Planck Institute for Brain Research, Frankfurt am Main, Germany
| | - Zahra Golipour
- Max Planck Institute for Brain Research, Frankfurt am Main, Germany
| | - Tatjana Tchumatchenko
- Max Planck Institute for Brain Research, Frankfurt am Main, Germany.,Institute of Experimental Epileptology and Cognition Research, Life and Brain Center, Universitätsklinikum Bonn, Bonn, Germany
| | - Hiroshi T Ito
- Max Planck Institute for Brain Research, Frankfurt am Main, Germany.
| |
Collapse
|
42
|
Bush D, Ólafsdóttir HF, Barry C, Burgess N. Ripple band phase precession of place cell firing during replay. Curr Biol 2021; 32:64-73.e5. [PMID: 34731677 PMCID: PMC8751637 DOI: 10.1016/j.cub.2021.10.033] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Revised: 09/06/2021] [Accepted: 10/14/2021] [Indexed: 11/17/2022]
Abstract
Neuronal “replay,” in which place cell firing during rest recapitulates recently experienced trajectories, is thought to mediate the transmission of information from hippocampus to neocortex, but the mechanism for this transmission is unknown. Here, we show that replay uses a phase code to represent spatial trajectories by the phase of firing relative to the 150- to 250-Hz “ripple” oscillations that accompany replay events. This phase code is analogous to the theta phase precession of place cell firing during navigation, in which place cells fire at progressively earlier phases of the 6- to 12-Hz theta oscillation as their place field is traversed, providing information about self-location that is additional to the rate code and a necessary precursor of replay. Thus, during replay, each ripple cycle contains a “forward sweep” of decoded locations along the recapitulated trajectory. Our results indicate a novel encoding of trajectory information during replay and implicates phase coding as a general mechanism by which the hippocampus transmits experienced and replayed sequential information to downstream targets. Place cells fire at successively earlier ripple band phases during replay Ripple band firing phase during replay encodes location within the place field This produces forward sweeps of place cell activity during each ripple cycle
Collapse
Affiliation(s)
- Daniel Bush
- UCL Institute of Cognitive Neuroscience, Queen Square, London, UK; UCL Institute of Neurology, Queen Square, London, UK.
| | - H Freyja Ólafsdóttir
- Donders Institute for Brain Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
| | - Caswell Barry
- UCL Department of Cell and Developmental Biology, Gower Street, London, UK.
| | - Neil Burgess
- UCL Institute of Cognitive Neuroscience, Queen Square, London, UK; UCL Institute of Neurology, Queen Square, London, UK
| |
Collapse
|
43
|
Yang AI, Dikecligil GN, Jiang H, Das SR, Stein JM, Schuele SU, Rosenow JM, Davis KA, Lucas TH, Gottfried JA. The what and when of olfactory working memory in humans. Curr Biol 2021; 31:4499-4511.e8. [PMID: 34450088 DOI: 10.1016/j.cub.2021.08.004] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2020] [Revised: 06/15/2021] [Accepted: 08/02/2021] [Indexed: 12/31/2022]
Abstract
Encoding and retaining novel sequences of sensory stimuli in working memory is crucial for adaptive behavior. A fundamental challenge for the central nervous system is to maintain each sequence item in an active and discriminable state, while also preserving their temporal context. Nested neural oscillations have been postulated to disambiguate the "what" and "when" of sequences, but the mechanisms by which these multiple streams of information are coordinated in the human brain remain unclear. Drawing from foundational animal studies, we recorded local field potentials from the human piriform cortex and hippocampus during a working memory task in which subjects experienced sequences of three distinct odors. Our data revealed a unique organization of odor memories across multiple timescales of the theta rhythm. During encoding, odors elicited greater gamma at distinct theta phases in both regions, time stamping their positions in the sequence, whereby the robustness of this effect was predictive of temporal order memory. During maintenance, stimulus-driven patterns of theta-coupled gamma were spontaneously reinstated in piriform cortex, recapitulating the order of the initial sequence. Replay events were time compressed across contiguous theta cycles, coinciding with periods of enhanced piriform-hippocampal theta-phase synchrony, and their prevalence forecasted subsequent recall accuracy on a trial-by-trial basis. Our data provide a novel link between endogenous replay orchestrated by the theta rhythm and short-term retention of sequential memories in the human brain.
Collapse
Affiliation(s)
- Andrew I Yang
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA.
| | - Gulce N Dikecligil
- Department of Psychology, School of Arts and Sciences, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Heidi Jiang
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
| | - Sandhitsu R Das
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Joel M Stein
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Stephan U Schuele
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
| | - Joshua M Rosenow
- Department of Neurosurgery, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
| | - Kathryn A Davis
- Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Timothy H Lucas
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Jay A Gottfried
- Department of Psychology, School of Arts and Sciences, University of Pennsylvania, Philadelphia, PA 19104, USA; Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA.
| |
Collapse
|
44
|
Hall-McMaster S, Dayan P, Schuck NW. Control over patch encounters changes foraging behavior. iScience 2021; 24:103005. [PMID: 34522853 PMCID: PMC8426201 DOI: 10.1016/j.isci.2021.103005] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2021] [Revised: 06/29/2021] [Accepted: 08/17/2021] [Indexed: 12/27/2022] Open
Abstract
Foraging is a common decision problem in natural environments. When new exploitable sites are always available, a simple optimal strategy is to leave a current site when its return falls below a single average reward rate. Here, we examined foraging in a more structured environment, with a limited number of sites that replenished at different rates and had to be revisited. When participants could choose sites, they visited fast-replenishing sites more often, left sites at higher levels of reward, and achieved a higher net reward rate. Decisions to exploit-or-leave a site were best explained with a computational model that included both the average reward rate for the environment and reward information about the unattended sites. This suggests that unattended sites influence leave decisions, in foraging environments where sites can be revisited. Being able to select sites during foraging increased visits to high value sites This visitation pattern was efficient, producing higher average reward rates Decisions to leave a site were influenced by information about alternative sites
Collapse
Affiliation(s)
- Sam Hall-McMaster
- Max Planck Institute for Human Development, Berlin 14195, Germany.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin 14195, Germany.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
| | - Peter Dayan
- Max Planck Institute for Biological Cybernetics, Tübingen 72076, Germany.,University of Tübingen, Tübingen 72074, Germany
| | - Nicolas W Schuck
- Max Planck Institute for Human Development, Berlin 14195, Germany.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin 14195, Germany.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
| |
Collapse
|
45
|
Pezzulo G, Zorzi M, Corbetta M. The secret life of predictive brains: what's spontaneous activity for? Trends Cogn Sci 2021; 25:730-743. [PMID: 34144895 PMCID: PMC8363551 DOI: 10.1016/j.tics.2021.05.007] [Citation(s) in RCA: 73] [Impact Index Per Article: 24.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2020] [Revised: 05/14/2021] [Accepted: 05/19/2021] [Indexed: 01/23/2023]
Abstract
Brains at rest generate dynamical activity that is highly structured in space and time. We suggest that spontaneous activity, as in rest or dreaming, underlies top-down dynamics of generative models. During active tasks, generative models provide top-down predictive signals for perception, cognition, and action. When the brain is at rest and stimuli are weak or absent, top-down dynamics optimize the generative models for future interactions by maximizing the entropy of explanations and minimizing model complexity. Spontaneous fluctuations of correlated activity within and across brain regions may reflect transitions between 'generic priors' of the generative model: low dimensional latent variables and connectivity patterns of the most common perceptual, motor, cognitive, and interoceptive states. Even at rest, brains are proactive and predictive.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Roma, Italy.
| | - Marco Zorzi
- Department of General Psychology and Padova Neuroscience Center (PNC), University of Padova, Padova, Italy; IRCCS San Camillo Hospital, Venice, Italy
| | - Maurizio Corbetta
- Department of Neuroscience and Padova Neuroscience Center (PNC), University of Padova, Padova, Italy; Venetian Institute of Molecular Medicine (VIMM), Fondazione Biomedica, Padova, Italy
| |
Collapse
|
46
|
Liu C, Ye Z, Chen C, Axmacher N, Xue G. Hippocampal Representations of Event Structure and Temporal Context during Episodic Temporal Order Memory. Cereb Cortex 2021; 32:1520-1534. [PMID: 34464439 DOI: 10.1093/cercor/bhab304] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2021] [Revised: 07/31/2021] [Accepted: 08/01/2021] [Indexed: 11/13/2022] Open
Abstract
The hippocampus plays an important role in representing spatial locations and sequences and in transforming representations. How these representational structures and operations support memory for the temporal order of random items is still poorly understood. We addressed this question by leveraging the method of loci, a powerful mnemonic strategy for temporal order memory that particularly recruits hippocampus-dependent computations of spatial locations and associations. Applying representational similarity analysis to functional magnetic resonance imaging activation patterns revealed that hippocampal subfields contained representations of multiple features of sequence structure, including spatial locations, location distance, and sequence boundaries, as well as episodic-like temporal context. Critically, the hippocampal CA1 exhibited spatial transformation of representational patterns, showing lower pattern similarity for items in same locations than closely matched different locations during retrieval, whereas the CA23DG exhibited sequential transformation of representational patterns, showing lower pattern similarity for items in near locations than in far locations during encoding. These transformations enabled the encoding of multiple items in the same location and disambiguation of adjacent items. Our results suggest that the hippocampus can flexibly reconfigure multiplexed event structure representations to support accurate temporal order memory.
Collapse
Affiliation(s)
- Chuqi Liu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute of Brain Research, Beijing Normal University, Beijing 100875, PR China
| | - Zhifang Ye
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute of Brain Research, Beijing Normal University, Beijing 100875, PR China.,Department of Psychology, University of Oregon, Eugene, OR 97403, USA
| | - Chuansheng Chen
- Department of Psychological Science, University of California, Irvine, CA 92697, USA
| | - Nikolai Axmacher
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute of Brain Research, Beijing Normal University, Beijing 100875, PR China.,Department of Neuropsychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, Bochum 44801, Germany
| | - Gui Xue
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute of Brain Research, Beijing Normal University, Beijing 100875, PR China
| |
Collapse
|
47
|
Wittkuhn L, Chien S, Hall-McMaster S, Schuck NW. Replay in minds and machines. Neurosci Biobehav Rev 2021; 129:367-388. [PMID: 34371078 DOI: 10.1016/j.neubiorev.2021.08.002] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Revised: 07/19/2021] [Accepted: 08/01/2021] [Indexed: 11/19/2022]
Abstract
Experience-related brain activity patterns reactivate during sleep, wakeful rest, and brief pauses from active behavior. In parallel, machine learning research has found that experience replay can lead to substantial performance improvements in artificial agents. Together, these lines of research suggest replay has a variety of computational benefits for decision-making and learning. Here, we provide an overview of putative computational functions of replay as suggested by machine learning and neuroscientific research. We show that replay can lead to faster learning, less forgetting, reorganization or augmentation of experiences, and support planning and generalization. In addition, we highlight the benefits of reactivating abstracted internal representations rather than veridical memories, and discuss how replay could provide a mechanism to build internal representations that improve learning and decision-making.
Collapse
Affiliation(s)
- Lennart Wittkuhn
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Lentzeallee 94, D-14195 Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Lentzeallee 94, D-14195 Berlin, Germany.
| | - Samson Chien
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Lentzeallee 94, D-14195 Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Lentzeallee 94, D-14195 Berlin, Germany
| | - Sam Hall-McMaster
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Lentzeallee 94, D-14195 Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Lentzeallee 94, D-14195 Berlin, Germany
| | - Nicolas W Schuck
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Lentzeallee 94, D-14195 Berlin, Germany; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Lentzeallee 94, D-14195 Berlin, Germany.
| |
Collapse
|
48
|
Nour MM, Liu Y, Arumuham A, Kurth-Nelson Z, Dolan RJ. Impaired neural replay of inferred relationships in schizophrenia. Cell 2021; 184:4315-4328.e17. [PMID: 34197734 PMCID: PMC8357425 DOI: 10.1016/j.cell.2021.06.012] [Citation(s) in RCA: 29] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2020] [Revised: 04/09/2021] [Accepted: 06/10/2021] [Indexed: 12/19/2022]
Abstract
An ability to build structured mental maps of the world underpins our capacity to imagine relationships between objects that extend beyond experience. In rodents, such representations are supported by sequential place cell reactivations during rest, known as replay. Schizophrenia is proposed to reflect a compromise in structured mental representations, with animal models reporting abnormalities in hippocampal replay and associated ripple activity during rest. Here, utilizing magnetoencephalography (MEG), we tasked patients with schizophrenia and control participants to infer unobserved relationships between objects by reorganizing visual experiences containing these objects. During a post-task rest session, controls exhibited fast spontaneous neural reactivation of presented objects that replayed inferred relationships. Replay was coincident with increased ripple power in hippocampus. Patients showed both reduced replay and augmented ripple power relative to controls, convergent with findings in animal models. These abnormalities are linked to impairments in behavioral acquisition and subsequent neural representation of task structure.
Collapse
Affiliation(s)
- Matthew M Nour
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Wellcome Centre for Human Neuroimaging (WCHN), University College London, London WC1N 3AR, UK; Department of Psychosis Studies, Institute of Psychiatry Psychology and Neuroscience, King's College London, London SE5 8AF, UK.
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China; Chinese Institute for Brain Research, Beijing 102206, China
| | - Atheeshaan Arumuham
- Department of Psychosis Studies, Institute of Psychiatry Psychology and Neuroscience, King's College London, London SE5 8AF, UK
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Deepmind, London NC1 4AG, UK
| | - Raymond J Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Wellcome Centre for Human Neuroimaging (WCHN), University College London, London WC1N 3AR, UK; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing 100875, China; BIH Visiting Professor, Stiftung Charité, Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité - Universitätsmedizin, Berlin, Germany
| |
Collapse
|
49
|
Hunt LT, Daw ND, Kaanders P, MacIver MA, Mugan U, Procyk E, Redish AD, Russo E, Scholl J, Stachenfeld K, Wilson CRE, Kolling N. Formalizing planning and information search in naturalistic decision-making. Nat Neurosci 2021; 24:1051-1064. [PMID: 34155400 DOI: 10.1038/s41593-021-00866-w] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2020] [Accepted: 03/23/2021] [Indexed: 02/05/2023]
Abstract
Decisions made by mammals and birds are often temporally extended. They require planning and sampling of decision-relevant information. Our understanding of such decision-making remains in its infancy compared with simpler, forced-choice paradigms. However, recent advances in algorithms supporting planning and information search provide a lens through which we can explain neural and behavioral data in these tasks. We review these advances to obtain a clearer understanding for why planning and curiosity originated in certain species but not others; how activity in the medial temporal lobe, prefrontal and cingulate cortices may support these behaviors; and how planning and information search may complement each other as means to improve future action selection.
Collapse
Affiliation(s)
- L T Hunt
- Department of Psychiatry, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| | - N D Daw
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - P Kaanders
- Department of Experimental Psychology, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - M A MacIver
- Center for Robotics and Biosystems, Department of Neurobiology, Department of Biomedical Engineering, Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA
| | - U Mugan
- Center for Robotics and Biosystems, Department of Neurobiology, Department of Biomedical Engineering, Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA
| | - E Procyk
- Univ Lyon, Université Claude Bernard Lyon 1, INSERM, Stem Cell and Brain Research Institute U1208, Bron, France
| | - A D Redish
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - E Russo
- Department of Theoretical Neuroscience, Central Institute of Mental Health, Mannheim, Germany.,Department of Psychiatry and Psychotherapy, University Medical Center, Johannes Gutenberg University, Mainz, Germany
| | - J Scholl
- Department of Experimental Psychology, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | | | - C R E Wilson
- Univ Lyon, Université Claude Bernard Lyon 1, INSERM, Stem Cell and Brain Research Institute U1208, Bron, France
| | - N Kolling
- Department of Psychiatry, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| |
Collapse
|
50
|
Huang Q, Zhang H, Luo H. Sequence structure organizes items in varied latent states of working memory neural network. eLife 2021; 10:67589. [PMID: 34308840 PMCID: PMC8328517 DOI: 10.7554/elife.67589] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2021] [Accepted: 07/25/2021] [Indexed: 01/23/2023] Open
Abstract
In memory experiences, events do not exist independently but are linked with each other via structure-based organization. Structure context largely influences memory behavior, but how it is implemented in the brain remains unknown. Here, we combined magnetoencephalogram (MEG) recordings, computational modeling, and impulse-response approaches to probe the latent states when subjects held a list of items in working memory (WM). We demonstrate that sequence context reorganizes WM items into distinct latent states, that is, being reactivated at different latencies during WM retention, and the reactivation profiles further correlate with recency behavior. In contrast, memorizing the same list of items without sequence task requirements weakens the recency effect and elicits comparable neural reactivations. Computational modeling further reveals a dominant function of sequence context, instead of passive memory decaying, in characterizing recency effect. Taken together, sequence structure context shapes the way WM items are stored in the human brain and essentially influences memory behavior.
Collapse
Affiliation(s)
- Qiaoli Huang
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China.,PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China.,Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| | - Huihui Zhang
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China.,PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China.,Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| | - Huan Luo
- School of Psychological and Cognitive Sciences, Peking University, Beijing, China.,PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China.,Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| |
Collapse
|