1
|
Huang J, Zhang Z, Ruan X. An Improved Dyna-Q Algorithm Inspired by the Forward Prediction Mechanism in the Rat Brain for Mobile Robot Path Planning. Biomimetics (Basel) 2024; 9:315. [PMID: 38921195 PMCID: PMC11202125 DOI: 10.3390/biomimetics9060315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2024] [Revised: 05/04/2024] [Accepted: 05/08/2024] [Indexed: 06/27/2024] Open
Abstract
The traditional Model-Based Reinforcement Learning (MBRL) algorithm has high computational cost, poor convergence, and poor performance in robot spatial cognition and navigation tasks, and it cannot fully explain the ability of animals to quickly adapt to environmental changes and learn a variety of complex tasks. Studies have shown that vicarious trial and error (VTE) and the hippocampus forward prediction mechanism in rats and other mammals can be used as key components of action selection in MBRL to support "goal-oriented" behavior. Therefore, we propose an improved Dyna-Q algorithm inspired by the forward prediction mechanism of the hippocampus to solve the above problems and tackle the exploration-exploitation dilemma of Reinforcement Learning (RL). This algorithm alternately presents the potential path in the future for mobile robots and dynamically adjusts the sweep length according to the decision certainty, so as to determine action selection. We test the performance of the algorithm in a two-dimensional maze environment with static and dynamic obstacles, respectively. Compared with classic RL algorithms like State-Action-Reward-State-Action (SARSA) and Dyna-Q, the algorithm can speed up spatial cognition and improve the global search ability of path planning. In addition, our method reflects key features of how the brain organizes MBRL to effectively solve difficult tasks such as navigation, and it provides a new idea for spatial cognitive tasks from a biological perspective.
Collapse
Affiliation(s)
- Jing Huang
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
- Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing 100124, China
| | - Ziheng Zhang
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
- Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing 100124, China
| | - Xiaogang Ruan
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
- Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing 100124, China
| |
Collapse
|
2
|
Reinshagen A. Grid cells: the missing link in understanding Parkinson's disease? Front Neurosci 2024; 18:1276714. [PMID: 38389787 PMCID: PMC10881698 DOI: 10.3389/fnins.2024.1276714] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2023] [Accepted: 01/24/2024] [Indexed: 02/24/2024] Open
Abstract
The mechanisms underlying Parkinson's disease (PD) are complex and not fully understood, and the box-and-arrow model among other current models present significant challenges. This paper explores the potential role of the allocentric brain and especially its grid cells in several PD motor symptoms, including bradykinesia, kinesia paradoxa, freezing of gait, the bottleneck phenomenon, and their dependency on cueing. It is argued that central hubs, like the locus coeruleus and the pedunculopontine nucleus, often narrowly interpreted in the context of PD, play an equally important role in governing the allocentric brain as the basal ganglia. Consequently, the motor and secondary motor (e.g., spatially related) symptoms of PD linked with dopamine depletion may be more closely tied to erroneous computation by grid cells than to the basal ganglia alone. Because grid cells and their associated central hubs introduce both spatial and temporal information to the brain influencing velocity perception they may cause bradykinesia or hyperkinesia as well. In summary, PD motor symptoms may primarily be an allocentric disturbance resulting from virtual faulty computation by grid cells revealed by dopamine depletion in PD.
Collapse
|
3
|
Nwakama CA, Durand-de Cuttoli R, Oketokoun ZM, Brown SO, Haller JE, Méndez A, Farshbaf MJ, Cho YZ, Ahmed S, Leng S, Ables JL, Sweis BM. Diabetes alters neuroeconomically dissociable forms of mental accounting. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.04.574210. [PMID: 38260368 PMCID: PMC10802482 DOI: 10.1101/2024.01.04.574210] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/24/2024]
Abstract
Those with diabetes mellitus are at high-risk of developing psychiatric disorders, yet the link between hyperglycemia and alterations in motivated behavior has not been explored in detail. We characterized value-based decision-making behavior of a streptozocin-induced diabetic mouse model on a naturalistic neuroeconomic foraging paradigm called Restaurant Row. Mice made self-paced choices while on a limited time-budget accepting or rejecting reward offers as a function of cost (delays cued by tone-pitch) and subjective value (flavors), tested daily in a closed-economy system across months. We found streptozocin-treated mice disproportionately undervalued less-preferred flavors and inverted their meal-consumption patterns shifted toward a more costly strategy that overprioritized high-value rewards. We discovered these foraging behaviors were driven by impairments in multiple decision-making systems, including the ability to deliberate when engaged in conflict and cache the value of the passage of time in the form of sunk costs. Surprisingly, diabetes-induced changes in behavior depended not only on the type of choice being made but also the salience of reward-scarcity in the environment. These findings suggest complex relationships between glycemic regulation and dissociable valuation algorithms underlying unique cognitive heuristics and sensitivity to opportunity costs can disrupt fundamentally distinct computational processes and could give rise to psychiatric vulnerabilities.
Collapse
|
4
|
Dragoi G. The generative grammar of the brain: a critique of internally generated representations. Nat Rev Neurosci 2024; 25:60-75. [PMID: 38036709 DOI: 10.1038/s41583-023-00763-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/18/2023] [Indexed: 12/02/2023]
Abstract
The past decade of progress in neurobiology has uncovered important organizational principles for network preconfiguration and neuronal selection that suggest a generative grammar exists in the brain. In this Perspective, I discuss the competence of the hippocampal neural network to generically express temporally compressed sequences of neuronal firing that represent novel experiences, which is envisioned as a form of generative neural syntax supporting a neurobiological perspective on brain function. I compare this neural competence with the hippocampal network performance that represents specific experiences with higher fidelity after new learning during replay, which is envisioned as a form of neural semantic that supports a complementary neuropsychological perspective. I also demonstrate how the syntax of network competence emerges a priori during early postnatal life and is followed by the later development of network performance that enables rapid encoding and memory consolidation. Thus, I propose that this generative grammar of the brain is essential for internally generated representations, which are crucial for the cognitive processes underlying learning and memory, prospection, and inference, which ultimately underlie our reason and representation of the world.
Collapse
Affiliation(s)
- George Dragoi
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT, USA.
- Department of Neuroscience, Yale University School of Medicine, New Haven, CT, USA.
- Wu Tsai Institute, Yale University, New Haven, CT, USA.
| |
Collapse
|
5
|
Fraser KM, Chen BJ, Janak PH. Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance. Eur J Neurosci 2024; 59:220-237. [PMID: 38093522 PMCID: PMC10841748 DOI: 10.1111/ejn.16210] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Revised: 11/09/2023] [Accepted: 11/15/2023] [Indexed: 01/23/2024]
Abstract
Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.
Collapse
Affiliation(s)
- Kurt M. Fraser
- Department of Psychological & Brain Sciences, Krieger School of Arts & Sciences, Johns Hopkins University, Baltimore, MD, 21218
| | - Bridget J. Chen
- Department of Psychological & Brain Sciences, Krieger School of Arts & Sciences, Johns Hopkins University, Baltimore, MD, 21218
| | - Patricia H. Janak
- Department of Psychological & Brain Sciences, Krieger School of Arts & Sciences, Johns Hopkins University, Baltimore, MD, 21218
- Solomon H. Snyder Department of Neuroscience, School of Medicine, Johns Hopkins University, Baltimore, MD, 21205
- Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD 21218
| |
Collapse
|
6
|
Sherman BE, Turk-Browne NB, Goldfarb EV. Multiple Memory Subsystems: Reconsidering Memory in the Mind and Brain. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2024; 19:103-125. [PMID: 37390333 PMCID: PMC10756937 DOI: 10.1177/17456916231179146] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/02/2023]
Abstract
The multiple-memory-systems framework-that distinct types of memory are supported by distinct brain systems-has guided learning and memory research for decades. However, recent work challenges the one-to-one mapping between brain structures and memory types central to this taxonomy, with key memory-related structures supporting multiple functions across substructures. Here we integrate cross-species findings in the hippocampus, striatum, and amygdala to propose an updated framework of multiple memory subsystems (MMSS). We provide evidence for two organizational principles of the MMSS theory: First, opposing memory representations are colocated in the same brain structures; second, parallel memory representations are supported by distinct structures. We discuss why this burgeoning framework has the potential to provide a useful revision of classic theories of long-term memory, what evidence is needed to further validate the framework, and how this novel perspective on memory organization may guide future research.
Collapse
Affiliation(s)
| | | | - Elizabeth V Goldfarb
- Department of Psychology, Yale University
- Wu Tsai Institute, Yale University
- Department of Psychiatry, Yale University
- National Center for PTSD, West Haven, USA
| |
Collapse
|
7
|
Varin C, Cornil A, Houtteman D, Bonnavion P, de Kerchove d'Exaerde A. The respective activation and silencing of striatal direct and indirect pathway neurons support behavior encoding. Nat Commun 2023; 14:4982. [PMID: 37591838 PMCID: PMC10435545 DOI: 10.1038/s41467-023-40677-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Accepted: 08/03/2023] [Indexed: 08/19/2023] Open
Abstract
The basal ganglia are known to control actions and modulate movements. Neuronal activity in the two efferent pathways of the dorsal striatum is critical for appropriate behavioral control. Previous evidence has led to divergent conclusions on the respective engagement of both pathways during actions. Using calcium imaging to evaluate how neurons in the direct and indirect pathways encode behaviors during self-paced spontaneous explorations in an open field, we observed that the two striatal pathways exhibit distinct tuning properties. Supervised learning algorithms revealed that direct pathway neurons encode behaviors through their activation, whereas indirect pathway neurons exhibit behavior-specific silencing. These properties remain stable for weeks. Our findings highlight a complementary encoding of behaviors with congruent activations in the direct pathway encoding multiple accessible behaviors in a given context, and in the indirect pathway encoding the suppression of competing behaviors. This model reconciles previous conflicting conclusions on motor encoding in the striatum.
Collapse
Affiliation(s)
- Christophe Varin
- Université Libre de Bruxelles (ULB), ULB Neuroscience Institute, Neurophysiology Laboratory, Brussels, Belgium
| | - Amandine Cornil
- Université Libre de Bruxelles (ULB), ULB Neuroscience Institute, Neurophysiology Laboratory, Brussels, Belgium
| | - Delphine Houtteman
- Université Libre de Bruxelles (ULB), ULB Neuroscience Institute, Neurophysiology Laboratory, Brussels, Belgium
| | - Patricia Bonnavion
- Université Libre de Bruxelles (ULB), ULB Neuroscience Institute, Neurophysiology Laboratory, Brussels, Belgium
| | - Alban de Kerchove d'Exaerde
- Université Libre de Bruxelles (ULB), ULB Neuroscience Institute, Neurophysiology Laboratory, Brussels, Belgium.
| |
Collapse
|
8
|
Levcik D, Sugi AH, Aguilar-Rivera M, Pochapski JA, Baltazar G, Pulido LN, Villas-Boas CA, Fuentes-Flores R, Nicola SM, Da Cunha C. Nucleus Accumbens Shell Neurons Encode the Kinematics of Reward Approach Locomotion. Neuroscience 2023; 524:181-196. [PMID: 37330195 PMCID: PMC10527230 DOI: 10.1016/j.neuroscience.2023.06.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 05/30/2023] [Accepted: 06/02/2023] [Indexed: 06/19/2023]
Abstract
The nucleus accumbens (NAc) is considered an interface between motivation and action, with NAc neurons playing an important role in promoting reward approach. However, the encoding by NAc neurons that contributes to this role remains unknown. We recorded 62 NAc neurons in male Wistar rats (n = 5) running towards rewarded locations in an 8-arm radial maze. Variables related to locomotor approach kinematics were the best predictors of the firing rate for most NAc neurons. Nearly 18% of the recorded neurons were inhibited during the entire approach run (locomotion-off cells), suggesting that reduction in firing of these neurons promotes initiation of locomotor approach. 27% of the neurons presented a peak of activity during acceleration followed by a valley during deceleration (acceleration-on cells). Together, these neurons accounted for most of the speed and acceleration encoding identified in our analysis. In contrast, a further 16% of neurons presented a valley during acceleration followed by a peak just prior to or after reaching reward (deceleration-on cells). These findings suggest that these three classes of NAc neurons influence the time course of speed changes during locomotor approach to reward.
Collapse
Affiliation(s)
- David Levcik
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Institute of Physiology of the Czech Academy of Sciences, Videnska 1083, 142 20 Prague, Czech Republic
| | - Adam H Sugi
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil; Department of Biochemistry, Universidade Federal do Paraná, Curitiba, Brazil
| | - Marcelo Aguilar-Rivera
- Department of Bioengineering, University of California, 9500 Gilman Drive MC 0412, La Jolla, San Diego 92093, USA
| | - José A Pochapski
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil
| | - Gabriel Baltazar
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil; Department of Biochemistry, Universidade Federal do Paraná, Curitiba, Brazil
| | - Laura N Pulido
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil
| | - Cyrus A Villas-Boas
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil
| | - Romulo Fuentes-Flores
- Departamento de Neurociencia, Facultad de Medicina, Universidad de Chile, Av. Independencia 1027, Independencia 8380453, Santiago, Chile
| | - Saleem M Nicola
- Department of Neuroscience, Albert Einstein College of Medicine, 1300 Morris Park Ave, Bronx, NY 10461, USA; Department of Psychiatry, Albert Einstein College of Medicine, New York, USA
| | - Claudio Da Cunha
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil; Department of Biochemistry, Universidade Federal do Paraná, Curitiba, Brazil.
| |
Collapse
|
9
|
Harvey RE, Robinson HL, Liu C, Oliva A, Fernandez-Ruiz A. Hippocampo-cortical circuits for selective memory encoding, routing, and replay. Neuron 2023; 111:2076-2090.e9. [PMID: 37196658 PMCID: PMC11146684 DOI: 10.1016/j.neuron.2023.04.015] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 03/15/2023] [Accepted: 04/12/2023] [Indexed: 05/19/2023]
Abstract
Traditionally considered a homogeneous cell type, hippocampal pyramidal cells have been recently shown to be highly diverse. However, how this cellular diversity relates to the different hippocampal network computations that support memory-guided behavior is not yet known. We show that the anatomical identity of pyramidal cells is a major organizing principle of CA1 assembly dynamics, the emergence of memory replay, and cortical projection patterns in rats. Segregated pyramidal cell subpopulations encoded trajectory and choice-specific information or tracked changes in reward configuration respectively, and their activity was selectively read out by different cortical targets. Furthermore, distinct hippocampo-cortical assemblies coordinated the reactivation of complementary memory representations. These findings reveal the existence of specialized hippocampo-cortical subcircuits and provide a cellular mechanism that supports the computational flexibility and memory capacities of these structures.
Collapse
Affiliation(s)
- Ryan E Harvey
- Department of Neurobiology & Behavior, Cornell University, Ithaca, NY, USA
| | - Heath L Robinson
- Department of Neurobiology & Behavior, Cornell University, Ithaca, NY, USA
| | - Can Liu
- Department of Neurobiology & Behavior, Cornell University, Ithaca, NY, USA
| | - Azahara Oliva
- Department of Neurobiology & Behavior, Cornell University, Ithaca, NY, USA.
| | | |
Collapse
|
10
|
James A, Reynaud-Bouret P, Mezzadri G, Sargolini F, Bethus I, Muzy A. Strategy inference during learning via cognitive activity-based credit assignment models. Sci Rep 2023; 13:9408. [PMID: 37296163 PMCID: PMC10256696 DOI: 10.1038/s41598-023-33604-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Accepted: 04/15/2023] [Indexed: 06/12/2023] Open
Abstract
We develop a method for selecting meaningful learning strategies based solely on the behavioral data of a single individual in a learning experiment. We use simple Activity-Credit Assignment algorithms to model the different strategies and couple them with a novel hold-out statistical selection method. Application on rat behavioral data in a continuous T-maze task reveals a particular learning strategy that consists in chunking the paths used by the animal. Neuronal data collected in the dorsomedial striatum confirm this strategy.
Collapse
Affiliation(s)
| | | | - Giulia Mezzadri
- Columbia University, Cognition and Decision Lab, New York, USA
| | - Francesca Sargolini
- Aix Marseille Université, CNRS, Laboratoire de Neurosciences Cognitives, Marseille, France
| | | | | |
Collapse
|
11
|
Knowledge generalization and the costs of multitasking. Nat Rev Neurosci 2023; 24:98-112. [PMID: 36347942 DOI: 10.1038/s41583-022-00653-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/12/2022] [Indexed: 11/10/2022]
Abstract
Humans are able to rapidly perform novel tasks, but show pervasive performance costs when attempting to do two things at once. Traditionally, empirical and theoretical investigations into the sources of such multitasking interference have largely focused on multitasking in isolation to other cognitive functions, characterizing the conditions that give rise to performance decrements. Here we instead ask whether multitasking costs are linked to the system's capacity for knowledge generalization, as is required to perform novel tasks. We show how interrogation of the neurophysiological circuitry underlying these two facets of cognition yields further insights for both. Specifically, we demonstrate how a system that rapidly generalizes knowledge may induce multitasking costs owing to sharing of task contingencies between contexts in neural representations encoded in frontoparietal and striatal brain regions. We discuss neurophysiological insights suggesting that prolonged learning segregates such representations by refining the brain's model of task-relevant contingencies, thereby reducing information sharing between contexts and improving multitasking performance while reducing flexibility and generalization. These proposed neural mechanisms explain why the brain shows rapid task understanding, multitasking limitations and practice effects. In short, multitasking limits are the price we pay for behavioural flexibility.
Collapse
|
12
|
Mahr JB, Fischer B. Internally Triggered Experiences of Hedonic Valence in Nonhuman Animals: Cognitive and Welfare Considerations. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2022; 18:688-701. [PMID: 36288434 DOI: 10.1177/17456916221120425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Do any nonhuman animals have hedonically valenced experiences not directly caused by stimuli in their current environment? Do they, like us humans, experience anticipated or previously experienced pains and pleasures as respectively painful and pleasurable? We review evidence from comparative neuroscience about hippocampus-dependent simulation in relation to this question. Hippocampal sharp-wave ripples and theta oscillations have been found to instantiate previous and anticipated experiences. These hippocampal activations coordinate with neural reward and fear centers as well as sensory and cortical areas in ways that are associated with conscious episodic mental imagery in humans. Moreover, such hippocampal “re- and preplay” has been found to contribute to instrumental decision making, the learning of value representations, and the delay of rewards in rats. The functional and structural features of hippocampal simulation are highly conserved across mammals. This evidence makes it reasonable to assume that internally triggered experiences of hedonic valence (IHVs) are pervasive across (at least) all mammals. This conclusion has important welfare implications. Most prominently, IHVs act as a kind of “welfare multiplier” through which the welfare impacts of any given experience of pain or pleasure are increased through each future retrieval. However, IHVs also have practical implications for welfare assessment and cause prioritization.
Collapse
Affiliation(s)
| | - Bob Fischer
- Department of Philosophy, Texas State University
| |
Collapse
|
13
|
Tennant SA, Clark H, Hawes I, Tam WK, Hua J, Yang W, Gerlei KZ, Wood ER, Nolan MF. Spatial representation by ramping activity of neurons in the retrohippocampal cortex. Curr Biol 2022; 32:4451-4464.e7. [PMID: 36099915 DOI: 10.1016/j.cub.2022.08.050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Revised: 07/05/2022] [Accepted: 08/17/2022] [Indexed: 11/18/2022]
Abstract
Neurons in the retrohippocampal cortices play crucial roles in spatial memory. Many retrohippocampal neurons have firing fields that are selectively active at specific locations, with memory for rewarded locations associated with reorganization of these firing fields. Whether this is the sole strategy for representing spatial memories is unclear. Here, we demonstrate that during a spatial memory task retrohippocampal neurons encode location through ramping activity that extends across segments of a linear track approaching and following a reward, with the rewarded location represented by offsets or switches in the slope of the ramping activity. Ramping representations could be maintained independently of trial outcome and cues marking the reward location, indicating that they result from recall of the track structure. When recorded in an open arena, neurons that generated ramping activity during the spatial memory task were more numerous than grid or border cells, with a majority showing spatial firing that did not meet criteria for classification as grid or border representations. Encoding of rewarded locations through offsets and switches in the slope of ramping activity also emerged in recurrent neural network models trained to solve a similar spatial memory task. Impaired performance of model networks following disruption of outputs from ramping neurons is consistent with this coding strategy supporting navigation to recalled locations of behavioral significance. Our results suggest that encoding of learned spaces by retrohippocampal networks employs both discrete firing fields and continuous ramping representations. We hypothesize that retrohippocampal ramping activity mediates readout of learned models for goal-directed navigation.
Collapse
Affiliation(s)
- Sarah A Tennant
- Centre for Discovery Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Harry Clark
- Centre for Discovery Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Ian Hawes
- Centre for Discovery Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Wing Kin Tam
- Centre for Discovery Brain Sciences, University of Edinburgh, Edinburgh, UK; Simons Initiative for the Developing Brain, University of Edinburgh, Edinburgh, UK
| | - Junji Hua
- Centre for Discovery Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Wannan Yang
- Centre for Discovery Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Klara Z Gerlei
- Centre for Discovery Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Emma R Wood
- Centre for Discovery Brain Sciences, University of Edinburgh, Edinburgh, UK; Simons Initiative for the Developing Brain, University of Edinburgh, Edinburgh, UK
| | - Matthew F Nolan
- Centre for Discovery Brain Sciences, University of Edinburgh, Edinburgh, UK; Simons Initiative for the Developing Brain, University of Edinburgh, Edinburgh, UK; Centre for Statistics, University of Edinburgh, Edinburgh, UK.
| |
Collapse
|
14
|
Hazon O, Minces VH, Tomàs DP, Ganguli S, Schnitzer MJ, Jercog PE. Noise correlations in neural ensemble activity limit the accuracy of hippocampal spatial representations. Nat Commun 2022; 13:4276. [PMID: 35879320 PMCID: PMC9314334 DOI: 10.1038/s41467-022-31254-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2021] [Accepted: 06/07/2022] [Indexed: 11/29/2022] Open
Abstract
Neurons in the CA1 area of the mouse hippocampus encode the position of the animal in an environment. However, given the variability in individual neurons responses, the accuracy of this code is still poorly understood. It was proposed that downstream areas could achieve high spatial accuracy by integrating the activity of thousands of neurons, but theoretical studies point to shared fluctuations in the firing rate as a potential limitation. Using high-throughput calcium imaging in freely moving mice, we demonstrated the limiting factors in the accuracy of the CA1 spatial code. We found that noise correlations in the hippocampus bound the estimation error of spatial coding to ~10 cm (the size of a mouse). Maximal accuracy was obtained using approximately [300-1400] neurons, depending on the animal. These findings reveal intrinsic limits in the brain's representations of space and suggest that single neurons downstream of the hippocampus can extract maximal spatial information from several hundred inputs.
Collapse
Affiliation(s)
| | | | - David P Tomàs
- Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | | | | | - Pablo E Jercog
- Stanford University, Stanford, CA, USA.
- Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain.
| |
Collapse
|
15
|
The ventral midline thalamus coordinates prefrontal-hippocampal neural synchrony during vicarious trial and error. Sci Rep 2022; 12:10940. [PMID: 35768454 PMCID: PMC9243057 DOI: 10.1038/s41598-022-14707-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Accepted: 06/10/2022] [Indexed: 11/22/2022] Open
Abstract
When faced with difficult choices, the possible outcomes are considered through a process known as deliberation. In rats, deliberation is thought to be reflected by pause-and-reorienting behaviors, better known as vicarious trial and errors (VTEs). While VTEs are thought to require medial prefrontal cortex (mPFC) and dorsal hippocampal (dHPC) interactions, no empirical evidence has yet demonstrated such a dual requirement. The nucleus reuniens (Re) of the ventral midline thalamus is anatomically connected with both the mPFC and dHPC, is required for HPC-dependent spatial memory tasks, and is critical for mPFC-dHPC neural synchronization. Currently, it is unclear if, or how, the Re is involved in deliberation. Therefore, by examining the role of the Re on VTE behaviors, we can better understand the anatomical and physiological mechanisms supporting deliberation. Here, we examined the impact of Re suppression on VTE behaviors and mPFC-dHPC theta synchrony during asymptotic performance of a HPC-dependent delayed alternation (DA) task. Pharmacological suppression of the Re increased VTE behaviors that occurred with repetitive choice errors. These errors were best characterized as perseverative behaviors, in which some rats repeatedly selected a goal arm that previously yielded no reward. We then examined the impact of Re suppression on mPFC-dHPC theta synchrony during VTEs. We found that during VTEs, Re inactivation was associated with a reduction in mPFC-dHPC theta coherence and mPFC-to-dHPC theta directionality. Our findings suggest that the Re contributes to deliberation by coordinating mPFC-dHPC neural interactions.
Collapse
|
16
|
Lubianiker N, Paret C, Dayan P, Hendler T. Neurofeedback through the lens of reinforcement learning. Trends Neurosci 2022; 45:579-593. [PMID: 35550813 DOI: 10.1016/j.tins.2022.03.008] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Revised: 02/11/2022] [Accepted: 03/24/2022] [Indexed: 11/29/2022]
Abstract
Despite decades of experimental and clinical practice, the neuropsychological mechanisms underlying neurofeedback (NF) training remain obscure. NF is a unique form of reinforcement learning (RL) task, during which participants are provided with rewarding feedback regarding desired changes in neural patterns. However, key RL considerations - including choices during practice, prediction errors, credit-assignment problems, or the exploration-exploitation tradeoff - have infrequently been considered in the context of NF. We offer an RL-based framework for NF, describing different internal states, actions, and rewards in common NF protocols, thus fashioning new proposals for characterizing, predicting, and hastening the course of learning. In this way we hope to advance current understanding of neural regulation via NF, and ultimately to promote its effectiveness, personalization, and clinical utility.
Collapse
Affiliation(s)
- Nitzan Lubianiker
- School of Psychological Sciences, Gershon H. Gordon Faculty of Social Sciences, Tel Aviv University, Tel Aviv, Israel; Sagol Brain Institute, Wohl Institute for Advanced Imaging, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel.
| | - Christian Paret
- School of Psychological Sciences, Gershon H. Gordon Faculty of Social Sciences, Tel Aviv University, Tel Aviv, Israel; Sagol Brain Institute, Wohl Institute for Advanced Imaging, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel; Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim/Heidelberg University, Mannheim, Germany
| | - Peter Dayan
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany; University of Tübingen, Tübingen, Germany
| | - Talma Hendler
- School of Psychological Sciences, Gershon H. Gordon Faculty of Social Sciences, Tel Aviv University, Tel Aviv, Israel; Sagol Brain Institute, Wohl Institute for Advanced Imaging, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel; Sagol school of Neuroscience, Tel Aviv University, Tel Aviv, Israel; Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| |
Collapse
|
17
|
Redish AD, Kepecs A, Anderson LM, Calvin OL, Grissom NM, Haynos AF, Heilbronner SR, Herman AB, Jacob S, Ma S, Vilares I, Vinogradov S, Walters CJ, Widge AS, Zick JL, Zilverstand A. Computational validity: using computation to translate behaviours across species. Philos Trans R Soc Lond B Biol Sci 2022; 377:20200525. [PMID: 34957854 PMCID: PMC8710889 DOI: 10.1098/rstb.2020.0525] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2021] [Accepted: 07/28/2021] [Indexed: 11/12/2022] Open
Abstract
We propose a new conceptual framework (computational validity) for translation across species and populations based on the computational similarity between the information processing underlying parallel tasks. Translating between species depends not on the superficial similarity of the tasks presented, but rather on the computational similarity of the strategies and mechanisms that underlie those behaviours. Computational validity goes beyond construct validity by directly addressing questions of information processing. Computational validity interacts with circuit validity as computation depends on circuits, but similar computations could be accomplished by different circuits. Because different individuals may use different computations to accomplish a given task, computational validity suggests that behaviour should be understood through the subject's point of view; thus, behaviour should be characterized on an individual level rather than a task level. Tasks can constrain the computational algorithms available to a subject and the observed subtleties of that behaviour can provide information about the computations used by each individual. Computational validity has especially high relevance for the study of psychiatric disorders, given the new views of psychiatry as identifying and mediating information processing dysfunctions that may show high inter-individual variability, as well as for animal models investigating aspects of human psychiatric disorders. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.
Collapse
Affiliation(s)
- A. David Redish
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | - Adam Kepecs
- Department of Neuroscience, Washington University in St. Louis, St Louis, MO 63110, USA
- Department of Psychiatry, Washington University in St. Louis, St Louis, MO 63110, USA
| | - Lisa M. Anderson
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| | - Olivia L. Calvin
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| | - Nicola M. Grissom
- Department of Psychology, University of Minnesota, Minneapolis, MN 55455, USA
| | - Ann F. Haynos
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| | | | - Alexander B. Herman
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| | - Suma Jacob
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| | - Sisi Ma
- Department of Medicine - Institute for Health Informatics, University of Minnesota, Minneapolis, MN 55455, USA
| | - Iris Vilares
- Department of Psychology, University of Minnesota, Minneapolis, MN 55455, USA
| | - Sophia Vinogradov
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| | - Cody J. Walters
- Graduate Program in Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | - Alik S. Widge
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| | - Jennifer L. Zick
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| | - Anna Zilverstand
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| |
Collapse
|
18
|
Marquardt K, Josey M, Kenton JA, Cavanagh JF, Holmes A, Brigman JL. Impaired cognitive flexibility following NMDAR-GluN2B deletion is associated with altered orbitofrontal-striatal function. Neuroscience 2021; 475:230-245. [PMID: 34656223 PMCID: PMC8592269 DOI: 10.1016/j.neuroscience.2021.07.028] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/02/2023]
Abstract
A common feature across neuropsychiatric disorders is inability to discontinue an action or thought once it has become detrimental. Reversal learning, a hallmark of executive control, requires plasticity within cortical, striatal and limbic circuits and is highly sensitive to disruption of N-methyl-d-aspartate receptor (NMDAR) function. In particular, selective deletion or antagonism of GluN2B containing NMDARs in cortical regions including the orbitofrontal cortex (OFC), promotes maladaptive perseveration. It remains unknown whether GluN2B functions to maintain local cortical activity necessary for reversal learning, or if it exerts a broader influence on the integration of neural activity across cortical and subcortical systems. To address this question, we utilized in vivo electrophysiology to record neuronal activity and local field potentials (LFP) in the orbitofrontal cortex and dorsal striatum (dS) of mice with deletion of GluN2B in neocortical and hippocampal principal cells while they performed touchscreen reversal learning. Reversal impairment produced by corticohippocampal GluN2B deletion was paralleled by an aberrant increase in functional connectivity between the OFC and dS. These alterations in coordination were associated with alterations in local OFC and dS firing activity. These data demonstrate highly dynamic patterns of cortical and striatal activity concomitant with reversal learning, and reveal GluN2B as a molecular mechanism underpinning the timing of these processes.
Collapse
Affiliation(s)
- Kristin Marquardt
- Department of Neurosciences, University of New Mexico School of Medicine, Albuquerque, NM, USA
| | - Megan Josey
- Department of Neurosciences, University of New Mexico School of Medicine, Albuquerque, NM, USA
| | - Johnny A Kenton
- Department of Neurosciences, University of New Mexico School of Medicine, Albuquerque, NM, USA
| | | | - Andrew Holmes
- Laboratory of Behavioral and Genomic Neuroscience, National Institute on Alcohol Abuse and Alcoholism, Bethesda, MD, USA
| | - Jonathan L Brigman
- Department of Neurosciences, University of New Mexico School of Medicine, Albuquerque, NM, USA; New Mexico Alcohol Research Center, UNM Health Sciences Center, Albuquerque, NM, USA.
| |
Collapse
|
19
|
Numan R. The Prefrontal-Hippocampal Comparator: Volition and Episodic Memory. Percept Mot Skills 2021; 128:2421-2447. [PMID: 34424092 DOI: 10.1177/00315125211041341] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
This review describes recent research that is relevant to the prefrontal-hippocampal comparator model with the following conclusions: 1. Hippocampal area CA1 serves, at least in part, as an associative match-mismatch comparator. 2. Voluntary movement strengthens episodic memories for goal-directed behavior. 3. Hippocampal theta power serves as a prediction error signal during hippocampal dependent tasks. 4. The self-referential component of episodic memory in humans is mediated by the corollary discharge (the efference copy of the action plan developed by prefrontal cortex and transmitted to hippocampus where it is stored as a working memory; CA1 uses this efference copy to compare the expected consequences of action to the actual consequences of action). 5. Impairments in the production or transmission of this corollary discharge may contribute to some of the symptoms of schizophrenia. Unresolved issues and suggestions for future research are discussed.
Collapse
Affiliation(s)
- Robert Numan
- Department of Psychology, Santa Clara University, Santa Clara, California, United States
| |
Collapse
|
20
|
Hunt LT, Daw ND, Kaanders P, MacIver MA, Mugan U, Procyk E, Redish AD, Russo E, Scholl J, Stachenfeld K, Wilson CRE, Kolling N. Formalizing planning and information search in naturalistic decision-making. Nat Neurosci 2021; 24:1051-1064. [PMID: 34155400 DOI: 10.1038/s41593-021-00866-w] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2020] [Accepted: 03/23/2021] [Indexed: 02/05/2023]
Abstract
Decisions made by mammals and birds are often temporally extended. They require planning and sampling of decision-relevant information. Our understanding of such decision-making remains in its infancy compared with simpler, forced-choice paradigms. However, recent advances in algorithms supporting planning and information search provide a lens through which we can explain neural and behavioral data in these tasks. We review these advances to obtain a clearer understanding for why planning and curiosity originated in certain species but not others; how activity in the medial temporal lobe, prefrontal and cingulate cortices may support these behaviors; and how planning and information search may complement each other as means to improve future action selection.
Collapse
Affiliation(s)
- L T Hunt
- Department of Psychiatry, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| | - N D Daw
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - P Kaanders
- Department of Experimental Psychology, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - M A MacIver
- Center for Robotics and Biosystems, Department of Neurobiology, Department of Biomedical Engineering, Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA
| | - U Mugan
- Center for Robotics and Biosystems, Department of Neurobiology, Department of Biomedical Engineering, Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA
| | - E Procyk
- Univ Lyon, Université Claude Bernard Lyon 1, INSERM, Stem Cell and Brain Research Institute U1208, Bron, France
| | - A D Redish
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - E Russo
- Department of Theoretical Neuroscience, Central Institute of Mental Health, Mannheim, Germany.,Department of Psychiatry and Psychotherapy, University Medical Center, Johannes Gutenberg University, Mainz, Germany
| | - J Scholl
- Department of Experimental Psychology, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | | | - C R E Wilson
- Univ Lyon, Université Claude Bernard Lyon 1, INSERM, Stem Cell and Brain Research Institute U1208, Bron, France
| | - N Kolling
- Department of Psychiatry, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| |
Collapse
|
21
|
Han MJ, Park CU, Kang S, Kim B, Nikolaidis A, Milham MP, Hong SJ, Kim SG, Baeg E. Mapping functional gradients of the striatal circuit using simultaneous microelectric stimulation and ultrahigh-field fMRI in non-human primates. Neuroimage 2021; 236:118077. [PMID: 33878384 DOI: 10.1016/j.neuroimage.2021.118077] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Revised: 03/26/2021] [Accepted: 04/07/2021] [Indexed: 02/07/2023] Open
Abstract
Advances in functional magnetic resonance imaging (fMRI) have significantly enhanced our understanding of the striatal system of both humans and non-human primates (NHP) over the last few decades. However, its circuit-level functional anatomy remains poorly understood, partly because in-vivo fMRI cannot directly perturb a brain system and map its casual input-output relationship. Also, routine 3T fMRI has an insufficient spatial resolution. We performed electrical microstimulation (EM) of the striatum in lightly-anesthetized NHPs while simultaneously mapping whole-brain activation, using contrast-enhanced fMRI at ultra-high-field 7T. By stimulating multiple positions along the striatum's main (dorsal-to-ventral) axis, we revealed its complex functional circuit concerning mutually connected subsystems in both cortical and subcortical areas. Indeed, within the striatum, there were distinct brain activation patterns across different stimulation sites. Specifically, dorsal stimulation revealed a medial-to-lateral elongated shape of activation in upper caudate and putamen areas, whereas ventral stimulation evoked areas confined to the medial and lower caudate. Such dorsoventral gradients also appeared in neocortical and thalamic activations, indicating consistent embedding profiles of the striatal system across the whole brain. These findings reflect different forms of within-circuit and inter-regional neuronal connectivity between the dorsal and ventromedial striatum. These patterns both shared and contrasted with previous anatomical tract-tracing and in-vivo resting-state fMRI studies. Our approach of combining microstimulation and whole-brain fMRI mapping in NHPs provides a unique opportunity to integrate our understanding of a targeted brain area's meso- and macro-scale functional systems.
Collapse
Affiliation(s)
- Min-Jun Han
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon, Republic of Korea; Department of Biomedical Engineering, Sungkyunkwan University, Suwon, Republic of Korea
| | - Chan-Ung Park
- Department of Biomedical Engineering, Sungkyunkwan University, Suwon, Republic of Korea
| | - Sangyun Kang
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon, Republic of Korea
| | - Byounghoon Kim
- Neuroscience, University of Wisconsin - Madison, Madison, WI, United States
| | - Aki Nikolaidis
- Center for the Developing Brain, Child Mind Institute, New York, NY, United States
| | - Michael P Milham
- Center for the Developing Brain, Child Mind Institute, New York, NY, United States; Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, New York, NY, United States
| | - Seok Jun Hong
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon, Republic of Korea; Department of Biomedical Engineering, Sungkyunkwan University, Suwon, Republic of Korea,; Center for the Developing Brain, Child Mind Institute, New York, NY, United States
| | - Seong-Gi Kim
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon, Republic of Korea; Department of Biomedical Engineering, Sungkyunkwan University, Suwon, Republic of Korea,.
| | - Eunha Baeg
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon, Republic of Korea; Department of Biomedical Engineering, Sungkyunkwan University, Suwon, Republic of Korea,.
| |
Collapse
|
22
|
Bennett MS. What Behavioral Abilities Emerged at Key Milestones in Human Brain Evolution? 13 Hypotheses on the 600-Million-Year Phylogenetic History of Human Intelligence. Front Psychol 2021; 12:685853. [PMID: 34393912 PMCID: PMC8358274 DOI: 10.3389/fpsyg.2021.685853] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Accepted: 06/16/2021] [Indexed: 01/24/2023] Open
Abstract
This paper presents 13 hypotheses regarding the specific behavioral abilities that emerged at key milestones during the 600-million-year phylogenetic history from early bilaterians to extant humans. The behavioral, intellectual, and cognitive faculties of humans are complex and varied: we have abilities as diverse as map-based navigation, theory of mind, counterfactual learning, episodic memory, and language. But these faculties, which emerge from the complex human brain, are likely to have evolved from simpler prototypes in the simpler brains of our ancestors. Understanding the order in which behavioral abilities evolved can shed light on how and why our brains evolved. To propose these hypotheses, I review the available data from comparative psychology and evolutionary neuroscience.
Collapse
|
23
|
Weglage M, Wärnberg E, Lazaridis I, Calvigioni D, Tzortzi O, Meletis K. Complete representation of action space and value in all dorsal striatal pathways. Cell Rep 2021; 36:109437. [PMID: 34320355 DOI: 10.1016/j.celrep.2021.109437] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Revised: 02/02/2021] [Accepted: 07/02/2021] [Indexed: 11/17/2022] Open
Abstract
The dorsal striatum plays a central role in the selection, execution, and evaluation of actions. An emerging model attributes action selection to the matrix and evaluation to the striosome compartment. Here, we use large-scale cell-type-specific calcium imaging to determine the activity of striatal projection neurons (SPNs) during motor and decision behaviors in the three major outputs of the dorsomedial striatum: Oprm1+ striosome versus D1+ direct and A2A+ indirect pathway SPNs. We find that Oprm1+ SPNs show complex tunings to simple movements and value-guided actions, which are conserved across many sessions in a single task but remap between contexts. During decision making, the SPN tuning profiles form a complete representation in which sequential SPN activity jointly encodes task progress and value. We propose that the three major output pathways in the dorsomedial striatum share a similarly complete representation of the entire action space, including task- and phase-specific signals of action value and choice.
Collapse
Affiliation(s)
- Moritz Weglage
- Department of Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Emil Wärnberg
- Department of Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Iakovos Lazaridis
- Department of Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Daniela Calvigioni
- Department of Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | - Ourania Tzortzi
- Department of Neuroscience, Karolinska Institutet, 171 77 Stockholm, Sweden
| | | |
Collapse
|
24
|
Huynh T, Alstatt K, Abram SV, Schmitzer-Torbert N. Vicarious Trial-and-Error Is Enhanced During Deliberation in Human Virtual Navigation in a Translational Foraging Task. Front Behav Neurosci 2021; 15:586159. [PMID: 33912018 PMCID: PMC8072010 DOI: 10.3389/fnbeh.2021.586159] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2020] [Accepted: 03/01/2021] [Indexed: 11/18/2022] Open
Abstract
Foraging tasks provide valuable insights into decision-making as animals decide how to allocate limited resources (such as time). In rodents, vicarious trial-and-error (back and forth movements), or VTE, is an important behavioral measure of deliberation which is enhanced early in learning and when animals are presented with difficult decisions. Using new translational versions of a rodent foraging task (the "Movie Row" and "Candy Row"), humans navigated a virtual maze presented on standard computers to obtain rewards (either short videos or candy) offered after a variable delay. Decision latencies were longer when participants were presented with difficult offers, overrode their preferences, and when they accepted an offer after rejecting a previous offer. In these situations, humans showed VTE-like behavior, where they were more likely to pause and/or reorient one or more times before making a decision. Behavior on these tasks replicated previous results from the rodent foraging task ("Restaurant Row") and a human version lacking a navigation component ("Web-Surf") and revealed some species differences. Compared to survey measures of delay-discounting, willingness to wait for rewards in the foraging task was not related to willingness to wait for hypothetical rewards. And, smoking status (use of cigarettes or e-cigarettes) was associated with stronger discounting of hypothetical future rewards, but was not well-related to performance on the foraging tasks. In contrast, individuals with overweight or obese BMI (≥25) did not show stronger delay-discounting, but individuals with BMI ≥ 25, and especially females, showed reduced sensitivity to sunk-costs (where their decisions were less sensitive to irrecoverable investments of effort) and less deliberation when presented with difficult offers. These data indicate that VTE is a behavioral index of deliberation in humans, and further support the Movie and Candy Row as translational tools to study decision-making in humans with the potential to provide novel insights about decision-making that are relevant to public health.
Collapse
Affiliation(s)
- Thach Huynh
- Department of Psychology, Wabash College, Crawfordsville, IN, United States
| | - Keanan Alstatt
- Department of Psychology, Wabash College, Crawfordsville, IN, United States
| | - Samantha V. Abram
- Sierra Pacific Mental Illness Research Education and Clinical Centers, San Francisco Veterans Affairs Medical Center, Veterans Affairs San Francisco Healthcare System, University of California, San Francisco, San Francisco, CA, United States
- Department of Psychiatry, University of California, San Francisco, San Francisco, CA, United States
- Mental Health Service, Veterans Affairs San Francisco Healthcare System, San Francisco, CA, United States
| | | |
Collapse
|
25
|
Shikano Y, Ikegaya Y, Sasaki T. Minute-encoding neurons in hippocampal-striatal circuits. Curr Biol 2021; 31:1438-1449.e6. [PMID: 33545048 DOI: 10.1016/j.cub.2021.01.032] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2020] [Revised: 11/16/2020] [Accepted: 01/11/2021] [Indexed: 11/16/2022]
Abstract
Animals process temporal information in an ever-changing environment, but the neuronal mechanisms of this process, especially on timescales longer than seconds, remain unresolved. Here, we designed a hippocampus-dependent task in which rats prospectively increased their reward-seeking behavior over a duration of minutes. During this timing behavior, hippocampal and striatal neurons represented successive time points on the order of minutes by gradually changing their firing rates and transiently increasing their firing rates at specific time points. These minute-encoding patterns progressively developed as the rats learned a time-reward relationship, and the patterns underwent flexible scaling in parallel with timing behavior. These observations suggest a neuronal basis in the hippocampal-striatal circuits that enables temporal processing and formation of episodic memory on a timescale of minutes.
Collapse
Affiliation(s)
- Yu Shikano
- Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Yuji Ikegaya
- Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan; Center for Information and Neural Networks, 1-4 Yamadaoka, Suita City, Osaka 565-0871, Japan; Institute for AI and Beyond, The University of Tokyo, Tokyo 113-0033, Japan
| | - Takuya Sasaki
- Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan; Precursory Research for Embryonic Science and Technology (PRESTO), Japan Science and Technology Agency (JST), 4-1-8 Honcho, Kawaguchi, Saitama 332-0012, Japan.
| |
Collapse
|
26
|
Peer M, Brunec IK, Newcombe NS, Epstein RA. Structuring Knowledge with Cognitive Maps and Cognitive Graphs. Trends Cogn Sci 2021; 25:37-54. [PMID: 33248898 PMCID: PMC7746605 DOI: 10.1016/j.tics.2020.10.004] [Citation(s) in RCA: 61] [Impact Index Per Article: 20.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2020] [Revised: 10/16/2020] [Accepted: 10/17/2020] [Indexed: 12/21/2022]
Abstract
Humans and animals use mental representations of the spatial structure of the world to navigate. The classical view is that these representations take the form of Euclidean cognitive maps, but alternative theories suggest that they are cognitive graphs consisting of locations connected by paths. We review evidence suggesting that both map-like and graph-like representations exist in the mind/brain that rely on partially overlapping neural systems. Maps and graphs can operate simultaneously or separately, and they may be applied to both spatial and nonspatial knowledge. By providing structural frameworks for complex information, cognitive maps and cognitive graphs may provide fundamental organizing schemata that allow us to navigate in physical, social, and conceptual spaces.
Collapse
Affiliation(s)
- Michael Peer
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Iva K Brunec
- Department of Psychology, Temple University, Philadelphia, PA 19122, USA
| | - Nora S Newcombe
- Department of Psychology, Temple University, Philadelphia, PA 19122, USA
| | - Russell A Epstein
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA.
| |
Collapse
|
27
|
McLaughlin AE, Diehl GW, Redish AD. Potential roles of the rodent medial prefrontal cortex in conflict resolution between multiple decision-making systems. INTERNATIONAL REVIEW OF NEUROBIOLOGY 2020; 158:249-281. [PMID: 33785147 PMCID: PMC8211383 DOI: 10.1016/bs.irn.2020.11.009] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Mammalian decision-making is mediated by the interaction of multiple, neurally and computationally separable decision systems. Having multiple systems requires a mechanism to manage conflict and converge onto the selection of singular actions. A long history of evidence has pointed to the prefrontal cortex as a central component in processing the interactions between distinct decision systems and resolving conflicts among them. In this chapter we review four theories of how that interaction might occur and identify how the medial prefrontal cortex in the rodent may be involved in each theory. We then present experimental predictions implied by the neurobiological data in the context of each theory as a starting point for future investigation of medial prefrontal cortex and decision-making.
Collapse
Affiliation(s)
- Amber E McLaughlin
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, United States
| | - Geoffrey W Diehl
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, United States
| | - A David Redish
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, United States.
| |
Collapse
|
28
|
Abstract
Humans and other animals use multiple strategies for making decisions. Reinforcement-learning theory distinguishes between stimulus-response (model-free; MF) learning and deliberative (model-based; MB) planning. The spatial-navigation literature presents a parallel dichotomy between navigation strategies. In "response learning," associated with the dorsolateral striatum (DLS), decisions are anchored to an egocentric reference frame. In "place learning," associated with the hippocampus, decisions are anchored to an allocentric reference frame. Emerging evidence suggests that the contribution of hippocampus to place learning may also underlie its contribution to MB learning by representing relational structure in a cognitive map. Here, we introduce a computational model in which hippocampus subserves place and MB learning by learning a "successor representation" of relational structure between states; DLS implements model-free response learning by learning associations between actions and egocentric representations of landmarks; and action values from either system are weighted by the reliability of its predictions. We show that this model reproduces a range of seemingly disparate behavioral findings in spatial and nonspatial decision tasks and explains the effects of lesions to DLS and hippocampus on these tasks. Furthermore, modeling place cells as driven by boundaries explains the observation that, unlike navigation guided by landmarks, navigation guided by boundaries is robust to "blocking" by prior state-reward associations due to learned associations between place cells. Our model, originally shaped by detailed constraints in the spatial literature, successfully characterizes the hippocampal-striatal system as a general system for decision making via adaptive combination of stimulus-response learning and the use of a cognitive map.
Collapse
|
29
|
He Q, Beveridge EH, Starnes J, Goodroe SC, Brown TI. Environmental overlap and individual encoding strategy modulate memory interference in spatial navigation. Cognition 2020; 207:104508. [PMID: 33172657 DOI: 10.1016/j.cognition.2020.104508] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2020] [Revised: 10/28/2020] [Accepted: 10/29/2020] [Indexed: 10/23/2022]
Abstract
There has been great interest in how previously acquired knowledge interacts with newly learned knowledge and how prior knowledge facilitates semantic and "schema" learning. In studies of episodic memory, it is broadly associated with interference. Very few studies have examined the balance between interference and facilitation over the course of temporally-extended events and its individual differences. In the present study, we recruited 120 participants for a two-day spatial navigation experiment, wherein participants on Day 2 navigated virtual routes that were learned from Day 1 while also learning new routes. Critically, half of the new mazes overlapped with the old mazes, while the other half did not, enabling us to examine interference and facilitation in the context of spatial episodic learning. Overall, we found that navigation performance in new mazes that overlapped with previously-learned routes was significantly worse than the new non-overlapping mazes, suggesting proactive interference. Interestingly, we found memory facilitation for new routes in familiar environments in locations where there was no direct overlap with the previously-learned routes. Cognitive map accuracy positively correlated with proactive interference. Moreover, participants with high self-report spatial ability and/or a preference for place-based learning experienced more proactive interference. Taken together, our results show that 1) both memory interference and facilitation can co-occur as a function of prior learning, 2) proactive interference within a route varied as a function of the degree of overlap with old knowledge, and 3) individual differences in spatial ability and strategy can modulate proactive interference.
Collapse
Affiliation(s)
- Qiliang He
- School of Psychology, Georgia Institute of Technology, USA
| | | | - Jon Starnes
- School of Psychology, Georgia Institute of Technology, USA
| | - Sarah C Goodroe
- Department of Experimental Psychology, University College London, UK
| | | |
Collapse
|
30
|
Jin N, Paffhausen BH, Duer A, Menzel R. Mushroom Body Extrinsic Neurons in Walking Bumblebees Correlate With Behavioral States but Not With Spatial Parameters During Exploratory Behavior. Front Behav Neurosci 2020; 14:590999. [PMID: 33192371 PMCID: PMC7606933 DOI: 10.3389/fnbeh.2020.590999] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2020] [Accepted: 09/16/2020] [Indexed: 11/21/2022] Open
Abstract
Central place foraging insects like honeybees and bumblebees learn to navigate efficiently between nest and feeding site. Essential components of this behavior can be moved to the laboratory. A major component of navigational learning is the active exploration of the test arena. These conditions have been used here to search for neural correlates of exploratory walking in the central arena (ground), and thigmotactic walking in the periphery (slope). We chose mushroom body extrinsic neurons (MBENs) because of their learning-related plasticity and their multi-modal sensitivities that may code relevant parameters in a brain state-dependent way. Our aim was to test whether MBENs code space-related components or are more involved in state-dependent processes characterizing exploration and thigmotaxis. MBENs did not respond selectively to body directions or locations. Their spiking activity differently correlated with walking speed depending on the animals' locations: on the ground, reflecting exploration, or on the slope, reflecting thigmotaxis. This effect depended on walking speed in different ways for different animals. We then asked whether these effects depended on spatial parameters or on the two states, exploration and thigmotaxis. Significant epochs of stable changes in spiking did not correlate with restricted locations in the arena, body direction, or walking transitions between ground and slope. We thus conclude that the walking speed dependencies are caused by the two states, exploration and thigmotaxis, rather than by spatial parameters.
Collapse
|
31
|
Neural signatures underlying deliberation in human foraging decisions. COGNITIVE AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2020; 19:1492-1508. [PMID: 31209734 DOI: 10.3758/s13415-019-00733-z] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
Humans have a remarkable capacity to mentally project themselves far ahead in time. This ability, which entails the mental simulation of events, is thought to be fundamental to deliberative decision making, as it allows us to search through and evaluate possible choices. Many decisions that humans make are foraging decisions, in which one must decide whether an available offer is worth taking, when compared to unknown future possibilities (i.e., the background). Using a translational decision-making paradigm designed to reveal decision preferences in rats, we found that humans engaged in deliberation when making foraging decisions. A key feature of this task is that preferences (and thus, value) are revealed as a function of serial choices. Like rats, humans also took longer to respond when faced with difficult decisions near their preference boundary, which was associated with prefrontal and hippocampal activation, exemplifying cross-species parallels in deliberation. Furthermore, we found that voxels within the visual cortices encoded neural representations of the available possibilities specifically following regret-inducing experiences, in which the subject had previously rejected a good offer only to encounter a low-valued offer on the subsequent trial.
Collapse
|
32
|
Goldenberg JE, Lentzou S, Ackert-Smith L, Knowlton H, Dash MB. Interindividual differences in memory system local field potential activity predict behavioral strategy on a dual-solution T-maze. Hippocampus 2020; 30:1313-1326. [PMID: 32894595 DOI: 10.1002/hipo.23258] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2020] [Revised: 08/11/2020] [Accepted: 08/13/2020] [Indexed: 12/31/2022]
Abstract
Individuals can use diverse behavioral strategies to navigate their environment including hippocampal-dependent place strategies reliant upon cognitive maps and striatal-dependent response strategies reliant upon egocentric body turns. The existence of multiple memory systems appears to facilitate successful navigation across a wide range of environmental and physiological conditions. The mechanisms by which these systems interact to ultimately generate a unitary behavioral response, however, remain unclear. We trained 20 male, Sprague-Dawley rats on a dual-solution T-maze while simultaneously recording local field potentials that were targeted to the dorsolateral striatum and dorsal hippocampus. Eight rats spontaneously exhibited a place strategy while the remaining 12 rats exhibited a response strategy. Interindividual differences in behavioral strategy were associated with distinct patterns of LFP activity between the dorsolateral striatum and dorsal hippocampus. Specifically, striatal-hippocampal theta activity was in-phase in response rats and out-of-phase in place rats and response rats exhibited elevated striatal-hippocampal coherence across a wide range of frequency bands. These contrasting striatal-hippocampal activity regimes were (a) present during both maze-learning and a 30 min premaze habituation period and (b) could be used to train support vector machines to reliably predict behavioral strategy. Distinct patterns of neuronal activity across multiple memory systems, therefore, appear to bias behavioral strategy selection and thereby contribute to interindividual differences in behavior.
Collapse
Affiliation(s)
| | - Stergiani Lentzou
- Program in Neuroscience, Middlebury College, Middlebury, Vermont, USA
| | - Lyn Ackert-Smith
- Program in Neuroscience, Middlebury College, Middlebury, Vermont, USA
| | - Harrison Knowlton
- Program in Neuroscience, Middlebury College, Middlebury, Vermont, USA
| | - Michael B Dash
- Program in Neuroscience, Middlebury College, Middlebury, Vermont, USA.,Department of Psychology, Middlebury College, Middlebury, Vermont, USA
| |
Collapse
|
33
|
Machine Learning for Neural Decoding. eNeuro 2020; 7:ENEURO.0506-19.2020. [PMID: 32737181 PMCID: PMC7470933 DOI: 10.1523/eneuro.0506-19.2020] [Citation(s) in RCA: 74] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Revised: 07/01/2020] [Accepted: 07/03/2020] [Indexed: 01/11/2023] Open
Abstract
Despite rapid advances in machine learning tools, the majority of neural decoding approaches still use traditional methods. Modern machine learning tools, which are versatile and easy to use, have the potential to significantly improve decoding performance. This tutorial describes how to effectively apply these algorithms for typical decoding problems. We provide descriptions, best practices, and code for applying common machine learning methods, including neural networks and gradient boosting. We also provide detailed comparisons of the performance of various methods at the task of decoding spiking activity in motor cortex, somatosensory cortex, and hippocampus. Modern methods, particularly neural networks and ensembles, significantly outperform traditional approaches, such as Wiener and Kalman filters. Improving the performance of neural decoding algorithms allows neuroscientists to better understand the information contained in a neural population and can help to advance engineering applications such as brain–machine interfaces. Our code package is available at github.com/kordinglab/neural_decoding.
Collapse
|
34
|
Gardner RS, Gold PE, Korol DL. Inactivation of the striatum in aged rats rescues their ability to learn a hippocampus-sensitive spatial navigation task. Neurobiol Learn Mem 2020; 172:107231. [DOI: 10.1016/j.nlm.2020.107231] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2019] [Revised: 02/29/2020] [Accepted: 04/13/2020] [Indexed: 01/23/2023]
|
35
|
Huang Y, Yaple ZA, Yu R. Goal-oriented and habitual decisions: Neural signatures of model-based and model-free learning. Neuroimage 2020; 215:116834. [PMID: 32283275 DOI: 10.1016/j.neuroimage.2020.116834] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2019] [Revised: 03/03/2020] [Accepted: 04/08/2020] [Indexed: 11/26/2022] Open
Abstract
Human decision-making is mainly driven by two fundamental learning processes: a slow, deliberative, goal-directed model-based process that maps out the potential outcomes of all options and a rapid habitual model-free process that enables reflexive repetition of previously successful choices. Although many model-informed neuroimaging studies have examined the neural correlates of model-based and model-free learning, the concordant activity among these two processes remains unclear. We used quantitative meta-analyses of functional magnetic resonance imaging experiments to identify the concordant activity pertaining to model-based and model-free learning over a range of reward-related paradigms. We found that: 1) both processes yielded concordant ventral striatum activity, 2) model-based learning activated the medial prefrontal cortex and orbital frontal cortex, and 3) model-free learning specifically activated the left globus pallidus and right caudate head. Our findings suggest that model-free and model-based decision making engage overlapping yet distinct neural regions. These stereotaxic maps improve our understanding of how deliberative goal-directed and reflexive habitual learning are implemented in the brain.
Collapse
Affiliation(s)
- Yi Huang
- NUS Graduate School for Integrative Sciences and Engineering, National University of Singapore, Singapore
| | - Zachary A Yaple
- Department of Psychology, National University of Singapore, Singapore
| | - Rongjun Yu
- NUS Graduate School for Integrative Sciences and Engineering, National University of Singapore, Singapore; Department of Psychology, National University of Singapore, Singapore.
| |
Collapse
|
36
|
Gahnstrom CJ, Spiers HJ. Striatal and hippocampal contributions to flexible navigation in rats and humans. Brain Neurosci Adv 2020; 4:2398212820979772. [PMID: 33426302 PMCID: PMC7755934 DOI: 10.1177/2398212820979772] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2020] [Accepted: 11/16/2020] [Indexed: 12/13/2022] Open
Abstract
The hippocampus has been firmly established as playing a crucial role in flexible navigation. Recent evidence suggests that dorsal striatum may also play an important role in such goal-directed behaviour in both rodents and humans. Across recent studies, activity in the caudate nucleus has been linked to forward planning and adaptation to changes in the environment. In particular, several human neuroimaging studies have found the caudate nucleus tracks information traditionally associated with that by the hippocampus. In this brief review, we examine this evidence and argue the dorsal striatum encodes the transition structure of the environment during flexible, goal-directed behaviour. We highlight that future research should explore the following: (1) Investigate neural responses during spatial navigation via a biophysically plausible framework explained by reinforcement learning models and (2) Observe the interaction between cortical areas and both the dorsal striatum and hippocampus during flexible navigation.
Collapse
Affiliation(s)
- Christoffer J. Gahnstrom
- Institute of Behavioural Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Hugo J. Spiers
- Institute of Behavioural Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| |
Collapse
|
37
|
Sosa M, Joo HR, Frank LM. Dorsal and Ventral Hippocampal Sharp-Wave Ripples Activate Distinct Nucleus Accumbens Networks. Neuron 2019; 105:725-741.e8. [PMID: 31864947 DOI: 10.1016/j.neuron.2019.11.022] [Citation(s) in RCA: 58] [Impact Index Per Article: 11.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Revised: 10/08/2019] [Accepted: 11/15/2019] [Indexed: 12/31/2022]
Abstract
Memories of positive experiences link places, events, and reward outcomes. These memories recruit interactions between the hippocampus and nucleus accumbens (NAc). Both dorsal and ventral hippocampus (dH and vH) project to the NAc, but it remains unknown whether dH and vH act in concert or separately to engage NAc representations related to space and reward. We recorded simultaneously from the dH, vH, and NAc of rats during an appetitive spatial task and focused on hippocampal sharp-wave ripples (SWRs) to identify times of memory reactivation across brain regions. Here, we show that dH and vH awake SWRs occur asynchronously and activate distinct and opposing patterns of NAc spiking. Only NAc neurons activated during dH SWRs were tuned to task- and reward-related information. These temporally and anatomically separable hippocampal-NAc interactions point to distinct channels of mnemonic processing in the NAc, with the dH-NAc channel specialized for spatial task and reward information. VIDEO ABSTRACT.
Collapse
Affiliation(s)
- Marielena Sosa
- Neuroscience Graduate Program, Kavli Institute for Fundamental Neuroscience, and Department of Physiology, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Hannah R Joo
- Neuroscience Graduate Program, Kavli Institute for Fundamental Neuroscience, and Department of Physiology, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Loren M Frank
- Neuroscience Graduate Program, Kavli Institute for Fundamental Neuroscience, and Department of Physiology, University of California, San Francisco, San Francisco, CA 94158, USA; Howard Hughes Medical Institute, San Francisco, CA 94158, USA.
| |
Collapse
|
38
|
Freedberg M, Toader AC, Wassermann EM, Voss JL. Competitive and cooperative interactions between medial temporal and striatal learning systems. Neuropsychologia 2019; 136:107257. [PMID: 31733236 DOI: 10.1016/j.neuropsychologia.2019.107257] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2018] [Revised: 09/13/2019] [Accepted: 11/06/2019] [Indexed: 01/20/2023]
Abstract
The striatum and medial temporal lobes (MTL) exhibit dissociable roles during learning. Whereas the striatum and its network of thalamic relays and cortical nodes are necessary for nondeclarative learning, the MTL and associated network are required for declarative learning. Several studies have suggested that these networks are functionally competitive during learning. Since these discoveries, however, evidence has accumulated that they can operate in a cooperative fashion. In this review, we discuss evidence for both competition and cooperation between these systems during learning, with the aim of reconciling these seemingly contradictory findings. Examples of cooperation between the striatum and MTL have been provided, especially during consolidation and generalization of knowledge, and do not appear to be precluded by differences in functional specialization. However, whether these systems cooperate or compete does seem to depend on the phase of learning and cognitive or motor aspects of the task. The involvement of other regions, such as midbrain dopaminergic nuclei and the prefrontal cortex, may promote and mediate cooperation between the striatum and the MTL during learning. Building on this body of research, we propose a model for striatum-MTL interactions in learning and memory and attempt to predict, in general terms, when cooperation or competition will occur.
Collapse
Affiliation(s)
- Michael Freedberg
- National Institute of Neurological Disorders and Stroke, 9000 Rockville Pike, 10 Center Drive, Bethesda, MD 20892, USA; Henry M. Jackson Foundation for the Advancement of Military Medicine, 6720A Rockledge Drive, Bethesda, MD 20892, USA.
| | - Andrew C Toader
- National Institute of Neurological Disorders and Stroke, 9000 Rockville Pike, 10 Center Drive, Bethesda, MD 20892, USA; Weill Cornell/Rockefeller/Sloan-Kettering Tri-Institutional MD-PhD Program, New York, NY 20892, USA.
| | - Eric M Wassermann
- National Institute of Neurological Disorders and Stroke, 9000 Rockville Pike, 10 Center Drive, Bethesda, MD 20892, USA.
| | - Joel L Voss
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA; Ken and Ruth Davee Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA; Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA; Northwestern University Interdepartmental Neuroscience Program, Northwestern University, Chicago, IL, USA.
| |
Collapse
|
39
|
Rusu SI, Pennartz CMA. Learning, memory and consolidation mechanisms for behavioral control in hierarchically organized cortico-basal ganglia systems. Hippocampus 2019; 30:73-98. [PMID: 31617622 PMCID: PMC6972576 DOI: 10.1002/hipo.23167] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2018] [Revised: 09/09/2019] [Accepted: 09/11/2019] [Indexed: 01/05/2023]
Abstract
This article aims to provide a synthesis on the question how brain structures cooperate to accomplish hierarchically organized behaviors, characterized by low‐level, habitual routines nested in larger sequences of planned, goal‐directed behavior. The functioning of a connected set of brain structures—prefrontal cortex, hippocampus, striatum, and dopaminergic mesencephalon—is reviewed in relation to two important distinctions: (a) goal‐directed as opposed to habitual behavior and (b) model‐based and model‐free learning. Recent evidence indicates that the orbitomedial prefrontal cortices not only subserve goal‐directed behavior and model‐based learning, but also code the “landscape” (task space) of behaviorally relevant variables. While the hippocampus stands out for its role in coding and memorizing world state representations, it is argued to function in model‐based learning but is not required for coding of action–outcome contingencies, illustrating that goal‐directed behavior is not congruent with model‐based learning. While the dorsolateral and dorsomedial striatum largely conform to the dichotomy between habitual versus goal‐directed behavior, ventral striatal functions go beyond this distinction. Next, we contextualize findings on coding of reward‐prediction errors by ventral tegmental dopamine neurons to suggest a broader role of mesencephalic dopamine cells, viz. in behavioral reactivity and signaling unexpected sensory changes. We hypothesize that goal‐directed behavior is hierarchically organized in interconnected cortico‐basal ganglia loops, where a limbic‐affective prefrontal‐ventral striatal loop controls action selection in a dorsomedial prefrontal–striatal loop, which in turn regulates activity in sensorimotor‐dorsolateral striatal circuits. This structure for behavioral organization requires alignment with mechanisms for memory formation and consolidation. We propose that frontal corticothalamic circuits form a high‐level loop for memory processing that initiates and temporally organizes nested activities in lower‐level loops, including the hippocampus and the ripple‐associated replay it generates. The evidence on hierarchically organized behavior converges with that on consolidation mechanisms in suggesting a frontal‐to‐caudal directionality in processing control.
Collapse
Affiliation(s)
- Silviu I Rusu
- Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, The Netherlands.,Research Priority Program Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
| | - Cyriel M A Pennartz
- Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, The Netherlands.,Research Priority Program Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
40
|
Distributed Encoding of Reinforcement in Rat Cortico-Striatal-Limbic Networks. Neuroscience 2019; 413:169-182. [PMID: 31229632 DOI: 10.1016/j.neuroscience.2019.06.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2019] [Revised: 05/24/2019] [Accepted: 06/12/2019] [Indexed: 11/22/2022]
Abstract
Decision-making in the mammalian brain typically involves multiple brain structures within the midbrain, thalamus, striatum, limbic system, and cortex. Although task specific contributions of each brain region have been identified, neurons responding to reinforcement have been found throughout these structures. We sought to determine if any brain area, or cluster of areas, are the source of information, and if the fidelity of information varies among the areas. We recorded simultaneous field potentials (FPs) in rats from seven brain regions as they completed a binary choice task. The FPs of a 0.5 s window following reinforcement were given as input to a classifier that attempted to predict whether or not the rat received reward on each trial. The classifier correctly categorized reward on 77% of trials. Any region-specific signal could be omitted without lowering accuracy. Frequencies above 40 Hz and signals recorded later than 0.25 s following reinforcement were necessary to achieve this accuracy. Further, the classifier was able to predict reinforcement outcome above chance levels when using FPs from any single recorded brain region. Some combinations of structures, however, were more predictive than others. Analysis of FPs prior to reward revealed most regions reflected the prior probability of reward. Lastly, analyses of information flow suggested reinforcement information does not originate within a single structure of the network, within the resolution afforded by FP recordings. These data suggest reward delivery information is rapidly distributed non-uniformly across the network, and there is no canonical flow of information about reward events in the recorded structures.
Collapse
|
41
|
Above and beyond the concrete: The diverse representational substrates of the predictive brain. Behav Brain Sci 2019; 43:e121. [PMID: 31317839 DOI: 10.1017/s0140525x19002000] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
In recent years, scientists have increasingly taken to investigate the predictive nature of cognition. We argue that prediction relies on abstraction, and thus theories of predictive cognition need an explicit theory of abstract representation. We propose such a theory of the abstract representational capacities that allow humans to transcend the "here-and-now." Consistent with the predictive cognition literature, we suggest that the representational substrates of the mind are built as a hierarchy, ranging from the concrete to the abstract; however, we argue that there are qualitative differences between elements along this hierarchy, generating meaningful, often unacknowledged, diversity. Echoing views from philosophy, we suggest that the representational hierarchy can be parsed into: modality-specific representations, instantiated on perceptual similarity; multimodal representations, instantiated primarily on the discovery of spatiotemporal contiguity; and categorical representations, instantiated primarily on social interaction. These elements serve as the building blocks of complex structures discussed in cognitive psychology (e.g., episodes, scripts) and are the inputs for mental representations that behave like functions, typically discussed in linguistics (i.e., predicators). We support our argument for representational diversity by explaining how the elements in our ontology are all required to account for humans' predictive cognition (e.g., in subserving logic-based prediction; in optimizing the trade-off between accurate and detailed predictions) and by examining how the neuroscientific evidence coheres with our account. In doing so, we provide a testable model of the neural bases of conceptual cognition and highlight several important implications to research on self-projection, reinforcement learning, and predictive-processing models of psychopathology.
Collapse
|
42
|
Cazin N, Llofriu Alonso M, Scleidorovich Chiodi P, Pelc T, Harland B, Weitzenfeld A, Fellous JM, Dominey PF. Reservoir computing model of prefrontal cortex creates novel combinations of previous navigation sequences from hippocampal place-cell replay with spatial reward propagation. PLoS Comput Biol 2019; 15:e1006624. [PMID: 31306421 PMCID: PMC6668845 DOI: 10.1371/journal.pcbi.1006624] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2018] [Revised: 07/31/2019] [Accepted: 06/11/2019] [Indexed: 12/03/2022] Open
Abstract
As rats learn to search for multiple sources of food or water in a complex environment, they generate increasingly efficient trajectories between reward sites. Such spatial navigation capacity involves the replay of hippocampal place-cells during awake states, generating small sequences of spatially related place-cell activity that we call “snippets”. These snippets occur primarily during sharp-wave-ripples (SWRs). Here we focus on the role of such replay events, as the animal is learning a traveling salesperson task (TSP) across multiple trials. We hypothesize that snippet replay generates synthetic data that can substantially expand and restructure the experience available and make learning more optimal. We developed a model of snippet generation that is modulated by reward, propagated in the forward and reverse directions. This implements a form of spatial credit assignment for reinforcement learning. We use a biologically motivated computational framework known as ‘reservoir computing’ to model prefrontal cortex (PFC) in sequence learning, in which large pools of prewired neural elements process information dynamically through reverberations. This PFC model consolidates snippets into larger spatial sequences that may be later recalled by subsets of the original sequences. Our simulation experiments provide neurophysiological explanations for two pertinent observations related to navigation. Reward modulation allows the system to reject non-optimal segments of experienced trajectories, and reverse replay allows the system to “learn” trajectories that it has not physically experienced, both of which significantly contribute to the TSP behavior. As rats search for multiple sources of food in a complex environment, they generate increasingly efficient trajectories between reward sites, across multiple trials. This spatial navigation optimization behavior can be measured in the laboratory using a traveling salesperson task (TSP). This likely involves the coordinated replay of place-cell “snippets” between successive trials. We hypothesize that “snippets” can be used by the prefrontal cortex (PFC) to implement a form of reward-modulated reinforcement learning. Our simulation experiments provide neurophysiological explanations for two pertinent observations related to navigation. Reward modulation allows the system to reject non-optimal segments of experienced trajectories, and reverse replay allows the system to “learn” trajectories that it has not physically experienced, both of which significantly contribute to the TSP behavior.
Collapse
Affiliation(s)
- Nicolas Cazin
- INSERM, U1093, Cognition Action Plasticité Sensorimotrice, Université de Bourgogne, Dijon, France
- Robot Cognition Laboratory, Institut Marey, INSERM, CNRS, UBFC, Dijon, France
| | - Martin Llofriu Alonso
- Department of Computer Science and Engineering, University of South Florida, Tampa, Florida, United States of America
| | - Pablo Scleidorovich Chiodi
- Department of Computer Science and Engineering, University of South Florida, Tampa, Florida, United States of America
| | - Tatiana Pelc
- Department of Psychology, University of Arizona, Tucson, Arizona, United States of America
| | - Bruce Harland
- Department of Psychology, University of Arizona, Tucson, Arizona, United States of America
| | - Alfredo Weitzenfeld
- Department of Computer Science and Engineering, University of South Florida, Tampa, Florida, United States of America
| | - Jean-Marc Fellous
- Department of Psychology, University of Arizona, Tucson, Arizona, United States of America
| | - Peter Ford Dominey
- INSERM, U1093, Cognition Action Plasticité Sensorimotrice, Université de Bourgogne, Dijon, France
- Robot Cognition Laboratory, Institut Marey, INSERM, CNRS, UBFC, Dijon, France
- * E-mail:
| |
Collapse
|
43
|
Neuronal representation of environmental boundaries in egocentric coordinates. Nat Commun 2019; 10:2772. [PMID: 31235693 PMCID: PMC6591168 DOI: 10.1038/s41467-019-10722-y] [Citation(s) in RCA: 52] [Impact Index Per Article: 10.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2019] [Accepted: 05/26/2019] [Indexed: 11/08/2022] Open
Abstract
Movement through space is a fundamental behavior for all animals. Cognitive maps of environments are encoded in the hippocampal formation in an allocentric reference frame, but motor movements that comprise physical navigation are represented within an egocentric reference frame. Allocentric navigational plans must be converted to an egocentric reference frame prior to implementation as overt behavior. Here we describe an egocentric spatial representation of environmental boundaries in the dorsomedial striatum. The hippocampus represents an allocentric map of space, however, motor movements used for navigation are defined in an egocentric framework. Here, the authors report that dorsomedial striatal neurons exhibit an egocentric representation of the boundaries in the environment.
Collapse
|
44
|
Li AWY, King J. Spatial memory and navigation in ageing: A systematic review of MRI and fMRI studies in healthy participants. Neurosci Biobehav Rev 2019; 103:33-49. [PMID: 31129234 DOI: 10.1016/j.neubiorev.2019.05.005] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2018] [Revised: 05/02/2019] [Accepted: 05/08/2019] [Indexed: 01/04/2023]
Abstract
AIM Spatial deficits are widely observed in normal ageing and early Alzheimer's disease. This review systematically examined neuroimaging evidence for structural and functional differences in the hippocampus (HC) associated with non-pathological age-related changes in allocentric spatial abilities. METHODS Databases were searched to identify peer-reviewed studies on allocentric spatial processing in normal ageing including MRI or fMRI data. 15 eligible studies were reviewed after applying exclusion criteria and quality assessment. RESULTS There was a marked deficit in allocentric spatial processing and trend towards egocentric strategies in older adults when compared to young controls or across the lifespan, associated in the majority of studies with HC volumetric changes, metabolic or microstructural indicators, and underactivity. A few studies reported no significant correlations. CONCLUSION Findings confirm literature supporting an age-related allocentric spatial processing deficit and a shift towards egocentric strategies. A majority of studies implicated HC atrophy, microstructural/metabolic alterations or functional changes in age-related allocentric spatial impairment. More sensitive imaging techniques and ecologically valid spatial tasks are needed to detect subtle changes in the HC and brain's navigational network.
Collapse
Affiliation(s)
- Adrienne W Y Li
- Department of Clinical, Educational and Health Psychology, University College London, Gower Street, London WC1E 6BT, United Kingdom.
| | - John King
- Department of Clinical, Educational and Health Psychology, University College London, Gower Street, London WC1E 6BT, United Kingdom
| |
Collapse
|
45
|
Bellmund JLS, Gärdenfors P, Moser EI, Doeller CF. Navigating cognition: Spatial codes for human thinking. Science 2019; 362:362/6415/eaat6766. [PMID: 30409861 DOI: 10.1126/science.aat6766] [Citation(s) in RCA: 234] [Impact Index Per Article: 46.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
The hippocampal formation has long been suggested to underlie both memory formation and spatial navigation. We discuss how neural mechanisms identified in spatial navigation research operate across information domains to support a wide spectrum of cognitive functions. In our framework, place and grid cell population codes provide a representational format to map variable dimensions of cognitive spaces. This highly dynamic mapping system enables rapid reorganization of codes through remapping between orthogonal representations across behavioral contexts, yielding a multitude of stable cognitive spaces at different resolutions and hierarchical levels. Action sequences result in trajectories through cognitive space, which can be simulated via sequential coding in the hippocampus. In this way, the spatial representational format of the hippocampal formation has the capacity to support flexible cognition and behavior.
Collapse
Affiliation(s)
- Jacob L S Bellmund
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, NTNU, Norwegian University of Science and Technology, Trondheim, Norway. .,Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands.,Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Peter Gärdenfors
- Department of Philosophy and Cognitive Science, Lund University, Lund, Sweden.,Centre for Artificial Intelligence, University of Technology Sydney, Sydney, Australia
| | - Edvard I Moser
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, NTNU, Norwegian University of Science and Technology, Trondheim, Norway
| | - Christian F Doeller
- Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, NTNU, Norwegian University of Science and Technology, Trondheim, Norway. .,Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| |
Collapse
|
46
|
Marquardt K, Josey M, Kenton JA, Cavanagh JF, Holmes A, Brigman JL. Impaired cognitive flexibility following NMDAR-GluN2B deletion is associated with altered orbitofrontal-striatal function. Neuroscience 2019; 404:338-352. [PMID: 30742964 PMCID: PMC6455963 DOI: 10.1016/j.neuroscience.2019.01.066] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2018] [Revised: 01/18/2019] [Accepted: 01/31/2019] [Indexed: 02/02/2023]
Abstract
A common feature across neuropsychiatric disorders is inability to discontinue an action or thought once it has become detrimental. Reversal learning, a hallmark of executive control, requires plasticity within cortical, striatal and limbic circuits and is highly sensitive to disruption of N-methyl-D-aspartate receptor (NMDAR) function. In particular, selective deletion or antagonism of GluN2B containing NMDARs in cortical regions including the orbitofrontal cortex (OFC), promotes maladaptive perseveration. It remains unknown whether GluN2B functions to maintain local cortical activity necessary for reversal learning, or if it exerts a broader influence on the integration of neural activity across cortical and subcortical systems. To address this question, we utilized in vivo electrophysiology to record neuronal activity and local field potentials (LFP) in the orbitofrontal cortex and dorsal striatum (dS) of mice with deletion of GluN2B in neocortical and hippocampal principal cells while they performed touchscreen reversal learning. Reversal impairment produced by corticohippocampal GluN2B deletion was paralleled by an aberrant increase in functional connectivity between the OFC and dS. These alterations in coordination were associated with alterations in local OFC and dS firing activity. These data demonstrate highly dynamic patterns of cortical and striatal activity concomitant with reversal learning, and reveal GluN2B as a molecular mechanism underpinning the timing of these processes.
Collapse
Affiliation(s)
- Kristin Marquardt
- Department of Neurosciences, University of New, Mexico, School of Medicine, Albuquerque, NM
| | - Megan Josey
- Department of Neurosciences, University of New, Mexico, School of Medicine, Albuquerque, NM
| | - Johnny A Kenton
- Department of Neurosciences, University of New, Mexico, School of Medicine, Albuquerque, NM
| | | | - Andrew Holmes
- Laboratory of Behavioral and Genomic Neuroscience, National Institute on Alcohol Abuse and Alcoholism, Bethesda, Maryland, USA
| | - Jonathan L Brigman
- Department of Neurosciences, University of New, Mexico, School of Medicine, Albuquerque, NM; New, Mexico, Alcohol Research Center, UNM Health Sciences Center, Albuquerque, NM.
| |
Collapse
|
47
|
Bozzatello P, Morese R, Valentini MC, Rocca P, Bosco F, Bellino S. Autobiographical memories, identity disturbance and brain functioning in patients with borderline personality disorder: An fMRI study. Heliyon 2019; 5:e01323. [PMID: 30949597 PMCID: PMC6430005 DOI: 10.1016/j.heliyon.2019.e01323] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2018] [Revised: 01/21/2019] [Accepted: 03/04/2019] [Indexed: 11/30/2022] Open
Abstract
Identity disturbance is a core feature of borderline personality disorder (BPD). Autobiographical memory is a process of reflective thinking through which we form links between elements of life and self. It can be considered as an indirect index of identity integration. The present study was aimed to investigate the differences in brain activity patterns between BPD patients with identity diffusion and healthy controls using fMRI. We enrolled 24 BPD patients and 24 healthy controls. Identity integration in patients and controls was assessed with the Identity Disturbance Questionnaire (IDQ) score and was significantly different (p = 0.001). We analysed hemodynamic response in the regions of interest during presentation of resolved and unresolved life events. With reference to the condition "resolved", increased cerebral activity in right anterior cingulate cortex (ACC), right medial prefrontal cortex (MPFC), right dorsolateral prefrontal cortex (DLPFC), and bilateral insula was registered in BPD patients compared with controls. In the condition "unresolved", increased brain activity was observed in patients in bilateral ACC, bilateral DLPFC, and right temporo-parietal junction. Hyperactivity in ACC and DLPFC in BPD patients with both conditions (resolved and unresolved contexts) may be due to an inefficient attempt to reconstruct a coherent narrative of life events (resolved or not).
Collapse
Affiliation(s)
- Paola Bozzatello
- Centre for Personality Disorders, Department of Neuroscience, University of Turin, Turin, Italy
| | - Rosalba Morese
- Department of Psychology, University of Turin, Italy.,Faculty of Communication Sciences, Università della Svizzera Italiana, Lugano, Switzerland
| | | | - Paola Rocca
- Department of Neuroscience, University of Turin, Turin, Italy.,Neuroscience Institute of Turin, Italy
| | - Francesca Bosco
- Department of Psychology, University of Turin, Italy.,Neuroscience Institute of Turin, Italy
| | - Silvio Bellino
- Centre for Personality Disorders, Department of Neuroscience, University of Turin, Turin, Italy.,Neuroscience Institute of Turin, Italy
| |
Collapse
|
48
|
Bourboulou R, Marti G, Michon FX, El Feghaly E, Nouguier M, Robbe D, Koenig J, Epsztein J. Dynamic control of hippocampal spatial coding resolution by local visual cues. eLife 2019; 8:44487. [PMID: 30822270 PMCID: PMC6397000 DOI: 10.7554/elife.44487] [Citation(s) in RCA: 38] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2018] [Accepted: 02/04/2019] [Indexed: 12/14/2022] Open
Abstract
The ability to flexibly navigate an environment relies on a hippocampal-dependent cognitive map. External space can be internally mapped at different spatial resolutions. However, whether hippocampal spatial coding resolution can rapidly adapt to local features of an environment remains unclear. To explore this possibility, we recorded the firing of hippocampal neurons in mice navigating virtual reality environments, embedding or not local visual cues (virtual 3D objects) in specific locations. Virtual objects enhanced spatial coding resolution in their vicinity with a higher proportion of place cells, smaller place fields, increased spatial selectivity and stability. This effect was highly dynamic upon objects manipulations. Objects also improved temporal coding resolution through improved theta phase precession and theta timescale spike coordination. We propose that the fast adaptation of hippocampal spatial coding resolution to local features of an environment could be relevant for large-scale navigation.
Collapse
Affiliation(s)
- Romain Bourboulou
- Institute of Neurobiology of the Mediterranean Sea (INMED), Turing Center for Living Systems, Aix-Marseille Université, INSERM, Marseille, France
| | - Geoffrey Marti
- Institute of Neurobiology of the Mediterranean Sea (INMED), Turing Center for Living Systems, Aix-Marseille Université, INSERM, Marseille, France
| | - François-Xavier Michon
- Institute of Neurobiology of the Mediterranean Sea (INMED), Turing Center for Living Systems, Aix-Marseille Université, INSERM, Marseille, France
| | - Elissa El Feghaly
- Institute of Neurobiology of the Mediterranean Sea (INMED), Turing Center for Living Systems, Aix-Marseille Université, INSERM, Marseille, France
| | - Morgane Nouguier
- Institute of Neurobiology of the Mediterranean Sea (INMED), Turing Center for Living Systems, Aix-Marseille Université, INSERM, Marseille, France
| | - David Robbe
- Institute of Neurobiology of the Mediterranean Sea (INMED), Turing Center for Living Systems, Aix-Marseille Université, INSERM, Marseille, France
| | - Julie Koenig
- Institute of Neurobiology of the Mediterranean Sea (INMED), Turing Center for Living Systems, Aix-Marseille Université, INSERM, Marseille, France.,Institut Universitaire de France, Paris, France
| | - Jerome Epsztein
- Institute of Neurobiology of the Mediterranean Sea (INMED), Turing Center for Living Systems, Aix-Marseille Université, INSERM, Marseille, France
| |
Collapse
|
49
|
The neuroanatomical correlates of individual differences in delay discounting: A voxel-based morphometry study. JOURNAL OF PACIFIC RIM PSYCHOLOGY 2019. [DOI: 10.1017/prp.2019.22] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Abstract
Delay discounting refers to the reduction in the present value of a future reward as the delay to that reward increases, which is related to various problematic behaviors, such as substance abuse. In this study, we explored the neuroanatomical correlates of delay discounting by employing voxel-based morphometry and the individual difference approach. We found that participants’ delay discounting, measured by the Monetary Choice Questionnaire, was correlated with the gray matter volume (GMV) of two cortical regions. On the one hand, individuals with a larger GMV of the orbitofrontal cortex (OFC) were likely to discount future values less steeply and choose large but delayed rewards. On the other hand, individuals with a larger GMV of the anterior cingulate cortex (ACC) are likely to discount the future value more steeply and prefer small but immediate rewards. Our study revealed the neuroanatomical correlates of delay discounting across the whole brain, and may help to understand the delay discounting in the frame of the hot versus cool system, which demonstrates the dynamics of resisting present temptation for future rewards.
Collapse
|
50
|
Cortico-Striatal Cross-Frequency Coupling and Gamma Genesis Disruptions in Huntington's Disease Mouse and Computational Models. eNeuro 2018; 5:eN-NWR-0210-18. [PMID: 30627632 PMCID: PMC6325534 DOI: 10.1523/eneuro.0210-18.2018] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2018] [Revised: 10/19/2018] [Accepted: 11/12/2018] [Indexed: 12/16/2022] Open
Abstract
Abnormal gamma band power across cortex and striatum is an important phenotype of Huntington's disease (HD) in both patients and animal models, but neither the origin nor the functional relevance of this phenotype is well understood. Here, we analyzed local field potential (LFP) activity in freely behaving, symptomatic R6/2 and Q175 mouse models and corresponding wild-type (WT) controls. We focused on periods of quiet rest, which show strong γ activity in HD mice. Simultaneous recording from motor cortex and its target area in dorsal striatum in the R6/2 model revealed exaggerated functional coupling over that observed in WT between the phase of delta frequencies (1-4 Hz) in cortex and striatum and striatal amplitude modulation of low γ frequencies (25-55 Hz; i.e., phase-amplitude coupling, PAC), but no evidence that abnormal cortical activity alone can account for the increase in striatal γ power. Both HD mouse models had stronger coupling of γ amplitude to δ phase and more unimodal phase distributions than their WT counterparts. To assess the possible role of striatal fast-spiking interneurons (FSIs) in these phenomena, we developed a computational model based on additional striatal recordings from Q175 mice. Changes in peak γ frequency and power ratio were readily reproduced by our computational model, accounting for several experimental findings reported in the literature. Our results suggest that HD is characterized by both a reorganization of cortico-striatal drive and specific population changes related to intrastriatal synaptic coupling.
Collapse
|