1
|
Parra-Barrero E, Vijayabaskaran S, Seabrook E, Wiskott L, Cheng S. A map of spatial navigation for neuroscience. Neurosci Biobehav Rev 2023; 152:105200. [PMID: 37178943 DOI: 10.1016/j.neubiorev.2023.105200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Revised: 04/13/2023] [Accepted: 04/24/2023] [Indexed: 05/15/2023]
Abstract
Spatial navigation has received much attention from neuroscientists, leading to the identification of key brain areas and the discovery of numerous spatially selective cells. Despite this progress, our understanding of how the pieces fit together to drive behavior is generally lacking. We argue that this is partly caused by insufficient communication between behavioral and neuroscientific researchers. This has led the latter to under-appreciate the relevance and complexity of spatial behavior, and to focus too narrowly on characterizing neural representations of space-disconnected from the computations these representations are meant to enable. We therefore propose a taxonomy of navigation processes in mammals that can serve as a common framework for structuring and facilitating interdisciplinary research in the field. Using the taxonomy as a guide, we review behavioral and neural studies of spatial navigation. In doing so, we validate the taxonomy and showcase its usefulness in identifying potential issues with common experimental approaches, designing experiments that adequately target particular behaviors, correctly interpreting neural activity, and pointing to new avenues of research.
Collapse
Affiliation(s)
- Eloy Parra-Barrero
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany; International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| | - Sandhiya Vijayabaskaran
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany
| | - Eddie Seabrook
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany
| | - Laurenz Wiskott
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany; International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| | - Sen Cheng
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany; International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany.
| |
Collapse
|
2
|
Levcik D, Sugi AH, Aguilar-Rivera M, Pochapski JA, Baltazar G, Pulido LN, Villas-Boas CA, Fuentes-Flores R, Nicola SM, Da Cunha C. Nucleus Accumbens Shell Neurons Encode the Kinematics of Reward Approach Locomotion. Neuroscience 2023; 524:181-196. [PMID: 37330195 PMCID: PMC10527230 DOI: 10.1016/j.neuroscience.2023.06.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 05/30/2023] [Accepted: 06/02/2023] [Indexed: 06/19/2023]
Abstract
The nucleus accumbens (NAc) is considered an interface between motivation and action, with NAc neurons playing an important role in promoting reward approach. However, the encoding by NAc neurons that contributes to this role remains unknown. We recorded 62 NAc neurons in male Wistar rats (n = 5) running towards rewarded locations in an 8-arm radial maze. Variables related to locomotor approach kinematics were the best predictors of the firing rate for most NAc neurons. Nearly 18% of the recorded neurons were inhibited during the entire approach run (locomotion-off cells), suggesting that reduction in firing of these neurons promotes initiation of locomotor approach. 27% of the neurons presented a peak of activity during acceleration followed by a valley during deceleration (acceleration-on cells). Together, these neurons accounted for most of the speed and acceleration encoding identified in our analysis. In contrast, a further 16% of neurons presented a valley during acceleration followed by a peak just prior to or after reaching reward (deceleration-on cells). These findings suggest that these three classes of NAc neurons influence the time course of speed changes during locomotor approach to reward.
Collapse
Affiliation(s)
- David Levcik
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Institute of Physiology of the Czech Academy of Sciences, Videnska 1083, 142 20 Prague, Czech Republic
| | - Adam H Sugi
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil; Department of Biochemistry, Universidade Federal do Paraná, Curitiba, Brazil
| | - Marcelo Aguilar-Rivera
- Department of Bioengineering, University of California, 9500 Gilman Drive MC 0412, La Jolla, San Diego 92093, USA
| | - José A Pochapski
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil
| | - Gabriel Baltazar
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil; Department of Biochemistry, Universidade Federal do Paraná, Curitiba, Brazil
| | - Laura N Pulido
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil
| | - Cyrus A Villas-Boas
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil
| | - Romulo Fuentes-Flores
- Departamento de Neurociencia, Facultad de Medicina, Universidad de Chile, Av. Independencia 1027, Independencia 8380453, Santiago, Chile
| | - Saleem M Nicola
- Department of Neuroscience, Albert Einstein College of Medicine, 1300 Morris Park Ave, Bronx, NY 10461, USA; Department of Psychiatry, Albert Einstein College of Medicine, New York, USA
| | - Claudio Da Cunha
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil; Department of Biochemistry, Universidade Federal do Paraná, Curitiba, Brazil.
| |
Collapse
|
3
|
Joshi V, Popp S, Werfel J, McCreery HF. Alignment with neighbours enables escape from dead ends in flocking models. J R Soc Interface 2022; 19:20220356. [PMID: 35975561 PMCID: PMC9382454 DOI: 10.1098/rsif.2022.0356] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Accepted: 07/25/2022] [Indexed: 11/12/2022] Open
Abstract
Coordinated movement in animal groups (flocks, schools, herds, etc.) is a classic and well-studied form of collective behaviour. Most theoretical studies consider agents in unobstructed spaces; however, many animals move in often complicated environments and must navigate around and through obstacles. Here we consider simulated agents behaving according to typical flocking rules, with the addition of repulsion from obstacles, and study their collective behaviour in environments with concave obstacles (dead ends). We find that groups of such agents heading for a goal can spontaneously escape dead ends without wall-following or other specialized behaviours, in what we term 'flocking escapes'. The mechanism arises when agents align with one another while heading away from the goal, forming a self-stable cluster that persists long enough to exit the obstacle and avoids becoming trapped again when turning back towards the goal. Solitary agents under the same conditions are never observed to escape. We show that alignment with neighbours reduces the effective turning speed of the group while letting individuals maintain high manoeuvrability when needed. The relative robustness of flocking escapes in our studies suggests that this emergent behaviour may be relevant for a variety of animal species.
Collapse
Affiliation(s)
- Varun Joshi
- School of Kinesiology, University of Michigan, Ann Arbor, MI 48109, USA
| | - Stefan Popp
- Department of Ecology and Evolutionary Biology, University of Arizona, Tucson, AZ 85721, USA
| | - Justin Werfel
- School of Engineering and Applied Sciences, Harvard University, Boston, MA 02134, USA
| | - Helen F. McCreery
- School of Engineering and Applied Sciences, Harvard University, Boston, MA 02134, USA
- Biology Department, University of Massachusetts, Boston, MA 02125, USA
| |
Collapse
|
4
|
Abstract
Humans and other animals use multiple strategies for making decisions. Reinforcement-learning theory distinguishes between stimulus-response (model-free; MF) learning and deliberative (model-based; MB) planning. The spatial-navigation literature presents a parallel dichotomy between navigation strategies. In "response learning," associated with the dorsolateral striatum (DLS), decisions are anchored to an egocentric reference frame. In "place learning," associated with the hippocampus, decisions are anchored to an allocentric reference frame. Emerging evidence suggests that the contribution of hippocampus to place learning may also underlie its contribution to MB learning by representing relational structure in a cognitive map. Here, we introduce a computational model in which hippocampus subserves place and MB learning by learning a "successor representation" of relational structure between states; DLS implements model-free response learning by learning associations between actions and egocentric representations of landmarks; and action values from either system are weighted by the reliability of its predictions. We show that this model reproduces a range of seemingly disparate behavioral findings in spatial and nonspatial decision tasks and explains the effects of lesions to DLS and hippocampus on these tasks. Furthermore, modeling place cells as driven by boundaries explains the observation that, unlike navigation guided by landmarks, navigation guided by boundaries is robust to "blocking" by prior state-reward associations due to learned associations between place cells. Our model, originally shaped by detailed constraints in the spatial literature, successfully characterizes the hippocampal-striatal system as a general system for decision making via adaptive combination of stimulus-response learning and the use of a cognitive map.
Collapse
|
5
|
Sosa M, Joo HR, Frank LM. Dorsal and Ventral Hippocampal Sharp-Wave Ripples Activate Distinct Nucleus Accumbens Networks. Neuron 2019; 105:725-741.e8. [PMID: 31864947 DOI: 10.1016/j.neuron.2019.11.022] [Citation(s) in RCA: 58] [Impact Index Per Article: 11.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Revised: 10/08/2019] [Accepted: 11/15/2019] [Indexed: 12/31/2022]
Abstract
Memories of positive experiences link places, events, and reward outcomes. These memories recruit interactions between the hippocampus and nucleus accumbens (NAc). Both dorsal and ventral hippocampus (dH and vH) project to the NAc, but it remains unknown whether dH and vH act in concert or separately to engage NAc representations related to space and reward. We recorded simultaneously from the dH, vH, and NAc of rats during an appetitive spatial task and focused on hippocampal sharp-wave ripples (SWRs) to identify times of memory reactivation across brain regions. Here, we show that dH and vH awake SWRs occur asynchronously and activate distinct and opposing patterns of NAc spiking. Only NAc neurons activated during dH SWRs were tuned to task- and reward-related information. These temporally and anatomically separable hippocampal-NAc interactions point to distinct channels of mnemonic processing in the NAc, with the dH-NAc channel specialized for spatial task and reward information. VIDEO ABSTRACT.
Collapse
Affiliation(s)
- Marielena Sosa
- Neuroscience Graduate Program, Kavli Institute for Fundamental Neuroscience, and Department of Physiology, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Hannah R Joo
- Neuroscience Graduate Program, Kavli Institute for Fundamental Neuroscience, and Department of Physiology, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Loren M Frank
- Neuroscience Graduate Program, Kavli Institute for Fundamental Neuroscience, and Department of Physiology, University of California, San Francisco, San Francisco, CA 94158, USA; Howard Hughes Medical Institute, San Francisco, CA 94158, USA.
| |
Collapse
|
6
|
Hippocampal pattern separation supports reinforcement learning. Nat Commun 2019; 10:1073. [PMID: 30842581 PMCID: PMC6403348 DOI: 10.1038/s41467-019-08998-1] [Citation(s) in RCA: 29] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2018] [Accepted: 02/13/2019] [Indexed: 11/08/2022] Open
Abstract
Animals rely on learned associations to make decisions. Associations can be based on relationships between object features (e.g., the three leaflets of poison ivy leaves) and outcomes (e.g., rash). More often, outcomes are linked to multidimensional states (e.g., poison ivy is green in summer but red in spring). Feature-based reinforcement learning fails when the values of individual features depend on the other features present. One solution is to assign value to multi-featural conjunctive representations. Here, we test if the hippocampus forms separable conjunctive representations that enables the learning of response contingencies for stimuli of the form: AB+, B-, AC-, C+. Pattern analyses on functional MRI data show the hippocampus forms conjunctive representations that are dissociable from feature components and that these representations, along with those of cortex, influence striatal prediction errors. Our results establish a novel role for hippocampal pattern separation and conjunctive representation in reinforcement learning.
Collapse
|
7
|
Gmaz JM, Carmichael JE, van der Meer MA. Persistent coding of outcome-predictive cue features in the rat nucleus accumbens. eLife 2018; 7:37275. [PMID: 30234485 PMCID: PMC6195350 DOI: 10.7554/elife.37275] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2018] [Accepted: 09/15/2018] [Indexed: 01/09/2023] Open
Abstract
The nucleus accumbens (NAc) is important for learning from feedback, and for biasing and invigorating behaviour in response to cues that predict motivationally relevant outcomes. NAc encodes outcome-related cue features such as the magnitude and identity of reward. However, little is known about how features of cues themselves are encoded. We designed a decision making task where rats learned multiple sets of outcome-predictive cues, and recorded single-unit activity in the NAc during performance. We found that coding of cue identity and location occurred alongside coding of expected outcome. Furthermore, this coding persisted both during a delay period, after the rat made a decision and was waiting for an outcome, and after the outcome was revealed. Encoding of cue features in the NAc may enable contextual modulation of on-going behaviour, and provide an eligibility trace of outcome-predictive stimuli for updating stimulus-outcome associations to inform future behaviour.
Collapse
Affiliation(s)
- Jimmie M Gmaz
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, United States
| | - James E Carmichael
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, United States
| | | |
Collapse
|
8
|
McCreery HF, Dix ZA, Breed MD, Nagpal R. Collective strategy for obstacle navigation during cooperative transport by ants. J Exp Biol 2016; 219:3366-3375. [DOI: 10.1242/jeb.143818] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2016] [Accepted: 08/16/2016] [Indexed: 11/20/2022]
Abstract
ABSTRACT
Group cohesion and consensus have primarily been studied in the context of discrete decisions, but some group tasks require making serial decisions that build on one another. We examine such collective problem solving by studying obstacle navigation during cooperative transport in ants. In cooperative transport, ants work together to move a large object back to their nest. We blocked cooperative transport groups of Paratrechina longicornis with obstacles of varying complexity, analyzing groups' trajectories to infer what kind of strategy the ants employed. Simple strategies require little information, but more challenging, robust strategies succeed with a wider range of obstacles. We found that transport groups use a stochastic strategy that leads to efficient navigation around simple obstacles, and still succeeds at difficult obstacles. While groups navigating obstacles preferentially move directly toward the nest, they change their behavior over time; the longer the ants are obstructed, the more likely they are to move away from the nest. This increases the chance of finding a path around the obstacle. Groups rapidly changed directions and rarely stalled during navigation, indicating that these ants maintain consensus even when the nest direction is blocked. Although some decisions were aided by the arrival of new ants, at many key points, direction changes were initiated within the group, with no apparent external cause. This ant species is highly effective at navigating complex environments, and implements a flexible strategy that works for both simple and more complex obstacles.
Collapse
Affiliation(s)
- Helen F. McCreery
- Department of Ecology and Evolutionary Biology, University of Colorado, Boulder, Ramaley N122, Campus Box 334, Boulder, CO 80309-0334, USA
| | - Zachary A. Dix
- Department of Ecology and Evolutionary Biology, University of Colorado, Boulder, Ramaley N122, Campus Box 334, Boulder, CO 80309-0334, USA
| | - Michael D. Breed
- Department of Ecology and Evolutionary Biology, University of Colorado, Boulder, Ramaley N122, Campus Box 334, Boulder, CO 80309-0334, USA
| | - Radhika Nagpal
- Department of Computer Science and Wyss Institute, Harvard University, 33 Oxford Street, Cambridge, MA 02138, USA
| |
Collapse
|
9
|
Nicola SM. Reassessing wanting and liking in the study of mesolimbic influence on food intake. Am J Physiol Regul Integr Comp Physiol 2016; 311:R811-R840. [PMID: 27534877 PMCID: PMC5130579 DOI: 10.1152/ajpregu.00234.2016] [Citation(s) in RCA: 34] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2016] [Accepted: 08/09/2016] [Indexed: 01/12/2023]
Abstract
Humans and animals such as rats and mice tend to overconsume calorie-dense foods, a phenomenon that likely contributes to obesity. One often-advanced explanation for why we preferentially consume sweet and fatty foods is that they are more "rewarding" than low-calorie foods. "Reward" has been subdivided into three interdependent psychological processes: hedonia (liking a food), reinforcement (formation of associations among stimuli, actions, and/or the food), and motivation (wanting the food). Research into these processes has focused on the mesolimbic system, which comprises both dopamine neurons in the ventral tegmental area and neurons in their major projection target, the nucleus accumbens. The mesolimbic system and closely connected structures are commonly referred to as the brain's "reward circuit." Implicit in this title is the assumption that "rewarding" experiences are generally the result of activity in this circuit. In this review, I argue that food intake and the preference for calorie-dense foods can be explained without reference to subjective emotions. Furthermore, the contribution of mesolimbic dopamine to food intake and preference may not be a general one of promoting or coordinating behaviors that result in the most reward or caloric intake but may instead be limited to the facilitation of a specific form of neural computation that results in conditioned approach behavior. Studies on the neural mechanisms of caloric intake regulation must address how sensory information about calorie intake affects not just the mesolimbic system but also many other forms of computation that govern other types of food-seeking and food-oriented behaviors.
Collapse
Affiliation(s)
- Saleem M Nicola
- Departments of Neuroscience and Psychiatry, Albert Einstein College of Medicine, Bronx, New York
| |
Collapse
|
10
|
Barkan S, Yom-Tov Y, Barnea A. A possible relation between new neuronal recruitment and migratory behavior inAcrocephaluswarblers. Dev Neurobiol 2014; 74:1194-209. [DOI: 10.1002/dneu.22198] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2014] [Revised: 05/28/2014] [Accepted: 06/02/2014] [Indexed: 12/13/2022]
Affiliation(s)
- Shay Barkan
- Department of Zoology; Tel-Aviv University; Tel-Aviv 61391 Israel
| | - Yoram Yom-Tov
- Department of Zoology; Tel-Aviv University; Tel-Aviv 61391 Israel
| | - Anat Barnea
- Department of Natural and Life Sciences; The Open University of Israel; Ra'anana 43107 Israel
| |
Collapse
|
11
|
Mannella F, Gurney K, Baldassarre G. The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis. Front Behav Neurosci 2013; 7:135. [PMID: 24167476 PMCID: PMC3805952 DOI: 10.3389/fnbeh.2013.00135] [Citation(s) in RCA: 94] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2013] [Accepted: 09/15/2013] [Indexed: 01/01/2023] Open
Abstract
Goal-directed behavior is a fundamental means by which animals can flexibly solve the challenges posed by variable external and internal conditions. Recently, the processes and brain mechanisms underlying such behavior have been extensively studied from behavioral, neuroscientific and computational perspectives. This research has highlighted the processes underlying goal-directed behavior and associated brain systems including prefrontal cortex, basal ganglia and, in particular therein, the nucleus accumbens (NAcc). This paper focusses on one particular process at the core of goal-directed behavior: how motivational value is assigned to goals on the basis of internal states and environmental stimuli, and how this supports goal selection processes. Various biological and computational accounts have been given of this problem and of related multiple neural and behavior phenomena, but we still lack an integrated hypothesis on the generation and use of value for goal selection. This paper proposes an hypothesis that aims to solve this problem and is based on this key elements: (a) amygdala and hippocampus establish the motivational value of stimuli and goals; (b) prefrontal cortex encodes various types of action outcomes; (c) NAcc integrates different sources of value, representing them in terms of a common currency with the aid of dopamine, and thereby plays a major role in selecting action outcomes within prefrontal cortex. The “goals” pursued by the organism are the outcomes selected by these processes. The hypothesis is developed in the context of a critical review of relevant biological and computational literature which offer it support. The paper shows how the hypothesis has the potential to integrate existing interpretations of motivational value and goal selection.
Collapse
Affiliation(s)
- Francesco Mannella
- Laboratory of Computational Embodied Neuroscience, Institute of Cognitive Sciences and Technologies, National Research Council Rome, Italy
| | | | | |
Collapse
|
12
|
McGinty VB, Lardeux S, Taha SA, Kim JJ, Nicola SM. Invigoration of reward seeking by cue and proximity encoding in the nucleus accumbens. Neuron 2013; 78:910-22. [PMID: 23764290 DOI: 10.1016/j.neuron.2013.04.010] [Citation(s) in RCA: 92] [Impact Index Per Article: 8.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/01/2013] [Indexed: 10/26/2022]
Abstract
A key function of the nucleus accumbens is to promote vigorous reward seeking, but the corresponding neural mechanism has not been identified despite many years of research. Here, we study cued flexible approach behavior, a form of reward seeking that strongly depends on the accumbens, and we describe a robust, single-cell neural correlate of behavioral vigor in the excitatory response of accumbens neurons to reward-predictive cues. Well before locomotion begins, this cue-evoked excitation predicts both the movement initiation latency and the speed of subsequent flexible approach responses, but not those of stereotyped, inflexible responses. Moreover, the excitation simultaneously signals the subject's proximity to the approach target, a signal that appears to mediate greater response vigor on trials that begin with the subject closer to the target. These results demonstrate a neural mechanism for response invigoration whereby accumbens neuronal encoding of reward availability and target proximity together drive the onset and speed of reward-seeking locomotion.
Collapse
Affiliation(s)
- Vincent B McGinty
- Department of Psychiatry, Albert Einstein College of Medicine of Yeshiva University, Bronx, NY 10461, USA
| | | | | | | | | |
Collapse
|
13
|
Abstract
Information processing in behaving animals has been the target of many studies in the striatum; however, its dynamics and complexity remain to a large extent unknown. Here, we chronically recorded neuronal populations in dorsal striatum as mice were exposed to a novel environment, a paradigm which enables the dissociation of locomotion and environmental recognition. The findings indicate that non-overlapping populations of striatal projection neurons-the medium spiny neurons-reliably encode locomotion and environmental identity, whereas two subpopulations of short-spike interneurons encode distinct information: the fast spiking interneurons preferentially encode locomotion whereas the second type of interneurons preferentially encodes environmental identity. The three neuronal subgroups used cell-type specific coding schemes. This study provides evidence for the existence of parallel processing circuits within the sensorimotor region of the striatum.
Collapse
|
14
|
Khamassi M, Humphries MD. Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies. Front Behav Neurosci 2012. [PMID: 23205006 PMCID: PMC3506961 DOI: 10.3389/fnbeh.2012.00079] [Citation(s) in RCA: 50] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
Abstract
Behavior in spatial navigation is often organized into map-based (place-driven) vs. map-free (cue-driven) strategies; behavior in operant conditioning research is often organized into goal-directed vs. habitual strategies. Here we attempt to unify the two. We review one powerful theory for distinct forms of learning during instrumental conditioning, namely model-based (maintaining a representation of the world) and model-free (reacting to immediate stimuli) learning algorithms. We extend these lines of argument to propose an alternative taxonomy for spatial navigation, showing how various previously identified strategies can be distinguished as “model-based” or “model-free” depending on the usage of information and not on the type of information (e.g., cue vs. place). We argue that identifying “model-free” learning with dorsolateral striatum and “model-based” learning with dorsomedial striatum could reconcile numerous conflicting results in the spatial navigation literature. From this perspective, we further propose that the ventral striatum plays key roles in the model-building process. We propose that the core of the ventral striatum is positioned to learn the probability of action selection for every transition between states of the world. We further review suggestions that the ventral striatal core and shell are positioned to act as “critics” contributing to the computation of a reward prediction error for model-free and model-based systems, respectively.
Collapse
Affiliation(s)
- Mehdi Khamassi
- Institut des Systèmes Intelligents et de Robotique, Université Pierre et Marie Curie Paris, France ; Centre National de la Recherche Scientifique, UMR7222 Paris, France
| | | |
Collapse
|
15
|
Shi Z, Sun X, Liu X, Chen S, Chang Q, Chen L, Song G, Li H. Evaluation of an Aβ1–40-induced cognitive deficit in rat using a reward-directed instrumental learning task. Behav Brain Res 2012; 234:323-33. [DOI: 10.1016/j.bbr.2012.07.006] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2012] [Revised: 07/03/2012] [Accepted: 07/06/2012] [Indexed: 01/04/2023]
|
16
|
Segmentation of spatial experience by hippocampal θ sequences. Nat Neurosci 2012; 15:1032-9. [PMID: 22706269 DOI: 10.1038/nn.3138] [Citation(s) in RCA: 218] [Impact Index Per Article: 18.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2012] [Accepted: 05/14/2012] [Indexed: 11/08/2022]
Abstract
The encoding and storage of experience by the hippocampus is essential for the formation of episodic memories and the transformation of individual experiences into semantic structures such as maps and schemas. The rodent hippocampus compresses ongoing experience into repeating theta sequences, but the factors determining the content of theta sequences are not understood. Here we first show that the spatial paths represented by theta sequences in rats extend farther in front of the rat during acceleration and higher running speeds and begin farther behind the rat during deceleration. Second, the length of the path is directly related to the length of the theta cycle and the number of gamma cycles in it. Finally, theta sequences represent the environment in segments or 'chunks'. These results imply that information encoded in theta sequences is subject to powerful modulation by behavior and task variables. Furthermore, these findings suggest a potential mechanism for the cognitive 'chunking' of experience.
Collapse
|
17
|
Penner MR, Mizumori SJY. Neural systems analysis of decision making during goal-directed navigation. Prog Neurobiol 2011; 96:96-135. [PMID: 21964237 DOI: 10.1016/j.pneurobio.2011.08.010] [Citation(s) in RCA: 47] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2011] [Revised: 08/06/2011] [Accepted: 08/29/2011] [Indexed: 10/17/2022]
Abstract
The ability to make adaptive decisions during goal-directed navigation is a fundamental and highly evolved behavior that requires continual coordination of perceptions, learning and memory processes, and the planning of behaviors. Here, a neurobiological account for such coordination is provided by integrating current literatures on spatial context analysis and decision-making. This integration includes discussions of our current understanding of the role of the hippocampal system in experience-dependent navigation, how hippocampal information comes to impact midbrain and striatal decision making systems, and finally the role of the striatum in the implementation of behaviors based on recent decisions. These discussions extend across cellular to neural systems levels of analysis. Not only are key findings described, but also fundamental organizing principles within and across neural systems, as well as between neural systems functions and behavior, are emphasized. It is suggested that studying decision making during goal-directed navigation is a powerful model for studying interactive brain systems and their mediation of complex behaviors.
Collapse
Affiliation(s)
- Marsha R Penner
- Department of Psychology, University of Washington, Seattle, WA 98195-1525, United States
| | | |
Collapse
|
18
|
The dorsomedial striatum mediates flexible choice behavior in spatial tasks. Behav Brain Res 2011; 220:288-93. [DOI: 10.1016/j.bbr.2011.02.008] [Citation(s) in RCA: 26] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2010] [Revised: 02/04/2011] [Accepted: 02/06/2011] [Indexed: 10/18/2022]
|
19
|
The hippocampus: hub of brain network communication for memory. Trends Cogn Sci 2011; 15:310-8. [PMID: 21696996 DOI: 10.1016/j.tics.2011.05.008] [Citation(s) in RCA: 136] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2011] [Revised: 05/17/2011] [Accepted: 05/17/2011] [Indexed: 01/26/2023]
Abstract
A complex brain network, centered on the hippocampus, supports episodic memories throughout their lifetimes. Classically, upon memory encoding during active behavior, hippocampal activity is dominated by theta oscillations (6-10Hz). During inactivity, hippocampal neurons burst synchronously, constituting sharp waves, which can propagate to other structures, theoretically supporting memory consolidation. This 'two-stage' model has been updated by new data from high-density electrophysiological recordings in animals that shed light on how information is encoded and exchanged between hippocampus, neocortex and subcortical structures such as the striatum. Cell assemblies (tightly related groups of cells) discharge together and synchronize across brain structures orchestrated by theta, sharp waves and slow oscillations, to encode information. This evolving dynamical schema is key to extending our understanding of memory processes.
Collapse
|
20
|
Lex B, Hauber W. Disconnection of the entorhinal cortex and dorsomedial striatum impairs the sensitivity to instrumental contingency degradation. Neuropsychopharmacology 2010; 35:1788-96. [PMID: 20357754 PMCID: PMC3055490 DOI: 10.1038/npp.2010.46] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
The capacity to detect changes in the causal efficacy of actions is mediated by a number of brain areas, including the entorhinal cortex (EC) and the posterior part of the dorsomedial striatum (pDMS). In this study we examined whether interactions between the EC and pDMS are required to detect changes in the instrumental contingency. Rats that received EC-pDMS disconnection lesions, that is, unilateral cell body lesions of the EC and contralateral dopamine depletions of the pDMS, were trained to press two levers, with one delivering food pellets and the other a sucrose solution. Thereafter, we tested whether rats were sensitive (1) to a selective devaluation of the value of one of two outcomes using a specific satiety procedure, and (2) to a selective degradation of one of two contingencies controlling instrumental choice behavior. Our results reveal that rats with EC-pDMS disconnection lesions were sensitive to outcome devaluation. However, unlike rats with sham lesions or unilateral EC and pDMS lesions, rats with EC-pDMS disconnection lesions showed a reduced sensitivity to contingency degradation. These findings suggest that EC and pDMS may be part of a neural system that supports the detection of changes in the causal relationship between an action and its consequences.
Collapse
Affiliation(s)
- Bjoern Lex
- Abteilung Tierphysiologie, Biologisches Institut, Universität Stuttgart, Stuttgart, Germany
| | - Wolfgang Hauber
- Abteilung Tierphysiologie, Biologisches Institut, Universität Stuttgart, Stuttgart, Germany,Department of Animal Physiology, Universität Stuttgart, Biologisches Institut, Abteilung Tierphysiologie, Pfaffenwaldring 57, D-70550 Stuttgart, Germany, Tel: +49-711-685-65003, Fax: +49-711-685-55000, E-mail:
| |
Collapse
|
21
|
Humphries MD, Prescott TJ. The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Prog Neurobiol 2009; 90:385-417. [PMID: 19941931 DOI: 10.1016/j.pneurobio.2009.11.003] [Citation(s) in RCA: 243] [Impact Index Per Article: 16.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2009] [Revised: 11/12/2009] [Accepted: 11/16/2009] [Indexed: 11/27/2022]
Abstract
The basal ganglia are often conceptualised as three parallel domains that include all the constituent nuclei. The 'ventral domain' appears to be critical for learning flexible behaviours for exploration and foraging, as it is the recipient of converging inputs from amygdala, hippocampal formation and prefrontal cortex, putatively centres for stimulus evaluation, spatial navigation, and planning/contingency, respectively. However, compared to work on the dorsal domains, the rich potential for quantitative theories and models of the ventral domain remains largely untapped, and the purpose of this review is to provide the stimulus for this work. We systematically review the ventral domain's structures and internal organisation, and propose a functional architecture as the basis for computational models. Using a full schematic of the structure of inputs to the ventral striatum (nucleus accumbens core and shell), we argue for the existence of many identifiable processing channels on the basis of unique combinations of afferent inputs. We then identify the potential information represented in these channels by reconciling a broad range of studies from the hippocampal, amygdala and prefrontal cortex literatures with known properties of the ventral striatum from lesion, pharmacological, and electrophysiological studies. Dopamine's key role in learning is reviewed within the three current major computational frameworks; we also show that the shell-based basal ganglia sub-circuits are well placed to generate the phasic burst and dip responses of dopaminergic neurons. We detail dopamine's modulation of ventral basal ganglia's inputs by its actions on pre-synaptic terminals and post-synaptic membranes in the striatum, arguing that the complexity of these effects hint at computational roles for dopamine beyond current ideas. The ventral basal ganglia are revealed as a constellation of multiple functional systems for the learning and selection of flexible behaviours and of behavioural strategies, sharing the common operations of selection-by-disinhibition and of dopaminergic modulation.
Collapse
Affiliation(s)
- Mark D Humphries
- Adaptive Behaviour Research Group, Department of Psychology, University of Sheffield, S10 2TN, UK.
| | | |
Collapse
|
22
|
Lex B, Hauber W. The role of dopamine in the prelimbic cortex and the dorsomedial striatum in instrumental conditioning. Cereb Cortex 2009; 20:873-83. [PMID: 19605519 DOI: 10.1093/cercor/bhp151] [Citation(s) in RCA: 65] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
The prelimbic (PL) region of the prefrontal cortex and the posterior subregion of the dorsomedial striatum (pDMS) are components of a corticostriatal circuit subserving instrumental learning. Here, we examined whether dopamine (DA) signals conveyed to the PL and pDMS are critical for instrumental learning. Rats with 6-hydroxydopamine or vehicle infusion into the PL and pDMS were trained to press 2 levers, either for food pellets or a sucrose solution. Thereafter, we tested whether the animals were sensitive 1) to a selective degradation of 1 of 2 outcomes using a specific satiety procedure and 2) to a selective degradation of 1 of 2 contingencies controlling instrumental behavior. Rats with PL DA depletion displayed a reduced rate of lever presses but appeared to be sensitive to outcome devaluation and contingency degradation. Thus, PL DA seems to modulate lever pressing but does not support instrumental conditioning. In contrast, rats with pDMS DA depletion had intact response rates and were sensitive to selective outcome devaluation; however, they showed a reduced sensitivity to contingency degradation. Therefore, pDMS DA signaling seems not to be involved in maintaining lever pressing but instead contributes to instrumental conditioning by supporting the detection of causal relationships between an action and its consequences.
Collapse
Affiliation(s)
- Bjoern Lex
- Abteilung Tierphysiologie, Biologisches Institut, Universität Stuttgart, D-70550 Stuttgart, Germany
| | | |
Collapse
|
23
|
Schwarting RK. Rodent models of serial reaction time tasks and their implementation in neurobiological research. Behav Brain Res 2009; 199:76-88. [DOI: 10.1016/j.bbr.2008.07.004] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2008] [Revised: 07/07/2008] [Accepted: 07/08/2008] [Indexed: 11/30/2022]
|
24
|
Fox C, Humphries M, Mitchinson B, Kiss T, Somogyvari Z, Prescott T. Technical integration of hippocampus, Basal Ganglia and physical models for spatial navigation. Front Neuroinform 2009; 3:6. [PMID: 19333376 PMCID: PMC2659166 DOI: 10.3389/neuro.11.006.2009] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2008] [Accepted: 02/20/2009] [Indexed: 01/03/2023] Open
Abstract
Computational neuroscience is increasingly moving beyond modeling individual neurons or neural systems to consider the integration of multiple models, often constructed by different research groups. We report on our preliminary technical integration of recent hippocampal formation, basal ganglia and physical environment models, together with visualisation tools, as a case study in the use of Python across the modelling tool-chain. We do not present new modeling results here. The architecture incorporates leaky-integrator and rate-coded neurons, a 3D environment with collision detection and tactile sensors, 3D graphics and 2D plots. We found Python to be a flexible platform, offering a significant reduction in development time, without a corresponding significant increase in execution time. We illustrate this by implementing a part of the model in various alternative languages and coding styles, and comparing their execution times. For very large-scale system integration, communication with other languages and parallel execution may be required, which we demonstrate using the BRAHMS framework's Python bindings.
Collapse
Affiliation(s)
- Charles Fox
- Adaptive Behaviour Research Group, Department of Psychology, University of Sheffield Sheffield, UK
| | | | | | | | | | | |
Collapse
|
25
|
Khamassi M, Mulder AB, Tabuchi E, Douchamps V, Wiener SI. Anticipatory reward signals in ventral striatal neurons of behaving rats. Eur J Neurosci 2009; 28:1849-66. [PMID: 18973599 DOI: 10.1111/j.1460-9568.2008.06480.x] [Citation(s) in RCA: 33] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
It has been proposed that the striatum plays a crucial role in learning to select appropriate actions, optimizing rewards according to the principles of 'Actor-Critic' models of trial-and-error learning. The ventral striatum (VS), as Critic, would employ a temporal difference (TD) learning algorithm to predict rewards and drive dopaminergic neurons. This study examined this model's adequacy for VS responses to multiple rewards in rats. The respective arms of a plus-maze provided rewards of varying magnitudes; multiple rewards were provided at 1-s intervals while the rat stood still. Neurons discharged phasically prior to each reward, during both initial approach and immobile waiting, demonstrating that this signal is predictive and not simply motor-related. In different neurons, responses could be greater for early, middle or late droplets in the sequence. Strikingly, this activity often reappeared after the final reward, as if in anticipation of yet another. In contrast, previous TD learning models show decremental reward-prediction profiles during reward consumption due to a temporal-order signal introduced to reproduce accurate timing in dopaminergic reward-prediction error signals. To resolve this inconsistency in a biologically plausible manner, we adapted the TD learning model such that input information is nonhomogeneously distributed among different neurons. By suppressing reward temporal-order signals and varying richness of spatial and visual input information, the model reproduced the experimental data. This validates the feasibility of a TD-learning architecture where different groups of neurons participate in solving the task based on varied input information.
Collapse
Affiliation(s)
- Mehdi Khamassi
- Laboratoire de Physiologie de la Perception et de l'Action, Collège de France, CNRS, Paris, France
| | | | | | | | | |
Collapse
|
26
|
Berke JD, Breck JT, Eichenbaum H. Striatal versus hippocampal representations during win-stay maze performance. J Neurophysiol 2009; 101:1575-87. [PMID: 19144741 DOI: 10.1152/jn.91106.2008] [Citation(s) in RCA: 83] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
The striatum and hippocampus are widely held to be components of distinct memory systems that can guide competing behavioral strategies. However, some electrophysiological studies have suggested that neurons in both structures encode spatial information and may therefore make similar contributions to behavior. In rats well trained to perform a win-stay radial maze task, we recorded simultaneously from dorsal hippocampus and from multiple striatal subregions, including both lateral areas implicated in motor responses to cues and medial areas that work cooperatively with hippocampus in cognitive operations. In each brain region, movement through the maze was accompanied by the continuous sequential activation of sets of projection neurons. Hippocampal neurons overwhelmingly were active at a single spatial location (place cells). Striatal projection neurons were active at discrete points within the progression of every trial-especially during choices or following reward delivery-regardless of spatial position. Place-cell-type firing was not observed even for medial striatal cells entrained to the hippocampal theta rhythm. We also examined neural coding in earlier training sessions, when rats made use of spatial working memory to guide choices, and again found that striatal cells did not show place-cell-type firing. Prospective or retrospective encoding of trajectory was not observed in either hippocampus or striatum, at either training stage. Our results indicate that, at least in this task, dorsal hippocampus uses a spatial foundation for information processing that is not substantially modulated by spatial working memory demands. By contrast, striatal cells do not use such a spatial foundation, even in medial subregions that cooperate with hippocampus in the selection of spatial strategies. The progressive dominance of a striatum-dependent strategy does not appear to be accompanied by large changes in striatal or hippocampal single-cell representations, suggesting that the conflict between strategies may be resolved elsewhere.
Collapse
Affiliation(s)
- Joshua D Berke
- University of Michigan, Department of Psychology, Ann Arbor, MI 48109-1109, USA.
| | | | | |
Collapse
|
27
|
Da Cunha C, Wietzikoski EC, Dombrowski P, Bortolanza M, Santos LM, Boschen SL, Miyoshi E. Learning processing in the basal ganglia: a mosaic of broken mirrors. Behav Brain Res 2008; 199:157-70. [PMID: 18977393 DOI: 10.1016/j.bbr.2008.10.001] [Citation(s) in RCA: 40] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2008] [Revised: 10/01/2008] [Accepted: 10/02/2008] [Indexed: 11/25/2022]
Abstract
In the present review we propose a model to explain the role of the basal ganglia in sensorimotor and cognitive functions based on a growing body of behavioural, anatomical, physiological, and neurochemical evidence accumulated over the last decades. This model proposes that the body and its surrounding environment are represented in the striatum in a fragmented and repeated way, like a mosaic consisting of the fragmented images of broken mirrors. Each fragment forms a functional unit representing articulated parts of the body with motion properties, objects of the environment which the subject can approach or manipulate, and locations the subject can move to. These units integrate the sensory properties and movements related to them. The repeated and widespread distribution of such units amplifies the combinatorial power of the associations among them. These associations depend on the phasic release of dopamine in the striatum triggered by the saliency of stimuli and will be reinforced by the rewarding consequences of the actions related to them. Dopamine permits synaptic plasticity in the corticostriatal synapses. The striatal units encoding the same stimulus/action send convergent projections to the internal segment of the globus pallidus (GPi) and to the substantia nigra pars reticulata (SNr) that stimulate or hold the action through a thalamus-frontal cortex pathway. According to this model, this is how the basal ganglia select actions based on environmental stimuli and store adaptive associations as nondeclarative memories such as motor skills, habits, and memories formed by Pavlovian and instrumental conditioning.
Collapse
Affiliation(s)
- Claudio Da Cunha
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Departamento de Farmacologia, UFPR, C.P. 19.031, 81.531-980 Curitiba PR, Brazil.
| | | | | | | | | | | | | |
Collapse
|
28
|
Kim YB, Huh N, Lee H, Baeg EH, Lee D, Jung MW. Encoding of action history in the rat ventral striatum. J Neurophysiol 2007; 98:3548-56. [PMID: 17942629 DOI: 10.1152/jn.00310.2007] [Citation(s) in RCA: 37] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
In a dynamic environment, animals need to update information about the rewards expected from their alternative actions continually to make optimal choices for its survival. Because the reward resulting from a given action can be substantially delayed, the process of linking a reward to its causative action would be facilitated by memory signals related to the animal's previous actions. Although the ventral striatum has been proposed to play a key role in updating the information about the rewards expected from specific actions, it is not known whether the signals related to previous actions exist in the ventral striatum. In the present study, we recorded neuronal ensemble activity in the rat ventral striatum during a visual discrimination task and investigated whether neuronal activity in the ventral striatum encoded signals related to animal's previous actions. The results show that many neurons modulated their activity according to the animal's goal choice in the previous trial, indicating that memory signals for previous actions are available in the ventral striatum. In contrast, few neurons conveyed signals on impending goal choice of the animal, suggesting the absence of decision signals in the ventral striatum. Memory signals for previous actions might contribute to the process of updating the estimates of rewards expected from alternative actions in the ventral striatum.
Collapse
Affiliation(s)
- Yun Bok Kim
- Neuroscience Laboratory, Institute for Medical Sciences, Ajou University School of Medicine, Suwon, Korea
| | | | | | | | | | | |
Collapse
|
29
|
A simple model of cued T-Maze learning based on basal ganglia anatomy and sequence replay. BMC Neurosci 2007. [PMCID: PMC4436284 DOI: 10.1186/1471-2202-8-s2-p82] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/03/2022] Open
|
30
|
Abstract
Place cells of the rat hippocampus are a dominant model system for understanding the role of the hippocampus in learning and memory at the level of single-unit and neural ensemble responses. A complete understanding of the information processing and computations performed by the hippocampus requires detailed knowledge about the properties of the representations that are present in hippocampal afferents and efferents in order to decipher the transformations that occur to these representations in the hippocampal circuitry. Neural recordings in behaving rats have revealed a number of brain areas that contain place-related firing properties in the parahippocampal regions and in other brain regions that are thought to interact with the hippocampus in certain behavioral tasks. Although investigators have just begun to scratch the surface in terms of understanding these properties, differences in the precise nature of the spatial firing between the hippocampus and these other regions promise to reveal important clues regarding the exact role of the hippocampus in learning and memory and the nature of its interactions with other brain systems to support adaptive behavior.
Collapse
Affiliation(s)
- James J Knierim
- Department of Neurobiology & Anatomy, W.M. Keck Center for the Neurobiology of Learning and Memory, University of Texas Medical School at Houston, 77225, USA.
| |
Collapse
|
31
|
Abstract
Many organisms, especially humans, are characterized by their capacity for intentional, goal-directed actions. However, similar behaviours often proceed automatically, as habitual responses to antecedent stimuli. How are goal-directed actions transformed into habitual responses? Recent work combining modern behavioural assays and neurobiological analysis of the basal ganglia has begun to yield insights into the neural basis of habit formation.
Collapse
Affiliation(s)
- Henry H Yin
- Laboratory for Integrative Neuroscience, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, 5625 Fishers Lane, TS-13, Bethesda, Maryland 20892, USA
| | | |
Collapse
|
32
|
Abstract
Many mammals spontaneously rear on their hind legs in response to novelty. The current paper is the first review of rearing behaviour, and is intended to collate findings from different perspectives that are not usually brought together. We suggest that rearing is a useful marker of environmental novelty, that the hippocampal formation is a crucial component of the system controlling rearing in novel environments, and that rearing is one of several ethological measures that can profitably be used to assess hippocampal learning and memory. Consideration is given to the following topics: the possible functions of rearing in information-gathering and escape behaviour; the modulation of rearing by various factors such as anxiety/ fear emotionality; comparative perspectives on rearing; neuroanatomical circuits involved in rearing with particular reference to the hippocampal formation and its afferents and efferents; and the role of the hippocampal formation in uncharted and mismatch environmental novelty. The review concludes with testable predictions about rearing, environmental novelty and the hippocampus.
Collapse
Affiliation(s)
- Colin Lever
- Department ofAnatomy and Developmental Biology, University College London, London, UK.
| | | | | |
Collapse
|
33
|
Abstract
During the 25 years since a motivational systems model was proposed to explain the brain mechanisms of aggressive behavior (D.B. Adams. Brain mechanisms for offense, defense, and submission. Behav. Brain. Sci. 2, (1979a) 200-241) considerable research has been carried out. Updating the model in the light of this research requires several changes. A previous distinction between submission and defense systems is abandoned and, instead, it is proposed that two distinct subsets of the defense motivational mechanism may be recognized, one for anti-predator defense and the other for consociate defense. Similarly, the offense motivational mechanism is now considered to have at least two subsets, one mediating territorial and the other competitive fighting. Data continue to indicate that the defense motivational mechanism is located in the midbrain central gray and adjoining tissue. Also data tend to support the hypothesis that the offense motivational mechanism is located in the hypothalamus at the level of the anterior hypothalamus. Consideration is also given to a motivational system for patrol/marking which is related to aggressive behavior. Research is reviewed that bears on the neural structure of motivating and releasing/directing stimuli and motor patterning mechanisms of offense, defense and patrol/marking, as well as the location of learning and hormonal effects, and attention is given to how the model can be tested.
Collapse
Affiliation(s)
- David B Adams
- Psychology Department, Wesleyan University Department of Psychology, Middletown, CT 06459, USA.
| |
Collapse
|
34
|
Prescott TJ, Montes González FM, Gurney K, Humphries MD, Redgrave P. A robot model of the basal ganglia: Behavior and intrinsic processing. Neural Netw 2006; 19:31-61. [PMID: 16153803 DOI: 10.1016/j.neunet.2005.06.049] [Citation(s) in RCA: 140] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2004] [Accepted: 06/09/2005] [Indexed: 11/20/2022]
Abstract
The existence of multiple parallel loops connecting sensorimotor systems to the basal ganglia has given rise to proposals that these nuclei serve as a selection mechanism resolving competitions between the alternative actions available in a given context. A strong test of this hypothesis is to require a computational model of the basal ganglia to generate integrated selection sequences in an autonomous agent, we therefore describe a robot architecture into which such a model is embedded, and require it to control action selection in a robotic task inspired by animal observations. Our results demonstrate effective action selection by the embedded model under a wide range of sensory and motivational conditions. When confronted with multiple, high salience alternatives, the robot also exhibits forms of behavioral disintegration that show similarities to animal behavior in conflict situations. The model is shown to cast light on recent neurobiological findings concerning behavioral switching and sequencing.
Collapse
Affiliation(s)
- Tony J Prescott
- Adaptive Behavior Research Group, Department of Psychology, University of Sheffield, Sheffield, Western Bank, South Yorkshire, Sheffield S10 2TN, UK.
| | | | | | | | | |
Collapse
|
35
|
Graybiel AM. The basal ganglia: learning new tricks and loving it. Curr Opin Neurobiol 2005; 15:638-44. [PMID: 16271465 DOI: 10.1016/j.conb.2005.10.006] [Citation(s) in RCA: 425] [Impact Index Per Article: 22.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2005] [Accepted: 10/21/2005] [Indexed: 11/30/2022]
Abstract
The field of basal ganglia research is exploding on every level - from discoveries at the molecular level to those based on human brain imaging. A remarkable series of new findings support the view that the basal ganglia are essential for some forms of learning-related plasticity. Other new findings are challenging some of the basic tenets of the field as it now stands. Combined with the new evidence on learning-related functions of the basal ganglia, these studies suggest that the basal ganglia are parts of a brain-wide set of adaptive neural systems promoting optimal motor and cognitive control.
Collapse
Affiliation(s)
- Ann M Graybiel
- Department of Brain and Cognitive Sciences and the McGovern Institute for Brain Research, Massachusetts Institute of Technology, 43 Vassar Street, 46-6133, Cambridge, MA, 02139 USA.
| |
Collapse
|