Place vs. Response Learning: History, Controversy, and Neurobiology

The well-worn route revisited: Striatal and hippocampal system contributions to familiar route navigation

Classic research has shown a division in the neuroanatomical structures that support flexible (e.g., short-cutting) and habitual (e.g., familiar route following) navigational behavior, with hippocampal-caudate systems associated with the former and putamen systems with the latter. There is, however, disagreement about whether the neural structures involved in navigation process particular forms of spatial information, such as associations between constellations of cues forming a cognitive map, versus single landmark-action associations, or alternatively, perform particular reinforcement learning algorithms that allow the use of different spatial strategies, so-called model-based (flexible) or model-free (habitual) forms of learning. We sought to test these theories by asking participants (N = 24) to navigate within a virtual environment through a previously learned, 9-junction route with distinctive landmarks at each junction while undergoing functional magnetic resonance imaging (fMRI). In a series of probe trials, we distinguished knowledge of individual landmark-action associations along the route versus knowledge of the correct sequence of landmark-action associations, either by having absent landmarks, or "out-of-sequence" landmarks. Under a map-based perspective, sequence knowledge would not require hippocampal systems, because there are no constellations of cues available for cognitive map formation. Within a learning-based model, however, responding based on knowledge of sequence would require hippocampal systems because prior context has to be utilized. We found that hippocampal-caudate systems were more active in probes requiring sequence knowledge, supporting the learning-based model. However, we also found greater putamen activation in probes where navigation based purely on sequence memory could be planned, supporting models of putamen function that emphasize its role in action sequencing.

Potential Link Between Exercise and N-Methyl-D-Aspartate Glutamate Receptors in Alcohol Use Disorder: Implications for Therapeutic Strategies

Alcohol use disorder (AUD) is a significant risk factor, accounting for approximately 13% of all deaths in the US. AUD not only destroys families but also causes economic losses due to reduced productivity, absenteeism, and healthcare expenses. Statistics revealing the sustained number of individuals affected by AUD over the years underscore the need for further understanding of the underlying pathophysiology to advance novel therapeutic strategies. Previous research has implicated the limbic brain regions N-methyl-D-aspartate glutamate receptors (NMDAR) in the emotional and behavioral effects of AUD. Given that aerobic exercise can modulate NMDAR activity and sensitivity to alcohol, this review presents a summary of clinical and basic science studies on NMDAR levels induced by alcohol consumption, as well as acute and protracted withdrawal, highlighting the potential role of aerobic exercise as an adjunctive therapy for AUD. Based on our findings, the utility of exercise in the modulation of reward-linked receptors and AUD may be mediated by its effects on NMDA signaling. These data support further consideration of the potential of aerobic exercise as a promising adjunctive therapy for AUD.

Intense training prevents the amnestic effect of inactivation of dorsomedial striatum and induces high resistance to extinction

A large body of evidence has shown that treatments that interfere with memory consolidation become ineffective when animals are subjected to an intense learning experience; this effect has been observed after systemic and local administration of amnestic drugs into several brain areas, including the striatum. However, the effects of amnestic treatments on the process of extinction after intense training have not been studied. Previous research demonstrated increased spinogenesis in the dorsomedial striatum, but not in the dorsolateral striatum after intense training, indicating that the dorsomedial striatum is involved in the protective effect of intense training. To investigate this issue, male Wistar rats, previously trained with low, moderate, or high levels of foot shock, were used to study the effect of tetrodotoxin inactivation of dorsomedial striatum on memory consolidation and subsequent extinction of inhibitory avoidance. Performance of the task was evaluated during seven extinction sessions. Tetrodotoxin produced a marked deficit of memory consolidation of inhibitory avoidance trained with low and moderate intensities of foot shock, but normal consolidation occurred when a relatively high foot shock was used. The protective effect of intense training was long-lasting, as evidenced by the high resistance to extinction exhibited throughout the extinction sessions. We discuss the possibility that increased dendritic spinogenesis in dorsomedial striatum may underly this protective effect, and how this mechanism may be related to the resilient memory typical of post-traumatic stress disorder (PTSD).

Memory consolidation affects the interplay of place and response navigation

Navigation through space is based on memory representations of landmarks ('place') or movement sequences ('response'). Over time, memory representations transform through consolidation. However, it is unclear how the transformation affects place and response navigation in humans. In the present study, healthy adults navigated to target locations in a virtual maze. The preference for using place and response strategies and the ability to recall place and response memories were tested after a delay of one hour (n = 31), one day (n = 30), or two weeks (n = 32). The different delays captured early-phase synaptic changes, changes after one night of sleep, and long-delay changes due to the reorganization of navigation networks. Our results show that the relative contributions of place and response navigation changed as a function of time. After a short delay of up to one day, participants preferentially used a place strategy and exhibited a high degree of visual landmark exploration. After a longer delay of two weeks, place strategy use decreased significantly. Participants now equally relied on place and response strategy use and increasingly repeated previously taken paths. Further analyses indicate that response strategy use predominantly occurred as a compensatory strategy in the absence of sufficient place memory. Over time, place memory faded before response memory. We suggest that the observed shift from place to response navigation is context-dependent since detailed landmark information, which strongly relied on hippocampal function, decayed faster than sequence information, which required less detail and depended on extra-hippocampal areas. We conclude that changes in place and response navigation likely reflect the reorganization of navigation networks during systems consolidation.

A map of spatial navigation for neuroscience

Spatial navigation has received much attention from neuroscientists, leading to the identification of key brain areas and the discovery of numerous spatially selective cells. Despite this progress, our understanding of how the pieces fit together to drive behavior is generally lacking. We argue that this is partly caused by insufficient communication between behavioral and neuroscientific researchers. This has led the latter to under-appreciate the relevance and complexity of spatial behavior, and to focus too narrowly on characterizing neural representations of space-disconnected from the computations these representations are meant to enable. We therefore propose a taxonomy of navigation processes in mammals that can serve as a common framework for structuring and facilitating interdisciplinary research in the field. Using the taxonomy as a guide, we review behavioral and neural studies of spatial navigation. In doing so, we validate the taxonomy and showcase its usefulness in identifying potential issues with common experimental approaches, designing experiments that adequately target particular behaviors, correctly interpreting neural activity, and pointing to new avenues of research.

Function and biochemistry of the dorsolateral prefrontal cortex during placebo analgesia: how the certainty of prior experiences shapes endogenous pain relief

Prior experiences, conditioning cues, and expectations of improvement are essential for placebo analgesia expression. The dorsolateral prefrontal cortex is considered a key region for converting these factors into placebo responses. Since dorsolateral prefrontal cortex neuromodulation can attenuate or amplify placebo, we sought to investigate dorsolateral prefrontal cortex biochemistry and function in 38 healthy individuals during placebo analgesia. After conditioning participants to expect pain relief from a placebo "lidocaine" cream, we collected baseline magnetic resonance spectroscopy (1H-MRS) at 7 Tesla over the right dorsolateral prefrontal cortex. Following this, functional magnetic resonance imaging scans were collected during which identical noxious heat stimuli were delivered to the control and placebo-treated forearm sites. There was no significant difference in the concentration of gamma-aminobutyric acid, glutamate, Myo-inositol, or N-acetylaspartate at the level of the right dorsolateral prefrontal cortex between placebo responders and nonresponders. However, we identified a significant inverse relationship between the excitatory neurotransmitter glutamate and pain rating variability during conditioning. Moreover, we found placebo-related activation within the right dorsolateral prefrontal cortex and altered functional magnetic resonance imaging coupling between the dorsolateral prefrontal cortex and the midbrain periaqueductal gray, which also correlated with dorsolateral prefrontal cortex glutamate. These data suggest that the dorsolateral prefrontal cortex formulates stimulus-response relationships during conditioning, which are then translated to altered cortico-brainstem functional relationships and placebo analgesia expression.

Chronic alcohol consumption shifts learning strategies and synaptic plasticity from hippocampus to striatum-dependent pathways

Introduction

The hippocampus and striatum have dissociable roles in memory and are necessary for spatial and procedural/cued learning, respectively. Emotionally charged, stressful events promote the use of striatal- over hippocampus-dependent learning through the activation of the amygdala. An emerging hypothesis suggests that chronic consumption of addictive drugs similarly disrupt spatial/declarative memory while facilitating striatum-dependent associative learning. This cognitive imbalance could contribute to maintain addictive behaviors and increase the risk of relapse.

Methods

We first examined, in C57BL/6 J male mice, whether chronic alcohol consumption (CAC) and alcohol withdrawal (AW) might modulate the respective use of spatial vs. single cue-based learning strategies, using a competition protocol in the Barnes maze task. We then performed in vivo electrophysiological studies in freely moving mice to assess learning-induced synaptic plasticity in both the basolateral amygdala (BLA) to dorsal hippocampus (dCA1) and BLA to dorsolateral striatum (DLS) pathways.

Results

We found that both CAC and early AW promote the use of cue-dependent learning strategies, and potentiate plasticity in the BLA → DLS pathway while reducing the use of spatial memory and depressing BLA → dCA1 neurotransmission.

Discussion

These results support the view that CAC disrupt normal hippocampo-striatal interactions, and suggest that targeting this cognitive imbalance through spatial/declarative task training could be of great help to maintain protracted abstinence in alcoholic patients.

Temporal context and latent state inference in the hippocampal splitter signal

The hippocampus is thought to enable the encoding and retrieval of ongoing experience, the organization of that experience into structured representations like contexts, maps, and schemas, and the use of these structures to plan for the future. A central goal is to understand what the core computations supporting these functions are, and how these computations are realized in the collective action of single neurons. A potential access point into this issue is provided by 'splitter cells', hippocampal neurons that fire differentially on the overlapping segment of trajectories that differ in their past and/or future. However, the literature on splitter cells has been fragmented and confusing, owing to differences in terminology, behavioral tasks, and analysis methods across studies. In this review, we synthesize consistent findings from this literature, establish a common set of terms, and translate between single-cell and ensemble perspectives. Most importantly, we examine the combined findings through the lens of two major theoretical ideas about hippocampal function: representation of temporal context and latent state inference. We find that unique signature properties of each of these models are necessary to account for the data, but neither theory, by itself, explains all of its features. Specifically, the temporal gradedness of the splitter signal is strong support for temporal context, but is hard to explain using state models, while its flexibility and task-dependence is naturally accounted for using state inference, but poses a challenge otherwise. These theories suggest a number of avenues for future work, and we believe their application to splitter cells is a timely and informative domain for testing and refining theoretical ideas about hippocampal function.

NMDA receptor blockade in the dorsolateral striatum impairs consolidation but not retrieval of habit memory

The present study investigated whether N-methyl-d-aspartate (NMDA) receptors in the dorsolateral striatum (DLS) mediate consolidation and retrieval of habit memory. Adult male Long-Evans rats were trained in a response learning version of a water plus-maze task in which rats were reinforced to make a habitual and consistent body-turn response at the maze choice point in order to mount a hidden escape platform. Prior research indicates that acquisition, consolidation, and retrieval in this task requires DLS function. The present study consisted of two experiments. In Experiment 1, rats received intra-DLS post-training injections of the NMDA receptor antagonist 2-amino-5- phosphonopentanoic acid (AP5; 2 µg/side) to examine the role of NMDA receptors in consolidation of habit memory. In Experiment 2, different groups of rats received a single pre-retrieval injection of AP5 in the DLS (AP5; 2 µg/side) during the last day of maze training to examine the potential role of NMDA receptors in retrieval of habit memory. Results indicated that post-training intra-DLS AP5 injections impaired memory consolidation. However, administration of AP5 at the same dose that impaired consolidation had no effect on memory retrieval. The findings are consistent with previous research indicating a role for NMDA receptors in the DLS in memory consolidation, and suggest that NMDA-dependent synaptic activity in the DLS may not be a critical component of habit memory retrieval.

Loss of GABA co-transmission from cholinergic neurons impairs behaviors related to hippocampal, striatal, and medial prefrontal cortex functions

Introduction: Altered signaling or function of acetylcholine (ACh) has been reported in various neurological diseases, including Alzheimer's disease, Tourette syndrome, epilepsy among others. Many neurons that release ACh also co-transmit the neurotransmitter gamma-aminobutyrate (GABA) at synapses in the hippocampus, striatum, substantia nigra, and medial prefrontal cortex (mPFC). Although ACh transmission is crucial for higher brain functions such as learning and memory, the role of co-transmitted GABA from ACh neurons in brain function remains unknown. Thus, the overarching goal of this study was to investigate how a systemic loss of GABA co-transmission from ACh neurons affected the behavioral performance of mice. Methods: To do this, we used a conditional knock-out mouse of the vesicular GABA transporter (vGAT) crossed with the ChAT-Cre driver line to selectively ablate GABA co-transmission at ACh synapses. In a comprehensive series of standardized behavioral assays, we compared Cre-negative control mice with Cre-positive vGAT knock-out mice of both sexes. Results: Loss of GABA co-transmission from ACh neurons did not disrupt the animal's sociability, motor skills or sensation. However, in the absence of GABA co-transmission, we found significant alterations in social, spatial and fear memory as well as a reduced reliance on striatum-dependent response strategies in a T-maze. In addition, male conditional knockout (CKO) mice showed increased locomotion. Discussion: Taken together, the loss of GABA co-transmission leads to deficits in higher brain functions and behaviors. Therefore, we propose that ACh/GABA co-transmission modulates neural circuitry involved in the affected behaviors.

Navigation task and action space drive the emergence of egocentric and allocentric spatial representations

In general, strategies for spatial navigation could employ one of two spatial reference frames: egocentric or allocentric. Notwithstanding intuitive explanations, it remains unclear however under what circumstances one strategy is chosen over another, and how neural representations should be related to the chosen strategy. Here, we first use a deep reinforcement learning model to investigate whether a particular type of navigation strategy arises spontaneously during spatial learning without imposing a bias onto the model. We then examine the spatial representations that emerge in the network to support navigation. To this end, we study two tasks that are ethologically valid for mammals—guidance, where the agent has to navigate to a goal location fixed in allocentric space, and aiming, where the agent navigates to a visible cue. We find that when both navigation strategies are available to the agent, the solutions it develops for guidance and aiming are heavily biased towards the allocentric or the egocentric strategy, respectively, as one might expect. Nevertheless, the agent can learn both tasks using either type of strategy. Furthermore, we find that place-cell-like allocentric representations emerge preferentially in guidance when using an allocentric strategy, whereas egocentric vector representations emerge when using an egocentric strategy in aiming. We thus find that alongside the type of navigational strategy, the nature of the task plays a pivotal role in the type of spatial representations that emerge.

Most species rely on navigation in space to find water, food, and mates, as well as to return home. When navigating, humans and animals can use one of two reference frames: one based on stable landmarks in the external environment, such as moving due north and then east, or one centered on oneself, such as moving forward and turning left. However, it remains unclear how these reference frames are chosen and interact in navigation tasks, as well as how they are supported by representations in the brain. We therefore modeled two navigation tasks that would each benefit from using one of these reference frames, and trained an artificial agent to learn to solve them through trial and error. Our results show that when given the choice, the agent leveraged the appropriate reference frame to solve the task, but surprisingly could also use the other reference frame when constrained to do so. We also show that the representations that emerge to enable the agent to solve the tasks exist on a spectrum, and are more complex than commonly thought. These representations reflect both the task and reference frame being used, and provide useful insights for the design of experimental tasks to study the use of navigational strategies.

Chronic corticosterone administration in adolescence enhances dorsolateral striatum-dependent learning in adulthood

Previous evidence indicates a link between early life stress (ELS) in humans and a predisposition to psychopathologies that are characterized in part by maladaptive habitual behaviors. Stress and anxiety influence the relative use of mammalian memory systems implicated in these disorders. Specifically, cognitive memory functions of the hippocampus are typically impaired by stress/anxiety, whereas habit memory functions of the dorsolateral striatum (DLS) are enhanced. A stress/anxiety bias toward habit memory has largely been demonstrated in adult rodents and humans, and the effects of ELS on the later use of DLS-dependent habit memory in adult rodents have not been extensively examined. The present study addressed this question by chronically elevating corticosterone (CORT) during adolescence, and investigated the effects of this treatment on DLS-dependent habit learning in adulthood. In experiment 1, adolescent rats received a single daily injection of either CORT (5 mg/kg) or vehicle (cVEH) over 5 days and then matured undisturbed before training as adults in a DLS-dependent water plus-maze task. Rats administered CORT injections during adolescence displayed a strong trend toward enhanced learning during adulthood relative to vehicle-treated rats. Adolescent CORT administration also increased anxiety-like behavior in adulthood in an elevated plus-maze. In experiment 2, adolescent CORT administration enhanced task acquisition in adulthood, and this effect was blocked by concurrent administration of the glucocorticoid antagonist mifepristone (30 mg/kg). Taken together, these findings suggest that chronic elevation of glucocorticoids during adolescence are sufficient to facilitate habit learning in adulthood, and indicate that glucocorticoid function may be a potential underlying mechanism by which ELS influences subsequent habitual behaviors.

Choosing memory retrieval strategies: A critical role for inhibition in the dentate gyrus

Remembering the location of food is essential for survival. Rodents and humans employ mainly hippocampus-dependent spatial strategies, but when being stressed they shift to striatum-mediated stimulus-based strategies. To investigate underlying brain circuits, we tested mice with a heightened stress susceptibility due to a lack of the GABA-synthetizing enzyme GAD65 (GAD65−/− mice) in a dual solution task. Here, GAD65−/− mice preferred to locate a food reward in an open field via a proximal cue, while their wildtype littermates preferred a spatial strategy. The analysis of cFos co-activation across brain regions and of stress-induced mRNA expression changes of GAD65 pointed towards the hippocampal dorsal dentate gyrus (dDG) as a central structure for mediating stress effects on strategy choices via GAD65. Reducing the GAD65 expression locally in the dDG by a shRNA mediated knock down was sufficient to replicate the phenotype of the global GAD65 knock out and to increase dDG excitability. Using DREADD vectors to specifically interfere with dDG circuit activity during dual solution retrieval but not learning confirmed that the dDG modulates strategy choices and that a balanced excitability of this structure is necessary to establish spatial strategy preference. These data highlight the dDG as a critical hub for choosing between spatial and non-spatial foraging strategies.

•

Stress reduces spatial memory preferences for locating rewards in an open field.

•

GAD65 deficient mice show reduced preferences for spatial memory strategy.

•

Dorsal dentate gyrus knock down of GAD65 is sufficient to reduce spatial strategies.

•

Excitability in the dorsal dentate gyrus modulates retrieval strategy choices.

Progesterone rapidly alters the use of place and response memory during spatial navigation in female rats

17β-Estradiol (E2) and progesterone (P) influence place and response memory in female rats in spatial navigation tasks. Use of these memory systems is associated with the hippocampus and the dorsal striatum, respectively. Injections of E2 result in a well-established bias to use place memory, while much less is understood about the role of P. A total of 120 ovariectomized female rats were tested within a dual-solution T-maze task and treated with either low E2 (n = 24), high E2 (10 μg/kg; n = 24), or high E2 in combination with P (500 μg/kg) at three time points before testing: 15 min (n = 24), 1 h (n = 24), and 4 h (n = 24). Given alone, high E2 biases rats to the use of place memory, but this effect is reversed when P is given 1 h or 4 h before testing. This indicates that P may be playing an inhibitory role in the hippocampus during spatial tasks, which is consistent with past findings. Our findings show that P acts rapidly (within an hour) to affect performance during spatial tasks.

Spatial goal coding in the hippocampal formation

The mammalian hippocampal formation contains several distinct populations of neurons involved in representing self-position and orientation. These neurons, which include place, grid, head direction, and boundary-vector cells, are thought to collectively instantiate cognitive maps supporting flexible navigation. However, to flexibly navigate, it is necessary to also maintain internal representations of goal locations, such that goal-directed routes can be planned and executed. Although it has remained unclear how the mammalian brain represents goal locations, multiple neural candidates have recently been uncovered during different phases of navigation. For example, during planning, sequential activation of spatial cells may enable simulation of future routes toward the goal. During travel, modulation of spatial cells by the prospective route, or by distance and direction to the goal, may allow maintenance of route and goal-location information, supporting navigation on an ongoing basis. As the goal is approached, an increased activation of spatial cells may enable the goal location to become distinctly represented within cognitive maps, aiding goal localization. Lastly, after arrival at the goal, sequential activation of spatial cells may represent the just-taken route, enabling route learning and evaluation. Here, we review and synthesize these and other evidence for goal coding in mammalian brains, relate the experimental findings to predictions from computational models, and discuss outstanding questions and future challenges.

A Quantitative Framework for Identifying Patterns of Route-Use in Animal Movement Data

Animal movement along repeatedly used, “habitual” routes could emerge from a variety of cognitive mechanisms, as well as in response to a diverse set of environmental features. Because of the high conservation value of identifying wildlife movement corridors, there has been extensive work focusing on environmental factors that contribute to the emergence of habitual routes between protected habitats. In parallel, significant work has focused on disentangling the cognitive mechanisms underlying animal route use, as such movement patterns are of fundamental interest to the study of decision making and navigation. We reviewed the types of processes that can generate routine patterns of animal movement, suggested a new methodological workflow for classifying one of these patterns—high fidelity path reuse—in animal tracking data, and compared the prevalence of this pattern across four sympatric species of frugivorous mammals in Panama. We found the highest prevalence of route-use in kinkajous, the only nocturnal species in our study, and propose that further development of this method could help to distinguish the processes underlying the presence of specific routes in animal movement data.

Apparatus design and behavioural testing protocol for the evaluation of spatial working memory in mice through the spontaneous alternation T-maze

Spatial working memory can be assessed in mice through the spontaneous alternation T-maze test. The T-maze is a T-shaped apparatus featuring a stem (start arm) and two lateral goal arms (left and right arms). The procedure is based on the natural tendency of rodents to prefer exploring a novel arm over a familiar one, which induces them to alternate the choice of the goal arm across repeated trials. During the task, in order to successfully alternate choices across trials, an animal has to remember which arm had been visited in the previous trial, which makes spontaneous alternation T-maze an optimal test for spatial working memory. As this test relies on a spontaneous behaviour and does not require rewards, punishments or pre-training, it represents a particularly useful tool for cognitive evaluation, both time-saving and animal-friendly. We describe here in detail the apparatus and the protocol, providing representative results on wild-type healthy mice.

Neural systems and the emotion-memory link

The present brief review for this Special Issue summarizes some of the original research on the emotional modulation of memory. The review begins by highlighting the pioneering research from James L. McGaugh and colleagues demonstrating modulatory effects of post-training drug administration on memory consolidation, in particular the stress hormone epinephrine. The subsequent discovery of a critical role for the basolateral amygdala in emotional modulation of memory is described. Within the context of a multiple systems approach to memory focusing on selective roles for the hippocampus and dorsolateral striatum in cognitive and habit memory, the original studies indicating that robust emotional arousal can bias animals and humans toward the predominant use of habit memory are reviewed. This effect of emotional arousal on the relative use of multiple memory systems depends on a modulatory role of the basolateral amygdala. Finally, we briefly consider how an emotion-induced enhancement of dorsolateral striatal-dependent memory may be relevant to understanding maladaptive habitual behaviors in certain human psychopathologies.

Enhanced Hippocampus-Nidopallium Caudolaterale Connectivity during Route Formation in Goal-Directed Spatial Learning of Pigeons

Goal-directed spatial learning is crucial for the survival of animals, in which the formation of the route from the current location to the goal is one of the central problems. A distributed brain network comprising the hippocampus and prefrontal cortex has been shown to support such capacity, yet it is not fully understood how the most similar brain regions in birds, the hippocampus (Hp) and nidopallium caudolaterale (NCL), cooperate during route formation in goal-directed spatial learning. Hence, we examined neural activity in the Hp-NCL network of pigeons and explored the connectivity dynamics during route formation in a goal-directed spatial task. We found that behavioral changes in spatial learning during route formation are accompanied by modifications in neural patterns in the Hp-NCL network. Specifically, as pigeons learned to solve the task, the spectral power in both regions gradually decreased. Meanwhile, elevated hippocampal theta (5 to 12 Hz) connectivity and depressed connectivity in NCL were also observed. Lastly, the interregional functional connectivity was found to increase with learning, specifically in the theta frequency band during route formation. These results provide insight into the dynamics of the Hp-NCL network during spatial learning, serving to reveal the potential mechanism of avian spatial navigation.