Hippocampus and subregions of the dorsal striatum respond differently to a behavioral strategy change on a spatial navigation task

A role for the dorsolateral striatum in prospective action control

The dorsolateral striatum (DLS) is important for performing actions persistently, even when it becomes suboptimal, reflecting a function that is reflexive and habitual. However, there are also ways in which persistent behaviors can result from a more prospective, planning mode of behavior. To help tease apart these possibilities for DLS function, we trained animals to perform a lever press for reward and then inhibited the DLS in key test phases: as the task shifted from a 1-press to a 3-press rule (upshift), as the task was maintained, as the task shifted back to the one-press rule (downshift), and when rewards came independent of pressing. During DLS inhibition, animals always favored their initially learned strategy to press just once, particularly so during the free-reward period. DLS inhibition surprisingly changed performance speed bidirectionally depending on the task shifts. DLS inhibition thus encouraged habitual behavior, suggesting it could normally help adapt to changing conditions.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

A map of spatial navigation for neuroscience

Spatial navigation has received much attention from neuroscientists, leading to the identification of key brain areas and the discovery of numerous spatially selective cells. Despite this progress, our understanding of how the pieces fit together to drive behavior is generally lacking. We argue that this is partly caused by insufficient communication between behavioral and neuroscientific researchers. This has led the latter to under-appreciate the relevance and complexity of spatial behavior, and to focus too narrowly on characterizing neural representations of space-disconnected from the computations these representations are meant to enable. We therefore propose a taxonomy of navigation processes in mammals that can serve as a common framework for structuring and facilitating interdisciplinary research in the field. Using the taxonomy as a guide, we review behavioral and neural studies of spatial navigation. In doing so, we validate the taxonomy and showcase its usefulness in identifying potential issues with common experimental approaches, designing experiments that adequately target particular behaviors, correctly interpreting neural activity, and pointing to new avenues of research.

Task history dictates how the dorsolateral striatum controls action strategy and vigor

The dorsolateral striatum (DLS) is linked to the learning and honing of action routines. However, the DLS is also important for performing behaviors that have been successful in the past. The learning function can be thought of as prospective, helping to plan ongoing actions to be efficient and often optimal. The performance function is more retrospective, helping the animal continue to behave in a way that had worked previously. How the DLS manages this all is curious. What happens when a learned behavior becomes sub-optimal due to environment changes. In this case, the prospective function of the DLS would cause animals to (adaptively) learn and plan more optimal actions. In contrast, the retrospective function would cause animals to (maladaptively) favor the old behavior. Here we find that, during a change in learned task rules, DLS inhibition causes animals to adjust less rapidly to the new task (and to behave less vigorously) in a 'maladaptive' way. Yet, when the task is changed back to the initially learned rules, DLS inhibition instead causes a rapid and vigorous adjustment of behavior in an 'adaptive' way. These results show that inhibiting the DLS biases behavior towards initially acquired strategies, implying a more retrospective outlook in action selection when the DLS is offline. Thus, an active DLS could encourage planning and learning action routines more prospectively. Moreover, the DLS control over behavior can appear to be either advantageous/flexible or disadvantageous/inflexible depending on task context, and its control over vigor can change depending on task context.

Significant Statement

Basal ganglia networks aid behavioral learning (a prospective planning function) but also favor the use of old behaviors (a retrospective performance function), making it unclear what happens when learned behaviors become suboptimal. Here we inhibit the dorsolateral striatum (DLS) as animals encounter a change in task rules, and again when they shift back to those learned task rules. DLS inhibition reduces adjustment to new task rules (and reduces behavioral vigor), but it increases adjustment back to the initially learned task rules later (and increases vigor). Thus, in both cases, DLS inhibition favored the use of the initially learned behavioral strategy, which could appear either maladaptive or adaptive. We suggest that the DLS might promote a prospective orientation of action control.

Temporal context and latent state inference in the hippocampal splitter signal

The hippocampus is thought to enable the encoding and retrieval of ongoing experience, the organization of that experience into structured representations like contexts, maps, and schemas, and the use of these structures to plan for the future. A central goal is to understand what the core computations supporting these functions are, and how these computations are realized in the collective action of single neurons. A potential access point into this issue is provided by 'splitter cells', hippocampal neurons that fire differentially on the overlapping segment of trajectories that differ in their past and/or future. However, the literature on splitter cells has been fragmented and confusing, owing to differences in terminology, behavioral tasks, and analysis methods across studies. In this review, we synthesize consistent findings from this literature, establish a common set of terms, and translate between single-cell and ensemble perspectives. Most importantly, we examine the combined findings through the lens of two major theoretical ideas about hippocampal function: representation of temporal context and latent state inference. We find that unique signature properties of each of these models are necessary to account for the data, but neither theory, by itself, explains all of its features. Specifically, the temporal gradedness of the splitter signal is strong support for temporal context, but is hard to explain using state models, while its flexibility and task-dependence is naturally accounted for using state inference, but poses a challenge otherwise. These theories suggest a number of avenues for future work, and we believe their application to splitter cells is a timely and informative domain for testing and refining theoretical ideas about hippocampal function.

Lack of action monitoring as a prerequisite for habitual and chunked behavior: Behavioral and neural correlates

We previously reported the rapid development of habitual behavior in a discrete-trials instrumental task in which lever insertion and retraction act as reward-predictive cues delineating sequence execution. Here we asked whether lever cues or performance variables reflective of skill and automaticity might account for habitual behavior in male rats. Behavior in the discrete-trials habit-promoting task was compared with two task variants lacking the sequence-delineating cues of lever extension and retraction. We find that behavior is under goal-directed control in absence of sequence-delineating cues but not in their presence, and that skilled performance does not predict goal-directed vs. habitual behavior. Neural activity recordings revealed an engagement of dorsolateral striatum and a disengagement of dorsomedial striatum during the sequence execution of the habit-promoting task, specifically. Together, these results indicate that sequence delineation cues promote habit and differential engagement of striatal subregions during instrumental responding, a pattern that may reflect cue-elicited behavioral chunking.

Etiopathogenic Models of Psychosis Spectrum Illnesses Must Resolve Four Key Features

Etiopathogenic models for psychosis spectrum illnesses are converging on a number of key processes, such as the influence of specific genes on the synthesis of proteins important in synaptic functioning, alterations in how neurons respond to synaptic inputs and engage in synaptic pruning, and microcircuit dysfunction that leads to more global cortical information processing vulnerabilities. Disruptions in prefrontal operations then accumulate and propagate over time, interacting with environmental factors, developmental processes, and homeostatic mechanisms, eventually resulting in symptoms of psychosis and disability. However, there are 4 key features of psychosis spectrum illnesses that are of primary clinical relevance but have been difficult to assimilate into a single model and have thus far received little direct attention: 1) the bidirectionality of the causal influences for the emergence of psychosis, 2) the catastrophic clinical threshold seen in first episodes of psychosis and why it is irreversible in some individuals, 3) observed biotypes that are neurophysiologically distinct but clinically both convergent and divergent, and 4) a reconciliation of the role of striatal dopaminergic dysfunction with models of prefrontal cortical state instability. In this selective review, we briefly describe these 4 hallmark features and we argue that theoretically driven computational perspectives making use of both algorithmic and neurophysiologic models are needed to reduce this complexity and variability of psychosis spectrum illnesses in a principled manner.

Abnormal innate and learned behavior induced by neuron-microglia miscommunication is related to CA3 reconfiguration

Neuron-microglia communication through the Cx3cr1-Cx3cl1 axis is essential for the development and refinement of neural circuits, which determine their function into adulthood. In the present work we set out to extend the behavioral characterization of Cx3cr1^-/- mice evaluating innate behaviors and spatial navigation, both dependent on hippocampal function. Our results show that Cx3cr1-deficient mice, which show some changes in microglial and synaptic terminals morphology and density, exhibit alterations in activities of daily living and in the rapid encoding of novel spatial information that, nonetheless, improves with training. A neural substrate for these cognitive deficiencies was found in the form of synaptic dysfunction in the CA3 region of the hippocampus, with a marked impact on the mossy fiber (MF) pathway. A network analysis of the CA3 microcircuit reveals the effect of these synaptic alterations on the functional connectivity among CA3 neurons with diminished strength and topological reorganization in Cx3cr1-deficient mice. Neonatal population activity of the CA3 region in Cx3cr1-deficient mice shows a marked reorganization around the giant depolarizing potentials, the first form of network-driven activity of the hippocampus, suggesting that alterations found in adult subjects arise early on in postnatal development, a critical period of microglia-dependent neural circuit refinement. Our results show that interruption of the Cx3cr1-Cx3cl1/neuron-microglia axis leads to changes in CA3 configuration that affect innate and learned behaviors.

Computational validity: using computation to translate behaviours across species

We propose a new conceptual framework (computational validity) for translation across species and populations based on the computational similarity between the information processing underlying parallel tasks. Translating between species depends not on the superficial similarity of the tasks presented, but rather on the computational similarity of the strategies and mechanisms that underlie those behaviours. Computational validity goes beyond construct validity by directly addressing questions of information processing. Computational validity interacts with circuit validity as computation depends on circuits, but similar computations could be accomplished by different circuits. Because different individuals may use different computations to accomplish a given task, computational validity suggests that behaviour should be understood through the subject's point of view; thus, behaviour should be characterized on an individual level rather than a task level. Tasks can constrain the computational algorithms available to a subject and the observed subtleties of that behaviour can provide information about the computations used by each individual. Computational validity has especially high relevance for the study of psychiatric disorders, given the new views of psychiatry as identifying and mediating information processing dysfunctions that may show high inter-individual variability, as well as for animal models investigating aspects of human psychiatric disorders. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Dorsolateral Striatal Task-initiation Bursts Represent Past Experiences More than Future Action Plans

The dorsolateral striatum (DLS) is involved in learning and executing procedural actions. Cell ensembles in the DLS, but not the dorsomedial striatum (DMS), exhibit a burst of firing at the start of a well-learned action sequence ("task-bracketing"). However, it is currently unclear what information is contained in these bursts. Some theories suggest that these bursts should represent the procedural action sequence itself (that they should be about future action chains), whereas others suggest that they should contain representations of the current state of the world, taking into account primarily past information. In addition, the DLS local field potential shows transient bursts of power in the 50 Hz range (γ50) around the time a learned action sequence is initiated. However, it is currently unknown how bursts of activity in DLS cell ensembles and bursts of γ50 power in the DLS local field potential are related to each other. We found that DLS bursts at lap initiation in rats represented recently experienced reward locations more than future procedural actions, indicating that task-initiation DLS bursts contain primarily retrospective, rather than prospective, information to guide procedural actions. Furthermore, representations of past reward locations increased during periods of increased γ50 power in the DLS. There was no evidence of task-initiation bursts, increased γ50 power, or retrospective reward location information in the neighboring dorsomedial striatum. These data support a role for the DLS in model-free theories of procedural decision-making over planned action-chain theories, suggesting that procedural actions derive from representations of the current and recent past.SIGNIFICANCE STATEMENT While it is well-established that the dorsolateral striatum (DLS) plays a critical role in procedural decision-making, open questions remain about the kinds of representations contained in DLS ensemble activity that guide procedural actions. We found that DLS, but not DMS, cell ensembles contained nonlocal representations of past reward locations that appear moments before task-initiation DLS bursts. These retrospective representations were temporally linked to a rise in γ50 power that also preceded the characteristic DLS burst at task-initiation. These results support models of procedural decision-making based on associations between available actions and the current state of the world over models based on planning over action-chains.

What Behavioral Abilities Emerged at Key Milestones in Human Brain Evolution? 13 Hypotheses on the 600-Million-Year Phylogenetic History of Human Intelligence

This paper presents 13 hypotheses regarding the specific behavioral abilities that emerged at key milestones during the 600-million-year phylogenetic history from early bilaterians to extant humans. The behavioral, intellectual, and cognitive faculties of humans are complex and varied: we have abilities as diverse as map-based navigation, theory of mind, counterfactual learning, episodic memory, and language. But these faculties, which emerge from the complex human brain, are likely to have evolved from simpler prototypes in the simpler brains of our ancestors. Understanding the order in which behavioral abilities evolved can shed light on how and why our brains evolved. To propose these hypotheses, I review the available data from comparative psychology and evolutionary neuroscience.

Whole-Brain Mapping of Direct Inputs to Dopamine D1 and D2 Receptor-Expressing Medium Spiny Neurons in the Posterior Dorsomedial Striatum

The posterior dorsomedial striatum (pDMS) is mainly composed of medium spiny neurons (MSNs) expressing either dopamine D1 receptors (D1Rs) or D2Rs. Activation of these two MSN types produces opposing effects on addictive behaviors. However, it remains unclear whether pDMS D1-MSNs or D2-MSNs receive afferent inputs from different brain regions or whether the extrastriatal afferents express distinct dopamine receptors. To assess whether these afferents also contained D1Rs or D2Rs, we generated double transgenic mice, in which D1R-expressing and D2R-expressing neurons were fluorescently labeled. We used rabies virus-mediated retrograde tracing in these mice to perform whole-brain mapping of direct inputs to D1-MSNs or D2-MSNs in the pDMS. We found that D1-MSNs preferentially received inputs from the secondary motor, secondary visual, and cingulate cortices, whereas D2-MSNs received inputs from the primary motor and primary sensory cortices, and the thalamus. We also discovered that the bed nucleus of the stria terminalis (BNST) and the central nucleus of the amygdala (CeA) contained abundant D2R-expressing, but few D1R-expressing, neurons in a triple transgenic mouse model. Remarkably, although limited D1R or D2R expression was observed in extrastriatal neurons that projected to D1-MSNs or D2-MSNs, we found that cortical structures preferentially contained D1R-expressing neurons that projected to D1-MSNs or D2-MSNs, while the thalamus, substantia nigra pars compacta (SNc), and BNST had more D2R-expressing cells that projected to D2-MSNs. Taken together, these findings provide a foundation for future understanding of the pDMS circuit and its role in action selection and reward-based behaviors.

Melamine disrupts spatial reversal learning and learning strategy via inhibiting hippocampal BDNF-mediated neural activity

Although several studies showed adverse neurotoxic effects of melamine on hippocampus (HPC)-dependent learning and reversal learning, the evidence for this mechanism is still unknown. We recently demonstrated that intra-hippocampal melamine injection affected the induction of long-term depression, which is associated with novelty acquisition and memory consolidation. Here, we infused melamine into the HPC of rats, and employed behavioral tests, immunoblotting, immunocytochemistry and electrophysiological methods to sought evidence for its effects on cognitive flexibility. Rats with intra-hippocampal infusion of melamine displayed dose-dependent increase in trials to the criterion in reversal learning, with no locomotion or motivation defect. Compared with controls, melamine-treated rats avoided HPC-dependent place strategy. Meanwhile, the learning-induced BDNF level in the HPC neurons was significantly reduced. Importantly, bilateral intra-hippocampal BDNF infusion could effectively mitigate the suppressive effects of melamine on neural correlate with reversal performance, and rescue the strategy bias and reversal learning deficits. Our findings provide first evidence for the effect of melamine on cognitive flexibility and suggest that the reversal learning deficit is due to the inability to use place strategy. Furthermore, the suppressive effects of melamine on BDNF-mediated neural activity could be the mechanism, thus advancing the understanding of compulsive behavior in melamine-induced and other neuropsychiatric disorders.

Potential roles of the rodent medial prefrontal cortex in conflict resolution between multiple decision-making systems

Mammalian decision-making is mediated by the interaction of multiple, neurally and computationally separable decision systems. Having multiple systems requires a mechanism to manage conflict and converge onto the selection of singular actions. A long history of evidence has pointed to the prefrontal cortex as a central component in processing the interactions between distinct decision systems and resolving conflicts among them. In this chapter we review four theories of how that interaction might occur and identify how the medial prefrontal cortex in the rodent may be involved in each theory. We then present experimental predictions implied by the neurobiological data in the context of each theory as a starting point for future investigation of medial prefrontal cortex and decision-making.

Proceedings of the 2019 annual meeting of the Fetal Alcohol Spectrum Disorders Study Group

The 2019 Fetal Alcohol Spectrum Disorders Study Group (FASDSG) meeting was titled "Computational Approaches to Studying Behavioral Control and Individual Change". The theme was reflected in the presentations of two keynote speakers: A. David Redish, Ph.D., who spoke about computational psychiatry and vulnerabilities in decision-making processes, and Kevin Grimm, Ph.D., who spoke about contemporary machine learning approaches to studying individual change. The conference presented updates from three government agencies, and included short presentations by junior and senior investigators showcasing late-breaking FASD research. The conference was capped by H. Eugene Hoyme, M.D., FACMG, FAAP, the recipient of the 2019 Henry Rosett award for career-long contributions to the field.

Stress drives deliberative tendencies by influencing vicarious trial and error in decision making

Previous studies have reported the effects of stress on decision making. However, the wide range of findings make it difficult to identify the fundamental effects of stress on decision making and, therefore, how stress affects decision making remains unknown. To investigate the influence of stress on decision making, we employed "vicarious trial and error" (VTE), which refers to a rat's behavior of orienting the head toward options at a decision point. VTE is thought to reflect mental simulation for possible options preceding a decision. We examined effects of acute restraint stress on VTE in a T-maze choice task. VTE depended on learning and past reward outcomes. Acute restraint stress before rats ran the T-maze choice task induced VTE, especially in trials with low demand of VTE, and increased the number of head orientations and time spent during each VTE. On the other hand, stress did not affect task performance (probability of advantageous choice) and patterns of behavioral choice (win-stay lose-shift, exploration-exploitation). In addition, stress activated serotonergic and noradrenergic neurons in the dorsal raphe nucleus and locus coeruleus, which are modulators of impulsivity and attentional control in decision making. These results suggest that stress in decision making drives the VTE process, which may lead to deep consideration, over-thinking, and indecisiveness.

Neural substrates of habit

Active reward pursuit is supported by the balance between the cognitive and habitual control of behavior. The cognitive, goal-directed strategy relies on the prospective evaluation of anticipated consequences, which allows behavior to readily adapt when circumstances change. Repetition of successful actions promotes less cognitively taxing habits, in which behavior is automatically executed without prospective consideration. Disruption in either of these behavioral regulatory systems contributes to the symptoms that underlie many psychiatric disorders. Here, I review recently identified neural substrates, at multiple neural levels, that contribute to habits and outline gaps in knowledge that must be addressed to fully understand the neural mechanisms of behavioral control.

Aging is not equal across memory systems

The present experiments compared the effects of aging on learning several hippocampus- and striatum-sensitive tasks in young (3-4 month) and old (24-28 month) male Fischer-344 rats. Across three sets of tasks, aging was accompanied not only by deficits on hippocampal tasks but also by maintained or even enhanced abilities on striatal tasks. On two novel object recognition tasks, rats showed impaired performance on a hippocampal object location task but enhanced performance on a striatal object replacement task. On a dual solution task, young rats predominately used hippocampal solutions and old rats used striatal solutions. In addition, on two maze tasks optimally solved using either hippocampus-sensitive place or striatum-sensitive response strategies, relative to young rats, old rats had impaired learning on the place version but equivalent learning on the response version. Because glucose treatments can reverse deficits in learning and memory across many tasks and contexts, levels of available glucose in the brain may have particular importance in cognitive aging observed across tasks and memory systems. During place learning, training-related rises in extracellular glucose levels were attenuated in the hippocampus of old rats compared to young rats. In contrast, glucose levels in the striatum increased comparably in young and old rats trained on either the place or response task. These extracellular brain glucose responses to training paralleled the impairment in hippocampus-sensitive learning and the sparing of striatum-sensitive learning seen as rats age, suggesting a link between age-related changes in learning and metabolic substrate availability in these brain regions.

Dorsomedial prefrontal cortex and hippocampus represent strategic context even while simultaneously changing representation throughout a task session

Dorsomedial prefrontal cortex (dmPFC) and hippocampus (HPC) are thought to play complementary roles in a spatial working memory and decision-making network, where spatial information from HPC informs representations in dmPFC, and contextual information from dmPFC biases how HPC recalls that information. We recorded simultaneously from neural ensembles in rodent dmPFC and HPC as rats performed a rule-switching task, and found that ensembles in dmPFC and HPC simultaneously encoded task contingencies and other time-varying information. While ensembles in HPC transitioned to represent new contingencies at the same time as rats updated their strategies to be consistent with the new contingency, dmPFC ensembles transitioned earlier. Neural representations of other time-varying information also changed faster in dmPFC than in HPC. Our results suggest that HPC and dmPFC represent contingencies while simultaneously representing other information which changes over time, and that this contextual information is integrated into hippocampal representations more slowly than in dmPFC.

Women at Greater Sexual Risk for STIs/HIV Have a Lower Mesolimbic and Affective Bias Response to Sexual Stimuli

Young adult women in the United States have high rates of sexually transmitted infections, increasing the risk of human immunodeficiency virus (HIV). The underlying neurobiology of behaviors that increase the probability of contracting sexually-transmitted diseases (STIs) and HIV is just beginning to be explored. The current study assessed the link between sexual risk and the brain and behavioral response to sexual cues in emerging adult women. Our hypothesis was that women with more activity in reward/motivational circuitry would report higher sexual risk behaviors and would evidence higher positive affective bias to visual sexual stimuli. Women (n = 52; age = 18–24 years) who had protected sex 100% of the time (n = 17) vs. those who did not (n = 35), in the past 3 months, were compared on their brain response to 500 ms evocative (sex, aversive, food) vs. neutral cues in a blood-oxygen-level-dependent (BOLD) functional magnetic resonance imaging (fMRI) fast event-related design. Based on existing literature, an a priori anatomical “cue-reactive” mask was used to constrain the analyses. Self-reported sexual activity and the affective bias scores to sexual cues were examined as correlates with the brain response to cues. In contrast to our initial hypothesis, the higher sexual risk (Unprotected) group had significantly less activation in mesolimbic brain regions and lower (less positive) affective bias scores to sexual cues compared to the lower risk (Protected) group. As predicted, the brain response was positively correlated with sexual bias. Follow-up analyses showed an effect of partner “risk” (e.g., more vs. less knowledge of partner’s STIs/HIV status). This evidence suggests that women who have protected sex may view sexual-related stimuli more positively, reflected by a neural response in reward/motivational regions and more positive sexual bias scores. In contrast, young women at increased risk for STIs/HIV may feel more negatively about sexual-related stimuli, evidenced by a lower mesolimbic response and a less positive affective bias to sexual cues. These data may help identify young women who are at greatest risk for acquiring STIs and/or HIV, which carries added importance with the availability of new medications that can prevent HIV.

Striatal and hippocampal contributions to flexible navigation in rats and humans

The hippocampus has been firmly established as playing a crucial role in flexible navigation. Recent evidence suggests that dorsal striatum may also play an important role in such goal-directed behaviour in both rodents and humans. Across recent studies, activity in the caudate nucleus has been linked to forward planning and adaptation to changes in the environment. In particular, several human neuroimaging studies have found the caudate nucleus tracks information traditionally associated with that by the hippocampus. In this brief review, we examine this evidence and argue the dorsal striatum encodes the transition structure of the environment during flexible, goal-directed behaviour. We highlight that future research should explore the following: (1) Investigate neural responses during spatial navigation via a biophysically plausible framework explained by reinforcement learning models and (2) Observe the interaction between cortical areas and both the dorsal striatum and hippocampus during flexible navigation.

Multiscale Imaging Reveals Aberrant Functional Connectome Organization and Elevated Dorsal Striatal Arc Expression in Advanced Age

The functional connectome reflects a network architecture enabling adaptive behavior that becomes vulnerable in advanced age. The cellular mechanisms that contribute to altered functional connectivity in old age, however, are not known. Here we used a multiscale imaging approach to link age-related changes in the functional connectome to altered expression of the activity-dependent immediate-early gene Arc as a function of training to multitask on a working memory (WM)/biconditional association task (BAT). Resting-state fMRI data were collected from young and aged rats longitudinally at three different timepoints during cognitive training. After imaging, rats performed the WM/BAT and were immediately sacrificed to examine expression levels of Arc during task performance. Aged behaviorally impaired, but not young, rats had a subnetwork of increased connectivity between the anterior cingulate cortex (ACC) and dorsal striatum (DS) that was correlated with the use of a suboptimal response-based strategy during cognitive testing. Moreover, while young rats had stable rich-club organization across three scanning sessions, the rich-club organization of old rats increased with cognitive training. In a control group of young and aged rats that were longitudinally scanned at similar time intervals, but without cognitive training, ACC-DS connectivity and rich-club organization did not change between scans in either age group. These findings suggest that aberrant large-scale functional connectivity in aged animals is associated with altered cellular activity patterns within individual brain regions.

Distinct recruitment of dorsomedial and dorsolateral striatum erodes with extended training

Hypotheses of striatal orchestration of behavior ascribe distinct functions to striatal subregions, with the dorsolateral striatum (DLS) especially implicated in habitual and skilled performance. Thus neural activity patterns recorded from the DLS, but not the dorsomedial striatum (DMS), should be correlated with habitual and automatized performance. Here, we recorded DMS and DLS neural activity in rats during training in a task promoting habitual lever pressing. Despite improving performance across sessions, clear changes in corresponding neural activity patterns were not evident in DMS or DLS during early training. Although DMS and DLS activity patterns were distinct during early training, their activity was similar following extended training. Finally, performance after extended training was not associated with DMS disengagement, as would be predicted from prior work. These results suggest that behavioral sequences may continue to engage both striatal regions long after initial acquisition, when skilled performance is consolidated.

Hippocampal Theta-Gamma Coupling Reflects State-Dependent Information Processing in Decision Making

During decision making, hippocampal activity encodes information sometimes about present and sometimes about potential future plans. The mechanisms underlying this transition remain unknown. Building on the evidence that gamma oscillations at different frequencies (low gamma [LG], 30–55 Hz; high gamma [HG], 60–90 Hz; and epsilon, 100–140 Hz) reflect inputs from different circuits, we identified how changes in those frequencies reflect different information-processing states. Using a unique noradrenergic manipulation by clonidine, which shifted both neural representations and gamma states, we found that future representations depended on gamma components. These changes were identifiable on each cycle of theta as asymmetries in the theta cycle, which arose from changes within the ratio of LG and HG power and the underlying phases of those gamma rhythms within the theta cycle. These changes in asymmetry of the theta cycle reflected changes in representations of present and future on each theta cycle.

Neuronal representation of environmental boundaries in egocentric coordinates

Movement through space is a fundamental behavior for all animals. Cognitive maps of environments are encoded in the hippocampal formation in an allocentric reference frame, but motor movements that comprise physical navigation are represented within an egocentric reference frame. Allocentric navigational plans must be converted to an egocentric reference frame prior to implementation as overt behavior. Here we describe an egocentric spatial representation of environmental boundaries in the dorsomedial striatum.

The hippocampus represents an allocentric map of space, however, motor movements used for navigation are defined in an egocentric framework. Here, the authors report that dorsomedial striatal neurons exhibit an egocentric representation of the boundaries in the environment.

Acute Low Alcohol Disrupts Hippocampus-Striatum Neural Correlate of Learning Strategy by Inhibition of PKA/CREB Pathway in Rats

The hippocampus and striatum guide place-strategy and response-strategy learning, respectively, and they have dissociable roles in memory systems, which could compensate in case of temporary or permanent damage. Although acute alcohol (AA) treatment had been shown to have adverse effects on hippocampal function, whether it causes the functional compensation and the underlying mechanisms is unknown. In this study, rats treated with a low dose of AA avoided a hippocampus-dependent spatial strategy, instead preferring a striatum-dependent response strategy. Consistently, the learning-induced increase in hippocampal, but not striatal, pCREB was rendered less pronounced due to diminished activity of pPKA, but not pERK or pCaMKII. As rats approached the turn-decision area, Sp-cAMP, a PKA activator, was found to mitigate the inhibitory effect of AA on intra- and cross-structure synchronized neuronal oscillations, and rescue response-strategy bias and spatial learning deficits. Our study provides strong evidence of the critical link between neural couplings and strategy selection. Moreover, the PKA/CREB-signaling pathway is involved in the suppressive effect of AA on neural correlates of place-learning strategy. The novel important evidence provided here shows the functional couplings between the hippocampus and striatum in spatial learning processing and suggests possible avenues for therapeutic intervention.

Hippocampal proBDNF facilitates place learning strategy associated with neural activity in rats

Mature brain-derived neurotrophic factor has been shown to have a promotive effect on synaptic plasticity and spatial memory. The precursor of BDNF (proBDNF) has emerged as a protein against its mature form. However, it is unknown whether and how proBDNF regulates neural excitability and spatial behavior. Through infusion of cleavage-resistant proBDNF or its antibody into HPC, we sought evidence for the influences by employing multiple behavioral tests and recording hippocampal single-unit activity. Our behavioral findings showed that proBDNF induced beneficial effects on spatial learning by facilitating the use of the place strategy and inhibiting the response strategy, including (1) using more place search strategies but less response strategies, and (2) increasing the number of rats in choosing place strategies but not response strategies. Intriguingly, infusion of an anti-proBDNF antibody did not affect rats' training process but rendered the adaption to learning reversal training more difficult, indicating deficits in choosing the proper learning strategy. The training-induced increase in proBDNF promoted the firing rate of pyramidal neurons but not fast-spiking (FS) interneurons. Importantly, endogenous proBDNF facilitated the neural correlate of spatial, but not response, learning behavior. However, the anti-proBDNF antibody effectively reversed the strategy preference and inhibited neural activity. We herein propose that proBDNF exerts pivotal effects on neural excitability and the use of cognitive strategies to facilitate the spatial learning process.

Deliberation and Procedural Automation on a Two-Step Task for Rats

Current theories suggest that decision-making arises from multiple, competing action-selection systems. Rodent studies dissociate deliberation and procedural behavior, and find a transition from procedural to deliberative behavior with experience. However, it remains unknown how this transition from deliberative to procedural control evolves within single trials, or within blocks of repeated choices. We adapted for rats a two-step task which has been used to dissociate model-based from model-free decisions in humans. We found that a mixture of model-based and model-free algorithms was more likely to explain rat choice strategies on the task than either model-based or model-free algorithms alone. This task contained two choices per trial, which provides a more complex and non-discrete per-trial choice structure. This task structure enabled us to evaluate how deliberative and procedural behavior evolved within-trial and within blocks of repeated choice sequences. We found that vicarious trial and error (VTE), a behavioral correlate of deliberation in rodents, was correlated between the two choice points on a given lap. We also found that behavioral stereotypy, a correlate of procedural automation, increased with the number of repeated choices. While VTE at the first choice point decreased [corrected] with the number of repeated choices, VTE at the second choice point did not, and only increased after unexpected transitions within the task. This suggests that deliberation at the beginning of trials may correspond to changes in choice patterns, while mid-trial deliberation may correspond to an interruption of a procedural process.

Neural signatures of reinforcement learning correlate with strategy adoption during spatial navigation

Human navigation is generally believed to rely on two types of strategy adoption, route-based and map-based strategies. Both types of navigation require making spatial decisions along the traversed way although formal computational and neural links between navigational strategies and mechanisms of value-based decision making have so far been underexplored in humans. Here we employed functional magnetic resonance imaging (fMRI) while subjects located different objects in a virtual environment. We then modelled their paths using reinforcement learning (RL) algorithms, which successfully explained decision behavior and its neural correlates. Our results show that subjects used a mixture of route and map-based navigation and their paths could be well explained by the model-free and model-based RL algorithms. Furthermore, the value signals of model-free choices during route-based navigation modulated the BOLD signals in the ventro-medial prefrontal cortex (vmPFC), whereas the BOLD signals in parahippocampal and hippocampal regions pertained to model-based value signals during map-based navigation. Our findings suggest that the brain might share computational mechanisms and neural substrates for navigation and value-based decisions such that model-free choice guides route-based navigation and model-based choice directs map-based navigation. These findings open new avenues for computational modelling of wayfinding by directing attention to value-based decision, differing from common direction and distances approaches.

Regulation of habit formation in the dorsal striatum

Habits are an essential and pervasive component of our daily lives that allow us to efficiently perform routine tasks. But their disruption contributes to the symptoms that underlie many psychiatric diseases. Emerging data are revealing the cellular and molecular mechanisms of habit formation in the dorsal striatum. New data suggest that in both the dorsolateral and dorsomedial striatum histone deacetylase (HDAC) activity acts as a critical negative regulator of the transcriptional processes underlying habit formation. In this review, we discuss this recent work and draw conclusions relevant to the treatment of diseases marked by maladaptive habits.

Habit formation

Habits, both good ones and bad ones, are pervasive in animal behavior. Important frameworks have been developed to understand habits through psychological and neurobiological studies. This work has given us a rich understanding of brain networks that promote habits, and has also helped us to understand what constitutes a habitual behavior as opposed to a behavior that is more flexible and prospective. Mounting evidence from studies using neural recording methods suggests that habit formation is not a simple process. We review this evidence and take the position that habits could be sculpted from multiple dissociable changes in neural activity. These changes occur across multiple brain regions and even within single brain regions. This strategy of classifying components of a habit based on different brain signals provides a potentially useful new way to conceive of disorders that involve overly fixed behaviors as arising from different potential dysfunctions within the brain's habit network.

Vicarious trial and error

When rats come to a decision point, they sometimes pause and look back and forth as if deliberating over the choice; at other times, they proceed as if they have already made their decision. In the 1930s, this pause-and-look behaviour was termed 'vicarious trial and error' (VTE), with the implication that the rat was 'thinking about the future'. The discovery in 2007 that the firing of hippocampal place cells gives rise to alternating representations of each of the potential path options in a serial manner during VTE suggested a possible neural mechanism that could underlie the representations of future outcomes. More-recent experiments examining VTE in rats suggest that there are direct parallels to human processes of deliberative decision making, working memory and mental time travel.

Active Inference, epistemic value, and vicarious trial and error

Balancing habitual and deliberate forms of choice entails a comparison of their respective merits—the former being faster but inflexible, and the latter slower but more versatile. Here, we show that arbitration between these two forms of control can be derived from first principles within an Active Inference scheme. We illustrate our arguments with simulations that reproduce rodent spatial decisions in T-mazes. In this context, deliberation has been associated with vicarious trial and error (VTE) behavior (i.e., the fact that rodents sometimes stop at decision points as if deliberating between choice alternatives), whose neurophysiological correlates are “forward sweeps” of hippocampal place cells in the arms of the maze under consideration. Crucially, forward sweeps arise early in learning and disappear shortly after, marking a transition from deliberative to habitual choice. Our simulations show that this transition emerges as the optimal solution to the trade-off between policies that maximize reward or extrinsic value (habitual policies) and those that also consider the epistemic value of exploratory behavior (deliberative or epistemic policies)—the latter requiring VTE and the retrieval of episodic information via forward sweeps. We thus offer a novel perspective on the optimality principles that engender forward sweeps and VTE, and on their role on deliberate choice.

Contributions of Hippocampus and Striatum to Memory-Guided Behavior Depend on Past Experience

The hippocampal and striatal memory systems are thought to operate independently and in parallel in supporting cognitive memory and habits, respectively. Much of the evidence for this principle comes from double dissociation data, in which damage to brain structure A causes deficits in Task 1 but not Task 2, whereas damage to structure B produces the reverse pattern of effects. Typically, animals are explicitly trained in one task. Here, we investigated whether this principle continues to hold when animals concurrently learn two types of tasks. Rats were trained on a plus maze in either a spatial navigation or a cue-response task (sequential training), whereas a third set of rats acquired both (concurrent training). Subsequently, the rats underwent either sham surgery or neurotoxic lesions of the hippocampus (HPC), medial dorsal striatum (DSM), or lateral dorsal striatum (DSL), followed by retention testing. Finally, rats in the sequential training condition also acquired the novel "other" task. When rats learned one task, HPC and DSL selectively supported spatial navigation and cue response, respectively. However, when rats learned both tasks, HPC and DSL additionally supported the behavior incongruent with the processing style of the corresponding memory system. Thus, in certain conditions, the hippocampal and striatal memory systems can operate cooperatively and in synergism. DSM significantly contributed to performance regardless of task or training procedure. Experience with the cue-response task facilitated subsequent spatial learning, whereas experience with spatial navigation delayed both concurrent and subsequent response learning. These findings suggest that there are multiple operational principles that govern memory networks.

SIGNIFICANCE STATEMENT

Currently, we distinguish among several types of memories, each supported by a distinct neural circuit. The memory systems are thought to operate independently and in parallel. Here, we demonstrate that the hippocampus and the dorsal striatum memory systems operate independently and in parallel when rats learn one type of task at a time, but interact cooperatively and in synergism when rats concurrently learn two types of tasks. Furthermore, new learning is modulated by past experiences. These results can be explained by a model in which independent and parallel information processing that occurs in the separate memory-related neural circuits is supplemented by information transfer between the memory systems at the level of the cortex.

Manipulating Decisiveness in Decision Making: Effects of Clonidine on Hippocampal Search Strategies

Decisiveness is the ability to commit to a decision quickly and efficiently; in contrast, indecision entails the repeated consideration of multiple alternative possibilities. In humans, the α2-adrenergic receptor agonist clonidine increases decisiveness in tasks that require planning through unknown neural mechanisms. In rats, indecision is manifested as reorienting behaviors at choice points (vicarious trial and error [VTE]), during which hippocampal representations alternate between prospective options. To determine whether the increase in decisiveness driven by clonidine also entails changes in hippocampal search processes, we compared the effect of clonidine on spatial representations in hippocampal neural ensembles as rats passed through a T-shaped decision point. Consistent with previous experiments, hippocampal representations reflected both chosen and unchosen paths during VTE events under saline control conditions. Also, consistent with previous experiments, hippocampal representations reflected the chosen path more than the unchosen path when the animal did not show VTE at the choice point. Injection of clonidine suppressed the spatial representation of the unchosen path at the choice point on VTE laps and hastened the differentiation of spatial representations of the chosen path from the unchosen path on non-VTE laps to appear before reaching the choice point. These results suggest that the decisiveness seen under clonidine is due to limited exploration of potential options in hippocampus, and suggest novel roles for noradrenaline as a modulator of the hippocampal search processes. Significance statement: Clonidine, an α2-adrenergic receptor agonist, which decreases the level of noradrenaline in vivo, has an interesting effect in humans and other animals: it makes them more decisive. However, the mechanisms by which clonidine makes them more decisive remain unknown. Researchers have speculated that clonidine limits the amount of mental search that subjects do when planning options. We test this hypothesis by measuring the mental search strategy in rats through hippocampal recordings. We find that clonidine limits the options searched by rats, suggesting that noradrenaline also plays a role in balancing exploration and exploitation in internally simulated behaviors, similar to its role in balancing exploration and exploitation in external behaviors.

Habit formation coincides with shifts in reinforcement representations in the sensorimotor striatum

Evaluating outcomes of behavior is a central function of the striatum. In circuits engaging the dorsomedial striatum, sensitivity to goal value is accentuated during learning, whereas outcome sensitivity is thought to be minimal in the dorsolateral striatum and its habit-related corticostriatal circuits. However, a distinct population of projection neurons in the dorsolateral striatum exhibits selective sensitivity to rewards. Here, we evaluated the outcome-related signaling in such neurons as rats performed an instructional T-maze task for two rewards. As the rats formed maze-running habits and then changed behavior after reward devaluation, we detected outcome-related spike activity in 116 units out of 1,479 recorded units. During initial training, nearly equal numbers of these units fired preferentially either after rewarded runs or after unrewarded runs, and the majority were responsive at only one of two reward locations. With overtraining, as habits formed, firing in nonrewarded trials almost disappeared, and reward-specific firing declined. Thus error-related signaling was lost, and reward signaling became generalized. Following reward devaluation, in an extinction test, postgoal activity was nearly undetectable, despite accurate running. Strikingly, when rewards were then returned, postgoal activity reappeared and recapitulated the original early response pattern, with nearly equal numbers responding to rewarded and unrewarded runs and to single rewards. These findings demonstrate that outcome evaluation in the dorsolateral striatum is highly plastic and tracks stages of behavioral exploration and exploitation. These signals could be a new target for understanding compulsive behaviors that involve changes to dorsal striatum function.