A dual operator view of habitual behavior reflecting cortical and striatal dynamics

Striosomes Target Nigral Dopamine-Containing Neurons via Direct-D1 and Indirect-D2 Pathways Paralleling Classic Direct-Indirect Basal Ganglia Systems

Balanced activity of canonical direct D1 and indirect D2 basal ganglia pathways is considered a core requirement for normal movement, and their imbalance is an etiologic factor in movement and neuropsychiatric disorders. We present evidence for a conceptually equivalent pair of direct-D1 and indirect-D2 pathways that arise from striatal projection neurons (SPNs) of the striosome compartment rather than from SPNs of the matrix, as do the canonical pathways. These S-D1 and S-D2 striosomal pathways target substantia nigra dopamine-containing neurons instead of basal ganglia motor output nuclei. They modulate movement oppositely to the modulation by the canonical pathways: S-D1 is inhibitory and S-D2 is excitatory. The S-D1 and S-D2 circuits likely influence motivation for learning and action, complementing and reorienting canonical pathway modulation. A major conceptual reformulation of the classic direct-indirect pathway model of basal ganglia function is needed, as well as reconsideration of the effects of D2-targeting therapeutic drugs.

An Improved Dyna-Q Algorithm Inspired by the Forward Prediction Mechanism in the Rat Brain for Mobile Robot Path Planning

The traditional Model-Based Reinforcement Learning (MBRL) algorithm has high computational cost, poor convergence, and poor performance in robot spatial cognition and navigation tasks, and it cannot fully explain the ability of animals to quickly adapt to environmental changes and learn a variety of complex tasks. Studies have shown that vicarious trial and error (VTE) and the hippocampus forward prediction mechanism in rats and other mammals can be used as key components of action selection in MBRL to support "goal-oriented" behavior. Therefore, we propose an improved Dyna-Q algorithm inspired by the forward prediction mechanism of the hippocampus to solve the above problems and tackle the exploration-exploitation dilemma of Reinforcement Learning (RL). This algorithm alternately presents the potential path in the future for mobile robots and dynamically adjusts the sweep length according to the decision certainty, so as to determine action selection. We test the performance of the algorithm in a two-dimensional maze environment with static and dynamic obstacles, respectively. Compared with classic RL algorithms like State-Action-Reward-State-Action (SARSA) and Dyna-Q, the algorithm can speed up spatial cognition and improve the global search ability of path planning. In addition, our method reflects key features of how the brain organizes MBRL to effectively solve difficult tasks such as navigation, and it provides a new idea for spatial cognitive tasks from a biological perspective.

Social experience in adolescence shapes prefrontal cortex structure and function in adulthood

During adolescence, the prefrontal cortex (PFC) undergoes dramatic reorganization. PFC development is profoundly influenced by the social environment, disruptions to which may prime the emergence of psychopathology across the lifespan. We investigated the neurobehavioral consequences of isolation experienced in adolescence in mice, and in particular, the long-term consequences that were detectable even despite normalization of the social milieu. Isolation produced biases toward habit-like behavior at the expense of flexible goal seeking, plus anhedonic-like reward deficits. Behavioral phenomena were accompanied by neuronal dendritic spine over-abundance and hyper-excitability in the ventromedial PFC (vmPFC), which was necessary for the expression of isolation-induced habits and sufficient to trigger behavioral inflexibility in socially reared controls. Isolation activated cytoskeletal regulatory pathways otherwise suppressed during adolescence, such that repression of constituent elements prevented long-term isolation-induced neurosequelae. Altogether, our findings unveil an adolescent critical period and multi-model mechanism by which social experiences facilitate prefrontal cortical maturation.

Role of the basal ganglia in innate and learned behavioural sequences

Integrating individual actions into coherent, organised behavioural units, a process called chunking, is a fundamental, evolutionarily conserved process that renders actions automatic. In vertebrates, evidence points to the basal ganglia - a complex network believed to be involved in action selection - as a key component of action sequence encoding, although the underlying mechanisms are only just beginning to be understood. Central pattern generators control many innate automatic behavioural sequences that form some of the most basic behaviours in an animal's repertoire, and in vertebrates, brainstem and spinal pattern generators are under the control of higher order structures such as the basal ganglia. Evidence suggests that the basal ganglia play a crucial role in the concatenation of simpler behaviours into more complex chunks, in the context of innate behavioural sequences such as chain grooming in rats, as well as sequences in which innate capabilities and learning interact such as birdsong, and sequences that are learned from scratch, such as lever press sequences in operant behaviour. It has been proposed that the role of the striatum, the largest input structure of the basal ganglia, might lie in selecting and allowing the relevant central pattern generators to gain access to the motor system in the correct order, while inhibiting other behaviours. As behaviours become more complex and flexible, the pattern generators seem to become more dependent on descending signals. Indeed, during learning, the striatum itself may adopt the functional characteristics of a higher order pattern generator, facilitated at the microcircuit level by striatal neuropeptides.

Effects of Maternal Separation on Effort-based Responding for Sucrose Are Associated with c-Fos Expression in the Nucleus Accumbens Core

In both people and animals, exposure to adverse experiences early in life can alter neurodevelopment and lead to long-term behavioral effects, including effects on reward processing. In the current study, we use a well-validated rodent model of maternal neglect, maternal separation (MS), to investigate the impact of early life adversity on reward learning and motivation and identify associated modifications in cellular activation in reward-relevant areas. Litters of Long-Evans rats were separated from the dam for either 15 min (brief) or 180 min (prolonged)/day from postnatal day (PND)2 to PND14. As adults, offspring were trained to lever press for a sucrose pellet using fixed ratio (FR) schedules and motivation was tested using a progressive ratio (PR) schedule over 10 daily sessions to assess sustained effects on effort-based responding. Immunohistochemical staining for c-Fos was conducted in a subset of animals that underwent an additional PR session. While there were no effects on reward learning, both MS180 males and females demonstrated increased effort-based responding on the first day of PR testing, while only MS180 males demonstrated a sustained increase in effort across all 10 days. MS180-induced changes in c-Fos expression in the dorsal and ventral striatum were observed, with subregion-specific effects along the rostrocaudal axis. Moreover, regression analyses suggest that motivated responding for a sucrose food reward in MS180-exposed, but not MS15-exposed animals, was associated with increased c-Fos expression in the rostral nucleus accumbens core. These findings implicate specific striatal regions in sex-specific modulation of sustained effort-based reward behavior following early life adversity.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

Positive correlation between measures of habitual responding and motivated responding in mice

Habit and motivation are thought to be separate processes, with motivated behavior often considered to be goal directed, whereas habits are defined by the absence of goal-directed control over behavior. However, there has been increasing interrogation of the binary nature of habitual versus goal-directed behavior. Furthermore, although drug and alcohol exposure can promote the formation of habits, drug seeking itself can also be highly flexible, pointing toward the need for complex consideration of the parallel processes that drive behavior. The goal of the current study was to determine whether there was a relation between motivation-as measured by progressive ratio-and habit-as measured by contingency degradation-and whether this relation was affected by ethanol exposure history and sex. The results showed that these measures were positively correlated such that greater contingency insensitivity was associated with achieving higher break points on the progressive-ratio task. However, this relation depended on reinforcement schedule history, ethanol exposure history, and sex. These point to potential relations between measures of habit and motivation and stress the importance of carefully parsing behavioral findings and assays. These findings are also expected to inform future substance use research, as drug history may affect these relations.

Multiple Memory Subsystems: Reconsidering Memory in the Mind and Brain

The multiple-memory-systems framework-that distinct types of memory are supported by distinct brain systems-has guided learning and memory research for decades. However, recent work challenges the one-to-one mapping between brain structures and memory types central to this taxonomy, with key memory-related structures supporting multiple functions across substructures. Here we integrate cross-species findings in the hippocampus, striatum, and amygdala to propose an updated framework of multiple memory subsystems (MMSS). We provide evidence for two organizational principles of the MMSS theory: First, opposing memory representations are colocated in the same brain structures; second, parallel memory representations are supported by distinct structures. We discuss why this burgeoning framework has the potential to provide a useful revision of classic theories of long-term memory, what evidence is needed to further validate the framework, and how this novel perspective on memory organization may guide future research.

Layer 5 Intratelencephalic Neurons in the Motor Cortex Stably Encode Skilled Movement

The primary motor cortex (M1) and the dorsal striatum play a critical role in motor learning and the retention of learned behaviors. Motor representations of corticostriatal ensembles emerge during motor learning. In the coordinated reorganization of M1 and the dorsal striatum for motor learning, layer 5a (L5a) which connects M1 to the ipsilateral and contralateral dorsal striatum, should be a key layer. Although M1 L5a neurons represent movement-related activity in the late stage of learning, it is unclear whether the activity is retained as a memory engram. Here, using Tlx3-Cre male transgenic mice, we conducted two-photon calcium imaging of striatum-projecting L5a intratelencephalic (IT) neurons in forelimb M1 during late sessions of a self-initiated lever-pull task and in sessions after 6 d of nontraining following the late sessions. We found that trained male animals exhibited stable motor performance before and after the nontraining days. At the same time, we found that M1 L5a IT neurons strongly represented the well-learned forelimb movement but not uninstructed orofacial movements. A subset of M1 L5a IT neurons consistently coded the well-learned forelimb movement before and after the nontraining days. Inactivation of M1 IT neurons after learning impaired task performance when the lever was made heavier or when the target range of the pull distance was narrowed. These results suggest that a subset of M1 L5a IT neurons continuously represent skilled movement after learning and serve to fine-tune the kinematics of well-learned movement.SIGNIFICANCE STATEMENT Motor memory persists even when it is not used for a while. IT neurons in L5a of the M1 gradually come to represent skilled forelimb movements during motor learning. However, it remains to be determined whether these changes persist over a long period and how these neurons contribute to skilled movements. Here, we show that a subset of M1 L5a IT neurons retain information for skilled forelimb movements even after nontraining days. Furthermore, suppressing the activity of these neurons during skilled forelimb movements impaired behavioral stability and adaptability. Our results suggest the importance of M1 L5a IT neurons for tuning skilled forelimb movements over a long period.

Subunit-selective PI3-kinase control of action strategies in the medial prefrontal cortex

PI3-kinase (PI3K) is an intracellular signaling complex that is stimulated upon cocaine exposure and linked with the behavioral consequences of cocaine. We recently genetically silenced the PI3K p110β subunit in the medial prefrontal cortex following repeated cocaine in mice, reinstating the capacity of these mice to engage in prospective goal-seeking behavior. In the present short report, we address two follow-up hypotheses: 1) The control of decision-making behavior by PI3K p110β is attributable to neuronal signaling, and 2) PI3K p110β in the healthy (i.e., drug-naïve) medial prefrontal cortex has functional consequences in the control of reward-related decision-making strategies. In Experiment 1, we found that silencing neuronal p110β improved action flexibility following cocaine. In Experiment 2, we reduced PI3K p110β in drug-naïve mice that were extensively trained to respond for food reinforcers. Gene silencing caused mice to abandon goal-seeking strategies, unmasking habit-based behaviors that were propelled by interactions with the nucleus accumbens. Thus, PI3K control of goal-directed action strategies appears to act in accordance with an inverted U-shaped function, with "too much" (following cocaine) or "too little" (following p110β subunit silencing) obstructing goal seeking and causing mice to defer to habit-like response sequences.

A map of spatial navigation for neuroscience

Spatial navigation has received much attention from neuroscientists, leading to the identification of key brain areas and the discovery of numerous spatially selective cells. Despite this progress, our understanding of how the pieces fit together to drive behavior is generally lacking. We argue that this is partly caused by insufficient communication between behavioral and neuroscientific researchers. This has led the latter to under-appreciate the relevance and complexity of spatial behavior, and to focus too narrowly on characterizing neural representations of space-disconnected from the computations these representations are meant to enable. We therefore propose a taxonomy of navigation processes in mammals that can serve as a common framework for structuring and facilitating interdisciplinary research in the field. Using the taxonomy as a guide, we review behavioral and neural studies of spatial navigation. In doing so, we validate the taxonomy and showcase its usefulness in identifying potential issues with common experimental approaches, designing experiments that adequately target particular behaviors, correctly interpreting neural activity, and pointing to new avenues of research.

Revealing abrupt transitions from goal-directed to habitual behavior

A fundamental tenet of animal behavior is that decision-making involves multiple 'controllers.' Initially, behavior is goal-directed, driven by desired outcomes, shifting later to habitual control, where cues trigger actions independent of motivational state. Clark Hull's question from 1943 still resonates today: "Is this transition abrupt, or is it gradual and progressive?"1 Despite a century-long belief in gradual transitions, this question remains unanswered2,3 as current methods cannot disambiguate goal-directed versus habitual control in real-time. Here, we introduce a novel 'volitional engagement' approach, motivating animals by palatability rather than biological need. Offering less palatable water in the home cage4,5 reduced motivation to 'work' for plain water in an auditory discrimination task when compared to water-restricted animals. Using quantitative behavior and computational modeling6, we found that palatability-driven animals learned to discriminate as quickly as water-restricted animals but exhibited state-like fluctuations when responding to the reward-predicting cue-reflecting goal-directed behavior. These fluctuations spontaneously and abruptly ceased after thousands of trials, with animals now always responding to the reward-predicting cue. In line with habitual control, post-transition behavior displayed motor automaticity, decreased error sensitivity (assessed via pupillary responses), and insensitivity to outcome devaluation. Bilateral lesions of the habit-related dorsolateral striatum7 blocked transitions to habitual behavior. Thus, 'volitional engagement' reveals spontaneous and abrupt transitions from goal-directed to habitual behavior, suggesting the involvement of a higher-level process that arbitrates between the two.

Distinct mesoscale cortical dynamics encode search strategies during spatial navigation

Spatial navigation is a complex cognitive process that involves neural computations in distributed regions of the brain. Little is known about how cortical regions are coordinated when animals navigate novel spatial environments or how that coordination changes as environments become familiar. We recorded mesoscale calcium (Ca2+) dynamics across large swathes of the dorsal cortex in mice solving the Barnes maze, a 2D spatial navigation task where mice used random, serial, and spatial search strategies to navigate to the goal. Cortical dynamics exhibited patterns of repeated calcium activity with rapid and abrupt shifts between cortical activation patterns at sub-second time scales. We used a clustering algorithm to decompose the spatial patterns of cortical calcium activity in a low dimensional state space, identifying 7 states, each corresponding to a distinct spatial pattern of cortical activation, sufficient to describe the cortical dynamics across all the mice. When mice used serial or spatial search strategies to navigate to the goal, the frontal regions of the cortex were reliably activated for prolonged durations of time (> 1s) shortly after trial initiation. These frontal cortex activation events coincided with mice approaching the edge of the maze from the center and were preceded by temporal sequences of cortical activation patterns that were distinct for serial and spatial search strategies. In serial search trials, frontal cortex activation events were preceded by activation of the posterior regions of the cortex followed by lateral activation of one hemisphere. In spatial search trials, frontal cortical events were preceded by activation of posterior regions of the cortex followed by broad activation of the lateral regions of the cortex. Our results delineated cortical components that differentiate goal- and non-goal oriented spatial navigation strategies.

Amygdala AVPR1A mediates susceptibility to chronic social isolation in females

Females are more sensitive to social exclusion, which could contribute to their heightened susceptibility to anxiety disorders. Chronic social isolation stress (CSIS) for at least 7 weeks after puberty induces anxiety-related behavioral adaptations in female mice. Here, we show that Arginine vasopressin receptor 1a ( Avpr1a )-expressing neurons in the central nucleus of the amygdala (CeA) mediate these sex-specific effects, in part, via projections to the caudate putamen. Loss of function studies demonstrate that AVPR1A signaling in the CeA is required for effects of CSIS on anxiety-related behaviors in females but has no effect in males or group housed females. This sex-specificity is mediated by AVP produced by a subpopulation of neurons in the posterodorsal medial nucleus of the amygdala that project to the CeA. Estrogen receptor alpha signaling in these neurons also contributes to preferential sensitivity of females to CSIS. These data support new therapeutic applications for AVPR1A antagonists in women.

Elevated prefrontal dopamine interferes with the stress-buffering properties of behavioral control in female rats

Stress-linked disorders are more prevalent in women than in men and differ in their clinical presentation. Thus, investigating sex differences in factors that promote susceptibility or resilience to stress outcomes, and the circuit elements that mediate their effects, is important. In male rats, instrumental control over stressors engages a corticostriatal system involving the prelimbic cortex (PL) and dorsomedial striatum (DMS) that prevent many of the sequelae of stress exposure. Interestingly, control does not buffer against stress outcomes in females, and here, we provide evidence that the instrumental controlling response in females is supported instead by the dorsolateral striatum (DLS). Additionally, we used in vivo microdialysis, fluorescent in situ hybridization, and receptor subtype pharmacology to examine the contribution of prefrontal dopamine (DA) to the differential impact of behavioral control. Although both sexes preferentially expressed D1 receptor mRNA in PL GABAergic neurons, there were robust sex differences in the dynamic properties of prefrontal DA during controllable stress. Behavioral control potently attenuated stress-induced DA efflux in males, but not females, who showed a sustained DA increase throughout the entire stress session. Importantly, PL D1 receptor blockade (SCH 23390) shifted the proportion of striatal activity from the DLS to the DMS in females and produced the protective effects of behavioral control. These findings suggest a sex-selective mechanism in which elevated DA in the PL biases instrumental responding towards prefrontal-independent striatal circuitry, thereby eliminating the protective impact of coping with stress.

Differential processing of decision information in subregions of rodent medial prefrontal cortex

Decision-making involves multiple cognitive processes requiring different aspects of information about the situation at hand. The rodent medial prefrontal cortex (mPFC) has been hypothesized to be central to these abilities. Functional studies have sought to link specific processes to specific anatomical subregions, but past studies of mPFC have yielded controversial results, leaving the precise nature of mPFC function unclear. To settle this debate, we recorded from the full dorso-ventral extent of mPFC in each of 8 rats, as they performed a complex economic decision task. These data revealed four distinct functional domains within mPFC that closely mirrored anatomically identified subregions, including novel evidence to divide prelimbic cortex into dorsal and ventral components. We found that dorsal aspects of mPFC (ACC, dPL) were more involved in processing information about active decisions, while ventral aspects (vPL, IL) were more engaged in motivational factors.

Cilia in the Striatum Mediate Timing-Dependent Functions

Almost all brain cells contain cilia, antennae-like microtubule-based organelles. Yet, the significance of cilia, once considered vestigial organelles, in the higher-order brain functions is unknown. Cilia act as a hub that senses and transduces environmental sensory stimuli to generate an appropriate cellular response. Similarly, the striatum, a brain structure enriched in cilia, functions as a hub that receives and integrates various types of environmental information to drive appropriate motor response. To understand cilia's role in the striatum functions, we used loxP/Cre technology to ablate cilia from the dorsal striatum of male mice and monitored the behavioral consequences. Our results revealed an essential role for striatal cilia in the acquisition and brief storage of information, including learning new motor skills, but not in long-term consolidation of information or maintaining habitual/learned motor skills. A fundamental aspect of all disrupted functions was the "time perception/judgment deficit." Furthermore, the observed behavioral deficits form a cluster pertaining to clinical manifestations overlapping across psychiatric disorders that involve the striatum functions and are known to exhibit timing deficits. Thus, striatal cilia may act as a calibrator of the timing functions of the basal ganglia-cortical circuit by maintaining proper timing perception. Our findings suggest that dysfunctional cilia may contribute to the pathophysiology of neuro-psychiatric disorders, as related to deficits in timing perception.

Brain recognition of previously learned versus novel temporal sequences: a differential simultaneous processing

Memory for sequences is a central topic in neuroscience, and decades of studies have investigated the neural mechanisms underlying the coding of a wide array of sequences extended over time. Yet, little is known on the brain mechanisms underlying the recognition of previously memorized versus novel temporal sequences. Moreover, the differential brain processing of single items in an auditory temporal sequence compared to the whole superordinate sequence is not fully understood. In this magnetoencephalography (MEG) study, the items of the temporal sequence were independently linked to local and rapid (2–8 Hz) brain processing, while the whole sequence was associated with concurrent global and slower (0.1–1 Hz) processing involving a widespread network of sequentially active brain regions. Notably, the recognition of previously memorized temporal sequences was associated to stronger activity in the slow brain processing, while the novel sequences required a greater involvement of the faster brain processing. Overall, the results expand on well-known information flow from lower- to higher order brain regions. In fact, they reveal the differential involvement of slow and faster whole brain processing to recognize previously learned versus novel temporal information.

Making habits measurable beyond what they are not: A focus on associative dual-process models

Habits are the subject of intense international research. Under the associative dual-process model the outcome devaluation paradigm has been used extensively to classify behaviours as being either goal-directed (sensitive to shifts in the value of associated outcomes) or habitual (triggered by stimuli without anticipation of consequences). This has proven to be a useful framework for studying the neurobiology of habit and relevance of habits in clinical psychopathology. However, in recent years issues have been raised about this rather narrow definition of habits in comparison to habitual behaviour experienced in the real world. Specifically, defining habits as the absence of goal-directed control, the very specific set-ups required to demonstrate habit experimentally and the lack of direct evidence for habits as stimulus-response behaviours are viewed as problematic. In this review paper we address key critiques that have been raised about habit research within the framework of the associative dual-process model. We then highlight novel research approaches studying different features of habits with methods that expand beyond traditional paradigms.

Development and Validation of a Daily Habit Scale

Habits are defined as automatic behaviours triggered by cues and performed without awareness. They are difficult to control and mentally efficient, which contrasts with goal-directed behaviour, which is characterised by active thought, high computational effort, and the ability to modify this behaviour in response to a changing environment and contextual demands. Habits are not only defined by the frequency with which a behaviour is performed but represent a complex construct that also includes the strength and automaticity of the habitual behaviour. We report here the development and validation of a Daily Habit Scale (DHS) to assess the frequency, automaticity, and strength of daily habits in healthy individuals. Item reduction based on factor analysis resulted in a scale with 38 items grouped into eight factors explaining 52.91% of the variance. The DHS showed very good internal consistency (Cronbach alpha = 0.738) and test-retest reliability (Intraclass correlation coefficient = 0.892, p<0.001) as well as convergent and divergent reliability compared to other scales measuring habits. We found a significant effect of age, gender, anxiety, and depression on the DHS. Considering certain limitations of the DHS, such as not considering the context of performance of habits, and the absence of certain items, such as transportation use, the results of this study suggest that DHS is a reliable and valid measure of daily habits that can be used by both clinicians and researchers as a measure of daily habits.

Oscillation/Coincidence-Detection Models of Reward-Related Timing in Corticostriatal Circuits

The major tenets of beat-frequency/coincidence-detection models of reward-related timing are reviewed in light of recent behavioral and neurobiological findings. This includes the emphasis on a core timing network embedded in the motor system that is comprised of a corticothalamic-basal ganglia circuit. Therein, a central hub provides timing pulses (i.e., predictive signals) to the entire brain, including a set of distributed satellite regions in the cerebellum, cortex, amygdala, and hippocampus that are selectively engaged in timing in a manner that is more dependent upon the specific sensory, behavioral, and contextual requirements of the task. Oscillation/coincidence-detection models also emphasize the importance of a tuned ‘perception’ learning and memory system whereby target durations are detected by striatal networks of medium spiny neurons (MSNs) through the coincidental activation of different neural populations, typically utilizing patterns of oscillatory input from the cortex and thalamus or derivations thereof (e.g., population coding) as a time base. The measure of success of beat-frequency/coincidence-detection accounts, such as the Striatal Beat-Frequency model of reward-related timing (SBF), is their ability to accommodate new experimental findings while maintaining their original framework, thereby making testable experimental predictions concerning diagnosis and treatment of issues related to a variety of dopamine-dependent basal ganglia disorders, including Huntington’s and Parkinson’s disease.

Hierarchical Reinforcement Learning, Sequential Behavior, and the Dorsal Frontostriatal System

To effectively behave within ever-changing environments, biological agents must learn and act at varying hierarchical levels such that a complex task may be broken down into more tractable subtasks. Hierarchical reinforcement learning (HRL) is a computational framework that provides an understanding of this process by combining sequential actions into one temporally extended unit called an option. However, there are still open questions within the HRL framework, including how options are formed and how HRL mechanisms might be realized within the brain. In this review, we propose that the existing human motor sequence literature can aid in understanding both of these questions. We give specific emphasis to visuomotor sequence learning tasks such as the discrete sequence production task and the M × N (M steps × N sets) task to understand how hierarchical learning and behavior manifest across sequential action tasks as well as how the dorsal cortical-subcortical circuitry could support this kind of behavior. This review highlights how motor chunks within a motor sequence can function as HRL options. Furthermore, we aim to merge findings from motor sequence literature with reinforcement learning perspectives to inform experimental design in each respective subfield.

Study of the acute and repeated dose 28-day oral toxicity in mice treated with PT-31, a molecule with potential antipsychotic profile

Schizophrenia is a psychiatric disorder that affects 1% of the world population and is treated with antipsychotics, which may induce important biochemical and hematological alterations. Since it is necessary to verify the safety of new molecules with antipsychotic potential, the present study aimed to evaluate the oral toxicity of PT-31, a putative α₂-adrenoreceptor agonist, after acute (2000 mg/kg) and repeated doses (28 days) gavage treatment, in three different doses: minimum effective dose in animal models (10 mg/kg), twice the dose (20 mg/kg), and four times the dose (40 mg/kg), as recommended by the OECD guidelines. Balb/C female adult mice were used, and biochemical, hematological, and histopathological analyses were performed. PT-31 10 and 20 mg/kg did not cause biochemical alterations related to hepatic and renal toxicity, and neither altered glycemic and lipid profiles. The preclinical dose of PT-31 also did not promote mice histopathological changes in the liver, kidney, and brain. In the hematimetric parameters, PT-31 only increased HGB at 20 mg/kg, and MCH and MCHC at 40 mg/kg. However, all the tested doses of PT-31 showed platelet increase, which must be better investigated. Therefore, further studies are needed to investigate the safety of PT-31 as a potential antipsychotic drug.

Spatiotemporal reorganization of corticostriatal networks encodes motor skill learning

Motor skill learning requires the activity of the dorsal striatum, with a differential global implication of the dorsomedial and dorsolateral territories. We investigate here whether and how specific striatal neurons encode the acquisition and consolidation of a motor skill. Using ex vivo two-photon calcium imaging after rotarod training, we report that highly active (HA) striatal populations arise from distinct spatiotemporal reorganization in the dorsomedial (DMS) and dorsolateral (DLS) striatum networks and are correlated with learning performance. The DMS overall activity decreases in early training, with few and sparsely distributed HA cells, while the DLS shows a progressive and long-lasting formation of HA cell clusters. These reorganizations result from reinforcement of synaptic connections to the DMS and anatomical rearrangements to the DLS. Targeted silencing of DMS or DLS HA cells with the cFos-TRAP strategy strongly impairs individual performance. Our data reveal that discrete domains of striatal populations encode acquisition and long-lasting retention of a motor skill.

Towards an animal model of consciousness based on the platform theory

The evolution of intellectual capacities has brought forth a continuum of consciousness levels subserved by neuronal networks of varying complexity. Brain pathologies, neurodegenerative, and mental diseases affect conscious cognition and behavior. Although impairments in consciousness are among the most devastating consequences of neurological and mental diseases, valid and reliable animal models of consciousness, that could be used for preclinical research are missing. The platform theory holds that the brain enters a conscious operation mode, whenever mental representations of stimuli, associations, concepts, memories, and experiences are effortfully maintained (in working memory) and actively manipulated. We used the platform theory as a framework and evaluation standard to categorize behavioral paradigms with respect to the level of consciousness involved in task performance. According to the platform theory, a behavioral paradigm involves conscious cognitive operations, when the problem posed is unexpected, novel or requires the maintenance and manipulation of a large amount of information to perform cognitive operations on them. Conscious cognitive operations are associated with a relocation of processing resources and the redirection of attentional focus. A consciousness behavioral test battery is proposed that is composed of tests which are assumed to require higher levels of consciousness as compared to other tasks and paradigms. The consciousness test battery for rodents includes the following tests: Working memory in the radial arm maze, episodic-like memory, prospective memory, detour test, and operant conditioning with concurrent variable-interval variable-ratio schedules. Performance in this test battery can be contrasted with the performance in paradigms and tests that require lower levels of consciousness. Additionally, a second more comprehensive behavioral test battery is proposed to control for behavioral phenotypes not related to consciousness. Our theory could serve as a guidance for the decryption of the neurobiological basis of consciousness.

Computational validity: using computation to translate behaviours across species

We propose a new conceptual framework (computational validity) for translation across species and populations based on the computational similarity between the information processing underlying parallel tasks. Translating between species depends not on the superficial similarity of the tasks presented, but rather on the computational similarity of the strategies and mechanisms that underlie those behaviours. Computational validity goes beyond construct validity by directly addressing questions of information processing. Computational validity interacts with circuit validity as computation depends on circuits, but similar computations could be accomplished by different circuits. Because different individuals may use different computations to accomplish a given task, computational validity suggests that behaviour should be understood through the subject's point of view; thus, behaviour should be characterized on an individual level rather than a task level. Tasks can constrain the computational algorithms available to a subject and the observed subtleties of that behaviour can provide information about the computations used by each individual. Computational validity has especially high relevance for the study of psychiatric disorders, given the new views of psychiatry as identifying and mediating information processing dysfunctions that may show high inter-individual variability, as well as for animal models investigating aspects of human psychiatric disorders. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Enhanced habit formation in Tourette patients explained by shortcut modulation in a hierarchical cortico-basal ganglia model

Devaluation protocols reveal that Tourette patients show an increased propensity to habitual behaviors as they continue to respond to devalued outcomes in a cognitive stimulus-response-outcome association task. We use a neuro-computational model of hierarchically organized cortico-basal ganglia-thalamo-cortical loops to shed more light on habit formation and its alteration in Tourette patients. In our model, habitual behavior emerges from cortico-thalamic shortcut connections, where enhanced habit formation can be linked to faster plasticity in the shortcut or to a stronger feedback from the shortcut to the basal ganglia. We explore two major hypotheses of Tourette pathophysiology-local striatal disinhibition and increased dopaminergic modulation of striatal medium spiny neurons-as causes for altered shortcut activation. Both model changes altered shortcut functioning and resulted in higher rates of responses towards devalued outcomes, similar to what is observed in Tourette patients. We recommend future experimental neuroscientific studies to locate shortcuts between cortico-basal ganglia-thalamo-cortical loops in the human brain and study their potential role in health and disease.

Striatal dopamine signals are region specific and temporally stable across action-sequence habit formation

Habits are automatic, inflexible behaviors that develop slowly with repeated performance. Striatal dopamine signaling instantiates this habit-formation process, presumably region specifically and via ventral-to-dorsal and medial-to-lateral signal shifts. Here, we quantify dopamine release in regions implicated in these presumed shifts (ventromedial striatum [VMS], dorsomedial striatum [DMS], and dorsolateral striatum [DLS]) in rats performing an action-sequence task and characterize habit development throughout a 10-week training. Surprisingly, all regions exhibited stable dopamine dynamics throughout habit development. VMS and DLS signals did not differ between habitual and non-habitual animals, but DMS dopamine release increased during action-sequence initiation and decreased during action-sequence completion in habitual rats, whereas non-habitual rats showed opposite effects. Consistently, optogenetic stimulation of DMS dopamine release accelerated habit formation. Thus, we demonstrate that dopamine signals do not shift regionally during habit formation and that dopamine in DMS, but not VMS or DLS, determines habit bias, attributing “habit functions” to a region previously associated exclusively with non-habitual behavior.

•

Validation of a novel test that monitors habit development individually across time

•

Dopamine release during habit development is stable across relevant striatal regions

•

Only dopamine release in dorsomedial striatum correlates with habit development

•

Optogenetic stimulation of dorsomedial striatal dopamine accelerates habit formation

When the front fails, the rear wins. Cerebellar correlates of prefrontal dysfunction in cocaine-induced memory in male rats

Reciprocal pathways connecting the cerebellum to the prefrontal cortex provide a biological and functional substrate to modulate cognitive functions. Dysfunction of both medial prefrontal cortex (mPFC) and cerebellum underlie the phenotypes of several neuropsychiatric disorders that exhibit comorbidity with substance use disorder (SUD). In people with SUD, cue-action-reward associations appears to be particularly strong and salient, acting as powerful motivational triggers for craving and relapse. Studies of cue reactivity in human with SUD have shown cerebellar activations when drug-related cues are presented. Our preclinical research showed that cocaine-induced conditioned preference increases neural activity and upregulates perineuronal nets (PNNs) around Golgi interneurons in the posterior cerebellar cortex. In the present investigation, we aimed at evaluating cerebellar signatures of conditioned preference for cocaine when drug learning is established under mPFC impairment. We used lidocaine to temporarily inactivate in male rats either the Prelimbic (PL) or the Infralimbic (IL) cortices during cocaine-induced conditioning. The inactivation of the IL, but not the PL, encouraged the acquisition of preference for cocaine-related cues, increased posterior cerebellar cortex activity, and upregulated the expression of PNNs around Golgi interneurons. Moreover, IL impairment not only increased vGluT2- and vGAT-related activity around Golgi cells but also regulated PNNs differently on subpopulations of Golgi cells, increasing the number of neurogranin+ PNN-expressing Golgi cells. Our findings suggest that IL dysfunction may facilitate the acquisition of cocaine-induced memory and cerebellar drug-related learning hallmarks. Overall, IL perturbation during cocaine-induced Pavlovian learning increased cerebellar activity and drug effects. Importantly, cerebellum involvement requires a contingent experience with the drug, and it is not the effect of a mere inactivation of IL cortex.

Dysregulated expression of the alternatively spliced variant mRNAs of the mu opioid receptor gene, OPRM1, in the medial prefrontal cortex of male human heroin abusers and heroin self-administering male rats

Heroin, a mu agonist, acts through the mu opioid receptor. The mu opioid receptor gene, OPRM1, undergoes extensive alternative splicing, creating an array of splice variants that are conserved from rodent to humans. Increasing evidence suggests that these OPRM1 splice variants are pharmacologically important in mediating various actions of mu opioids, including analgesia, tolerance, physical dependence, rewarding behavior, as well as addiction. In the present study, we examine expression of the OPRM1 splice variant mRNAs in the medial prefrontal cortex (mPFC), one of the major brain regions involved in decision-making and drug-seeking behaviors, of male human heroin abusers and male rats that developed stable heroin-seeking behavior using an intravenous heroin self-administration (SA) model. The results show similar expression profiles among multiple OPRM1 splice variants in both human control subjects and saline control rats, illustrating conservation of OPRM1 alternative splicing from rodent to humans. Moreover, the expressions of several OPRM1 splice variant mRNAs were dysregulated in the postmortem mPFCs from heroin abusers compared to the control subjects. Similar patterns were observed in the rat heroin SA model. These findings suggest potential roles of the OPRM1 splice variants in heroin addiction that could be mechanistically explored using the rat heroin SA model.

Dorsal striatum coding for the timely execution of action sequences

The automatic initiation of actions can be highly functional. But occasionally these actions cannot be withheld and are released at inappropriate times, impulsively. Striatal activity has been shown to participate in the timing of action sequence initiation and it has been linked to impulsivity. Using a self-initiated task, we trained adult male rats to withhold a rewarded action sequence until a waiting time interval has elapsed. By analyzing neuronal activity we show that the striatal response preceding the initiation of the learned sequence is strongly modulated by the time subjects wait before eliciting the sequence. Interestingly, the modulation is steeper in adolescent rats, which show a strong prevalence of impulsive responses compared to adults. We hypothesize this anticipatory striatal activity reflects the animals’ subjective reward expectation, based on the elapsed waiting time, while the steeper waiting modulation in adolescence reflects age-related differences in temporal discounting, internal urgency states, or explore–exploit balance.

Striatonigrostriatal Spirals in Addiction

A biological reward system is integral to all animal life and humans are no exception. For millennia individuals have investigated this system and its influences on human behavior. In the modern day, with the US facing an ongoing epidemic of substance use without an effective treatment, these investigations are of paramount importance. It is well known that basal ganglia contribute to rewards and are involved in learning, approach behavior, economic choices, and positive emotions. This review aims to elucidate the physiological role of striatonigrostriatal (SNS) spirals, as part of basal ganglia circuits, in this reward system and their pathophysiological role in perpetuating addiction. Additionally, the main functions of neurotransmitters such as dopamine and glutamate and their receptors in SNS circuits will be summarized. With this information, the claim that SNS spirals are crucial intermediaries in the shift from goal-directed behavior to habitual behavior will be supported, making this circuit a viable target for potential therapeutic intervention in those with substance use disorders.

Maintained goal-directed control with overtraining on ratio schedules

It is thought that goal-directed control of actions weakens or becomes masked by habits over time. We tested the opposing hypothesis that goal-directed control becomes stronger over time, and that this growth is modulated by the overall action-outcome contiguity. Despite group differences in action-outcome contiguity early in training, rats trained under random and fixed ratio schedules showed equivalent goal-directed control of lever pressing that appeared to grow over time. We confirmed that goal-directed control was maintained after extended training under another type of ratio schedule-continuous reinforcement-using specific satiety and taste aversion devaluation methods. These results add to the growing literature showing that extensive training does not reliably weaken goal-directed control and that it may strengthen it, or at least maintain it.

Infralimbic cortex functioning across motivated behaviors: Can the differences be reconciled?

The rodent infralimbic cortex (IL) is implicated in higher order executive functions such as reward seeking and flexible decision making. However, the precise nature of its role in these processes is unclear. Early evidence indicated that the IL promotes the extinction and ongoing inhibition of fear conditioning and cocaine seeking. However, evidence spanning other behavioral domains, such as natural reward seeking and habit-based learning, suggests a more nuanced understanding of IL function. As techniques have advanced and more studies have examined IL function, identifying a unifying explanation for its behavioral function has become increasingly difficult. Here, we discuss evidence of IL function across motivated behaviors, including associative learning, drug seeking, natural reward seeking, and goal-directed versus habit-based behaviors, and emphasize how context-specific encoding and heterogeneous IL neuronal populations may underlie seemingly conflicting findings in the literature. Together, the evidence suggests that a major IL function is to facilitate the encoding and updating of contingencies between cues and behaviors to guide subsequent behaviors.

Going veggie: Identifying and overcoming the social and psychological barriers to veganism

We conceptualize the journey to ethical veganism in the stages of the transtheoretical model of change, from precontemplation through contemplation, preparation, action, and maintenance. At each stage, we explore the psychological barriers to progressing towards veganism, discuss how they manifest, and explore ways to overcome them. It is hoped that this paper can be used as a guide for animal advocates to identify the stage an individual is at, and understand and overcome the social and psychological barriers they may face to progressing. We argue that, while many people are ignorant of the cruel practices entailed in animal farming, many deliberately avoid thinking about the issue, are unable to appreciate the scale of the issue, and simply tend to favour the status quo. When engaging with the issue of farm animal suffering, meat-eaters are largely driven by cognitive dissonance, which manifests as motivated reasoning aimed at protecting one's image of oneself and one's society. This is facilitated by confirmation bias and complicit media which cater to the preferred views of their meat-eating audience. Even once convinced of veganism, habit and willpower present further barriers to acting on those beliefs. This is all in the context of a speciesist and carnistic culture where meat consumption is normal, farming is noble, and vegans are 'others'. We locate and elucidate each of these biases within the stages of the transtheoretical model and discuss the implications of this model for animal advocates and for further research.

New functions of the rodent prelimbic and infralimbic cortex in instrumental behavior

The prelimbic and infralimbic cortices of the rodent medial prefrontal cortex mediate the effects of context and goals on instrumental behavior. Recent work from our laboratory has expanded this understanding. Results have shown that the prelimbic cortex is important for the modulation of instrumental behavior by the context in which the behavior is learned (but not other contexts), with context potentially being broadly defined (to include at least previous behaviors). We have also shown that the infralimbic cortex is important in the expression of extensively-trained instrumental behavior, regardless of whether that behavior is expressed as a stimulus-response habit or a goal-directed action. Some of the most recent data suggest that infralimbic cortex may control the currently active behavioral state (e.g., habit vs. action or acquisition vs. extinction) when two states have been learned. We have also begun to examine prelimbic and infralimbic cortex function as key nodes of discrete circuits and have shown that prelimbic cortex projections to an anterior region of the dorsomedial striatum are important for expression of minimally-trained instrumental behavior. Overall, the use of an associative learning perspective on instrumental learning has allowed the research to provide new perspectives on how these two "cognitive" brain regions contribute to instrumental behavior.

Mixing memory and desire: How memory reactivation supports deliberative decision-making

Memories affect nearly every aspect of our mental life. They allow us to both resolve uncertainty in the present and to construct plans for the future. Recently, renewed interest in the role memory plays in adaptive behavior has led to new theoretical advances and empirical observations. We review key findings, with particular emphasis on how the retrieval of many kinds of memories affects deliberative action selection. These results are interpreted in a sequential inference framework, in which reinstatements from memory serve as "samples" of potential action outcomes. The resulting model suggests a central role for the dynamics of memory reactivation in determining the influence of different kinds of memory in decisions. We propose that representation-specific dynamics can implement a bottom-up "product of experts" rule that integrates multiple sets of action-outcome predictions weighted based on their uncertainty. We close by reviewing related findings and identifying areas for further research. This article is categorized under: Psychology > Reasoning and Decision Making Neuroscience > Cognition Neuroscience > Computation.

Parallel and hierarchical neural mechanisms for adaptive and predictive behavioral control

Our brain can be recognized as a network of largely hierarchically organized neural circuits that operate to control specific functions, but when acting in parallel, enable the performance of complex and simultaneous behaviors. Indeed, many of our daily actions require concurrent information processing in sensorimotor, associative, and limbic circuits that are dynamically and hierarchically modulated by sensory information and previous learning. This organization of information processing in biological organisms has served as a major inspiration for artificial intelligence and has helped to create in silico systems capable of matching or even outperforming humans in several specific tasks, including visual recognition and strategy-based games. However, the development of human-like robots that are able to move as quickly as humans and respond flexibly in various situations remains a major challenge and indicates an area where further use of parallel and hierarchical architectures may hold promise. In this article we review several important neural and behavioral mechanisms organizing hierarchical and predictive processing for the acquisition and realization of flexible behavioral control. Then, inspired by the organizational features of brain circuits, we introduce a multi-timescale parallel and hierarchical learning framework for the realization of versatile and agile movement in humanoid robots.

Dorsolateral Striatal Task-initiation Bursts Represent Past Experiences More than Future Action Plans

The dorsolateral striatum (DLS) is involved in learning and executing procedural actions. Cell ensembles in the DLS, but not the dorsomedial striatum (DMS), exhibit a burst of firing at the start of a well-learned action sequence ("task-bracketing"). However, it is currently unclear what information is contained in these bursts. Some theories suggest that these bursts should represent the procedural action sequence itself (that they should be about future action chains), whereas others suggest that they should contain representations of the current state of the world, taking into account primarily past information. In addition, the DLS local field potential shows transient bursts of power in the 50 Hz range (γ50) around the time a learned action sequence is initiated. However, it is currently unknown how bursts of activity in DLS cell ensembles and bursts of γ50 power in the DLS local field potential are related to each other. We found that DLS bursts at lap initiation in rats represented recently experienced reward locations more than future procedural actions, indicating that task-initiation DLS bursts contain primarily retrospective, rather than prospective, information to guide procedural actions. Furthermore, representations of past reward locations increased during periods of increased γ50 power in the DLS. There was no evidence of task-initiation bursts, increased γ50 power, or retrospective reward location information in the neighboring dorsomedial striatum. These data support a role for the DLS in model-free theories of procedural decision-making over planned action-chain theories, suggesting that procedural actions derive from representations of the current and recent past.SIGNIFICANCE STATEMENT While it is well-established that the dorsolateral striatum (DLS) plays a critical role in procedural decision-making, open questions remain about the kinds of representations contained in DLS ensemble activity that guide procedural actions. We found that DLS, but not DMS, cell ensembles contained nonlocal representations of past reward locations that appear moments before task-initiation DLS bursts. These retrospective representations were temporally linked to a rise in γ50 power that also preceded the characteristic DLS burst at task-initiation. These results support models of procedural decision-making based on associations between available actions and the current state of the world over models based on planning over action-chains.

The Contribution of Premotor Cortico-Striatal Projections to the Execution of Serial Order Sequences

Striatal activity is necessary to initiate and execute sequences of actions. The main excitatory input to the striatum comes from the cortex. While it is hypothesized that motor and premotor cortico-striatal projections are important to guide striatal activity during the execution of sequences of actions, technical limitations have made this challenging to address. Here, we implemented a task in mice that allows for the study of different moments to execute a serial order sequence consisting of two subsequences of actions. Using this task, we performed electrophysiological recordings in the premotor (M2) and primary motor (M1) cortices, and state-dependent optogenetic inhibitions of their cortico-striatal projections. We show that while both M2 and M1 contain activity modulations related to the execution of self-paced sequences, mainly, the premotor cortico-striatal projections contribute to the proper execution/structuring of these sequences.

Blocking NK1 receptors disrupts the sequential and temporal organization of chain grooming in rats

The basal ganglia are a group of sub-cortical structures believed to play a critical role in action selection and sequencing. The striatum is the largest input structure of the basal ganglia and contains the neuropeptide substance P in abundance. Recent computational work has suggested that substance P could play a critical role in action sequence performance and acquisition, but this has not been tested experimentally before. The aim of the present study was to test how blocking substance P's main NK1-type receptors affected the sequential and temporal organization of spontaneous behavioral patterns. We did this in rats by focusing on the grooming chain, an innate and highly stereotyped ordered sequence. We performed an open field experiment in which the NK1 receptor antagonist L-733,060 was injected intraperitoneally in rats at two doses (2 and 4 mg/kg/ml), in a within-subject counterbalanced design. We used first order transition probabilities, Variable Length Markov Models, entropy metrics and T-pattern analysis to evaluate the effects of L-733,060 on sequential and temporal aspects of spontaneously ordered behavioral sequences. Our results suggest that blocking NK1 receptors made the transitions between the grooming chain elements significantly more variable, the transition structure of the grooming bouts simpler, and it increased the probability of transitioning from active to inactive states. Overall, this suggest that blocking substance P receptors led to a general break down in the fluency of spontaneous behavioral sequences, suggesting that substance P could be playing a key role in the implementation of sequential patterns.

Noninvasive Brain Stimulation Rescues Cocaine-Induced Prefrontal Hypoactivity and Restores Flexible Behavior

BACKGROUND

To obtain desirable goals, individuals must predict the outcome of specific choices, use that information to direct appropriate actions, and adjust behavior accordingly in changing environments (behavioral flexibility). Substance use disorders are marked by impairments in behavioral flexibility along with decreased prefrontal cortical function that limits the efficacy of treatment strategies. Restoring prefrontal hypoactivity, ideally in a noninvasive manner, is an intriguing target for improving flexible behavior and treatment outcomes.

METHODS

A behavioral flexibility task was used in Long-Evans male rats (n = 97) in conjunction with electrophysiology, optogenetics, and a novel rat model of transcranial alternating current stimulation (tACS) to examine the prelimbic cortex (PrL) to nucleus accumbens (NAc) core circuit in behavioral flexibility and determine whether tACS can restore cocaine-induced neural and cognitive dysfunction.

RESULTS

Optogenetic inactivation revealed that the PrL-NAc core circuit is necessary for the ability to learn strategies to flexibly shift behavior. Cocaine self-administration history caused aberrant PrL-NAc core neural encoding and deficits in flexibility. Optogenetics that selectively activated the PrL-NAc core pathway prior to learning rescued cocaine-induced cognitive flexibility deficits. Remarkably, tACS prior to learning the task reestablished adaptive signaling in the PrL-NAc circuit and restored flexible behavior in a relatively noninvasive and frequency-specific manner.

CONCLUSIONS

We establish a role of NAc core-projecting PrL neurons in behavioral flexibility and provide a novel noninvasive brain stimulation method in rats to rescue cocaine-induced frontal hypofunction and restore flexible behavior, supporting a role of tACS as a therapeutic to treat cognitive deficits in substance use disorders.

Longitudinal Assessment of Working Memory Performance in the APPswe/PSEN1dE9 Mouse Model of Alzheimer's Disease Using an Automated Figure-8-Maze

Alzheimer's disease (AD) is a progressive neurodegenerative disorder, with a long preclinical and prodromal phase. To enable the study of disease mechanisms, AD has been modeled in many transgenic animal lines and cognitive functioning has been tested using several widely used behavioral tasks. These tasks, however, are not always suited for repeated longitudinal testing and are often associated with acute stress such as animal transfer, handling, novelty, or stress related to the task itself. This makes it challenging to relate cognitive dysfunction in animal models to cognitive decline observed in AD patients. Here, we designed an automated figure-8-maze (F8M) to test mice in a delayed alternation task (DAT) in a longitudinal manner. Mice were rewarded when they entered alternate sides of the maze on subsequent trials. Automation as well as connection of the F8M set-up with a home cage reduces experimenter interference and minimizes acute stress, thus making it suitable for longitudinal testing and facilitating clinical translation. In the present study, we monitored cognitive functioning of 2-month-old APPswe/PSEN1dE9 (APP/PS1) mice over a period of 4 months. The percentage of correct responses in the DAT did not differ between wild-type and transgenic mice from 2 to 6 months of age. However, 6-month-old mice displayed an increase in the number of consecutive incorrect responses. These results demonstrate the feasibility of longitudinal testing using an automated F8M and suggest that APP/PS1 mice are not impaired at delayed spatial alternation until 6 months of age under the current experimental conditions.

Effect of Experimental Manipulation of the Orbitofrontal Cortex on Short-Term Markers of Compulsive Behavior: A Theta Burst Stimulation Study

OBJECTIVE

Compulsive behaviors are a core feature of obsessive-compulsive spectrum disorders but appear across a broad spectrum of psychological conditions. It is thought that compulsions reflect a failure to override habitual behaviors "stamped in" through repeated practice and short-term distress reduction. Animal models suggest a possible causal role of the orbitofrontal cortex (OFC) in compulsive behaviors, but human studies have largely been limited by correlational designs. The goal of this study was to establish the first experimental evidence in humans for a mechanistic model in order to inform further experimental work and the eventual development of novel mechanistic treatments involving synergistic biological-behavioral pairings.

METHODS

After a baseline assessment, 69 individuals with compulsive behavior disorders were randomly assigned, in a double-blind, between-subjects design, to receive a single session of one of two active stimulation conditions targeting the left OFC: intermittent theta burst stimulation (iTBS), expected to increase OFC activity, or continuous TBS (cTBS), expected to decrease activity (both conditions, 600 pulses at 110% of target resting motor threshold). In both conditions, brain modulation was paired with a subsequent computer task providing practice in overriding a clinically relevant habit (an overlearned shock avoidance behavior), delivered during the expected window of OFC increase or decrease. Pre- and post-TBS functional MRI assessments were conducted of target engagement and compulsive behaviors performed in response to an idiographically designed stressful laboratory probe.

RESULTS

cTBS and iTBS modulated OFC activation in the expected directions. cTBS, relative to iTBS, exhibited a beneficial impact on acute laboratory assessments of compulsive behaviors 90 minutes after TBS. These acute behavioral effects persisted 1 week after cTBS.

CONCLUSIONS

Experimental modulation of the OFC, within the behavioral context of habit override training, affected short-term markers of compulsive behavior vulnerability. The findings help delineate a causal translational model, serving as an initial precursor to mechanistic intervention development.

Addiction history moderates the effect of prefrontal 10-Hz transcranial alternating current stimulation on habitual action selection

Individuals with substance use disorders (SUDs) transition more quickly from goal-directed to habitual action-selection, but the neural mechanisms underlying this phenomenon remain unclear. Data from animal models suggest that drugs of abuse can modify the neurocircuits that regulate action-selection, enhancing circuits that drive inflexible, habit-based stimulus-response (S-R) action-selection and weakening circuits that drive flexible, goal-directed actions. Here, we tested the effect of bilateral 10-Hz transcranial alternating current stimulation (10Ηz-tACs) of the dorsolateral prefrontal cortex on action-selection in men and women with a SUD history and an age- and sex-matched control group. We tested the hypothesis that true 10Ηz-tACS versus active sham stimulation would reduce perseverative errors after changed response contingencies for well-learned S-R associations, reflecting reduced habit-based action-selection, specifically in the SUD group. We found that 10 Hz-tACS increased perseverative errors in the control group, but in the SUD group, 10 Hz-tACS effects on perseverative errors depended on substance abuse duration: a longer addiction history was associated with a greater reduction of perseverative errors. These results suggest that 10Ηz-tACs altered circuit level dynamics regulating behavioral flexibility, and provide a foundation for future studies to test stimulation site, frequency, and timing specificity. Moreover, these data suggest that chronic substance abuse is associated with altered circuit dynamics that are ameliorated by 10Ηz-tACs. Determining the generalizability of these effects and their duration merits investigation as a direction for novel therapeutic interventions. These findings are timely based on growing interest in transcranial stimulation methods for treating SUDs.NEW & NOTEWORTHY Treating the executive dysfunction associated with addiction is hampered by redundancies in pharmacological regulation of different behavioral control circuits. Thus, nonpharmacological interventions hold promise for addiction treatment. Here, we show that, among people with an addiction history, 10-Hz transcranial alternating current stimulation (10Hz-tACS) of the dorsolateral prefrontal cortex can reduce habitual actions. The fact that 10Hz-tACS can regulate behavioral flexibility suggests its possible utility in reducing harmful habitual actions.

Spatially restricted inhibition of cholinergic interneurons in the dorsolateral striatum encourages behavioral exploration

When pursuing desirable outcomes, one must make the decision between exploring possible actions to obtain those outcomes and exploiting known strategies to maximize efficiency. The dorsolateral striatum (DLS) has been extensively studied with respect to how actions can develop into habits and has also been implicated as an area involved in governing exploitative behavior. Surprisingly, prior work has shown that DLS cholinergic interneurons (ChIs) are not involved in the canonical habit formation function ascribed to the DLS but are instead modulators of behavioral flexibility after initial learning. To further probe this, we evaluated the role of DLS ChIs in behavioral exploration during a brief instrumental training experiment. Through designer receptors exclusively activated by designer drugs (DREADDs) in ChAT-Cre rats, ChIs in the DLS were inhibited during specific phases of the experiment: instrumental training, free-reward delivery, at both times, or never. Without ChI activity during instrumental training, animals biased their responding toward an "optimal" strategy while continuing to work efficiently. This effect was observed again when contingencies were removed as animals with ChIs offline during that phase, regardless of ChI inhibition previously, decreased responding more than animals with ChIs intact. These findings build upon a growing body of literature implicating ChIs in the striatum as gate-keepers of behavioral flexibility and exploration.

Place vs. Response Learning: History, Controversy, and Neurobiology

The present article provides a historical review of the place and response learning plus-maze tasks with a focus on the behavioral and neurobiological findings. The article begins by reviewing the conflict between Edward C. Tolman's cognitive view and Clark L. Hull's stimulus-response (S-R) view of learning and how the place and response learning plus-maze tasks were designed to resolve this debate. Cognitive learning theorists predicted that place learning would be acquired faster than response learning, indicating the dominance of cognitive learning, whereas S-R learning theorists predicted that response learning would be acquired faster, indicating the dominance of S-R learning. Here, the evidence is reviewed demonstrating that either place or response learning may be dominant in a given learning situation and that the relative dominance of place and response learning depends on various parametric factors (i.e., amount of training, visual aspects of the learning environment, emotional arousal, et cetera). Next, the neurobiology underlying place and response learning is reviewed, providing strong evidence for the existence of multiple memory systems in the mammalian brain. Research has indicated that place learning is principally mediated by the hippocampus, whereas response learning is mediated by the dorsolateral striatum. Other brain regions implicated in place and response learning are also discussed in this section, including the dorsomedial striatum, amygdala, and medial prefrontal cortex. An exhaustive review of the neurotransmitter systems underlying place and response learning is subsequently provided, indicating important roles for glutamate, dopamine, acetylcholine, cannabinoids, and estrogen. Closing remarks are made emphasizing the historical importance of the place and response learning tasks in resolving problems in learning theory, as well as for examining the behavioral and neurobiological mechanisms of multiple memory systems. How the place and response learning tasks may be employed in the future for examining extinction, neural circuits of memory, and human psychopathology is also briefly considered.

Vulnerability to addiction

Addiction is a chronic brain disease that has dramatic health and socioeconomic consequences worldwide. Multiple approaches have been used for decades to clarify the neurobiological basis of this disease and to identify novel potential treatments. This review summarizes the main brain networks involved in the vulnerability to addiction and specific innovative technological approaches to investigate these neural circuits. First, the evolution of the definition of addiction across the Diagnostic and Statistical Manual of Mental Disorders (DSM) is revised. We next discuss several innovative experimental techniques that, combined with behavioral approaches, have allowed recent critical advances in understanding the neural circuits involved in addiction, including DREADDs, calcium imaging, and electrophysiology. All these techniques have been used to investigate specific neural circuits involved in vulnerability to addiction and have been extremely useful to clarify the neurobiological basis of each specific component of the addictive process. These novel tools targeting specific brain regions are of great interest to further understand the different aspects of this complex disease. This article is part of the special issue on 'Vulnerabilities to Substance Abuse.'.