A dual operator view of habitual behavior reflecting cortical and striatal dynamics

Lesions of ventrolateral striatum eliminate lose-shift but not win-stay behaviour in rats

Animals tend to repeat actions that are associated with reward delivery, whereas they tend to shift responses to alternate choices following reward omission. These so-called win-stay and lose-shift responses are employed by a wide range of animals in a variety of decision-making scenarios, and depend on dissociated regions of the striatum. Specifically, lose-shift responding is impaired by extensive excitotoxic lesions of the lateral striatum. Here we used focal lesions to assess whether dorsal and ventral regions of the lateral striatum contribute differently to this effect. We found that damage to ventrolateral striatum reduced lose-shift responding without impairing win-stay, motoric, or motivational aspects of behaviour in the task, whereas lesions confined to the dorsolateral striatum significantly impaired the ability of rats to complete trials of the task. Moreover, lesions to the dorsomedial striatum had no effect on either lose-shift or win-stay responding. Together, these data suggest a novel role of the ventral portion of the lateral striatum in driving lose-shift decisions.

Beyond simple tests of value: measuring addiction as a heterogeneous disease of computation-specific valuation processes

Addiction is considered to be a neurobiological disorder of learning and memory because addiction is capable of producing lasting changes in the brain. Recovering addicts chronically struggle with making poor decisions that ultimately lead to relapse, suggesting a view of addiction also as a neurobiological disorder of decision-making information processing. How the brain makes decisions depends on how decision-making processes access information stored as memories in the brain. Advancements in circuit-dissection tools and recent theories in neuroeconomics suggest that neurally dissociable valuation processes access distinct memories differently, and thus are uniquely susceptible as the brain changes during addiction. If addiction is to be considered a neurobiological disorder of memory, and thus decision-making, the heterogeneity with which information is both stored and processed must be taken into account in addiction studies. Addiction etiology can vary widely from person to person. We propose that addiction is not a single disease, nor simply a disorder of learning and memory, but rather a collection of symptoms of heterogeneous neurobiological diseases of distinct circuit-computation-specific decision-making processes.

Reciprocal amygdala-prefrontal interactions in learning

Animals constantly evaluate their environment in order to avoid potential threats and obtain reward in the form of food, shelter and social interactions. In order to appropriately respond to sensory cues from the environment, the brain needs to form and store multiple cue-outcome associations. These can then be used to form predictions of the valence of sounds, smells and other sensory inputs arising from the surroundings. However, these associations must be subject to constant update, as the environment can rapidly change. Failing to adapt to such change can be detrimental to survival. Several systems in the mammalian brain have evolved to perform these important behavioral functions. Among these systems, the amygdala and prefrontal cortex are prominent players. Although the amygdala has been shown to form strong cue-outcome associations, the prefrontal cortex is essential for modifying these associations through extinction and reversal learning, and synaptic plasticity occurring in the strong reciprocal connections between these structures is thought to underlie both adaptive and maladaptive learning. Here we review the synaptic organization of the amygdala-prefrontal circuit, and summarize the physiological and behavioral evidence for its involvement in appetitive and aversive learning.

Deliberation and Procedural Automation on a Two-Step Task for Rats

Current theories suggest that decision-making arises from multiple, competing action-selection systems. Rodent studies dissociate deliberation and procedural behavior, and find a transition from procedural to deliberative behavior with experience. However, it remains unknown how this transition from deliberative to procedural control evolves within single trials, or within blocks of repeated choices. We adapted for rats a two-step task which has been used to dissociate model-based from model-free decisions in humans. We found that a mixture of model-based and model-free algorithms was more likely to explain rat choice strategies on the task than either model-based or model-free algorithms alone. This task contained two choices per trial, which provides a more complex and non-discrete per-trial choice structure. This task structure enabled us to evaluate how deliberative and procedural behavior evolved within-trial and within blocks of repeated choice sequences. We found that vicarious trial and error (VTE), a behavioral correlate of deliberation in rodents, was correlated between the two choice points on a given lap. We also found that behavioral stereotypy, a correlate of procedural automation, increased with the number of repeated choices. While VTE at the first choice point decreased [corrected] with the number of repeated choices, VTE at the second choice point did not, and only increased after unexpected transitions within the task. This suggests that deliberation at the beginning of trials may correspond to changes in choice patterns, while mid-trial deliberation may correspond to an interruption of a procedural process.

Inactivation of prelimbic and infralimbic cortex respectively affects minimally-trained and extensively-trained goal-directed actions

Several studies have examined a role for the prelimbic cortex (PL) and infralimbic cortex (IL) in free operant behavior. The general conclusion has been that PL controls goal-directed actions (instrumental behaviors that are sensitive to reinforcer devaluation) whereas IL controls habits (instrumental behaviors that are not sensitive to reinforcer devaluation). To further examine the involvement of these regions in the expression of instrumental behavior, we first implanted male rats with bilateral guide cannulae into their PL, then trained two responses to produce a sucrose pellet reinforcer, R1 and R2, each in a distinct context. R1 received extensive training and R2 received minimal training. Rats then received lithium chloride injections either paired or unpaired with sucrose pellets in both contexts until paired rats rejected all pellets. Following acquisition, in Experiment 1, rats received either an infusion of saline or baclofen/muscimol into the PL and were tested (in extinction) on both R1 and R2. In vehicle controls, both responses were goal-directed actions, as indicated by their sensitivity to reinforcer devaluation. PL inactivation decreased expression of the minimally-trained action without affecting expression of the extensively-trained action. Experiment 2 utilized the same experimental design but with IL inactivation at test. The extensively-trained response was again a goal-directed action. However, now expression of the extensively-trained goal-directed action was suppressed by IL inactivation. The overall pattern of results suggests that the PL is involved in expression of minimally trained goal-directed behavior while the IL is involved in expression of extensively trained goal-directed behavior. This implies that the PL does not control all types of actions and the IL can control some types of actions. These results expand upon the traditional view that the PL controls action while the IL controls habit.

Infralimbic cortex is required for learning alternatives to prelimbic promoted associations through reciprocal connectivity

Prefrontal cortical areas mediate flexible adaptive control of behavior, but the specific contributions of individual areas and the circuit mechanisms through which they interact to modulate learning have remained poorly understood. Using viral tracing and pharmacogenetic techniques, we show that prelimbic (PreL) and infralimbic cortex (IL) exhibit reciprocal PreL↔IL layer 5/6 connectivity. In set-shifting tasks and in fear/extinction learning, activity in PreL is required during new learning to apply previously learned associations, whereas activity in IL is required to learn associations alternative to previous ones. IL→PreL connectivity is specifically required during IL-dependent learning, whereas reciprocal PreL↔IL connectivity is required during a time window of 12–14 h after association learning, to set up the role of IL in subsequent learning. Our results define specific and opposing roles of PreL and IL to together flexibly support new learning, and provide circuit evidence that IL-mediated learning of alternative associations depends on direct reciprocal PreL↔IL connectivity.

Prelimbic (PL) and infralimbic (IL) cortical areas are known to have complementary roles in learning and decision making. Here the authors report reciprocal connectivity between the two areas and elucidate their functional impact on different aspects of learning.

Implicit sequence learning in young people with Tourette syndrome with and without co-occurring attention-deficit/hyperactivity disorder

Impaired habit-learning has been proposed to underlie the tic symptoms of Tourette syndrome (TS). However, accounts differ in terms of how habit-learning is altered in TS, with some authors proposing habit formation is impaired due to a deficient 'chunking' mechanism, and others proposing habit-learning is overactive and tics reflect hyperlearned behaviours. Attention-deficit/hyperactivity disorder (ADHD) frequently co-occurs with TS and is known to affect cognitive function in young people with co-occurring TS and ADHD (TS + ADHD). It is unclear, however, how co-occurring ADHD symptoms affect habit-learning in TS. In this study, we investigated whether young people with TS would show deficient or hyperactive habit-learning, and assessed the effects of co-occurring ADHD symptoms on habit-learning in TS. Participants aged 9-17 years with TS (n = 18), TS + ADHD (n = 17), ADHD (n = 13), and typical development (n = 20) completed a motor sequence learning task to assess habit-learning. We used a 2 (TS-yes, TS-no) × 2 (ADHD-yes, ADHD-no) factorial analysis to test the effects of TS, ADHD, and their interaction on accuracy and reaction time indices of sequence learning. TS was associated with intact sequence learning, but a tendency for difficulty transitioning from sequenced to non-sequenced performance was suggestive of hyper-learning. ADHD was associated with significantly poorer accuracy during acquisition of the sequence, indicative of impaired habit-learning. There were no interactions between the TS and ADHD factors, indicating young people with TS + ADHD showed both TS- and ADHD-related atypicalities in habit-learning.

Dorsolateral Striatum Engagement Interferes with Early Discrimination Learning

In current models, learning the relationship between environmental stimuli and the outcomes of actions involves both stimulus-driven and goal-directed systems, mediated in part by the DLS and DMS, respectively. However, though these models emphasize the importance of the DLS in governing actions after extensive experience has accumulated, there is growing evidence of DLS engagement from the onset of training. Here, we used in vivo photosilencing to reveal that DLS recruitment interferes with early touchscreen discrimination learning. We also show that the direct output pathway of the DLS is preferentially recruited and causally involved in early learning and find that silencing the normal contribution of the DLS produces plasticity-related alterations in a PL-DMS circuit. These data provide further evidence suggesting that the DLS is recruited in the construction of stimulus-elicited actions that ultimately automate behavior and liberate cognitive resources for other demands, but with a cost to performance at the outset of learning.

Pharmacological Blockade of Adenosine A2A but Not A1 Receptors Enhances Goal-Directed Valuation in Satiety-Based Instrumental Behavior

The balance and smooth shift between flexible, goal-directed behaviors and repetitive, habitual actions are critical to optimal performance of behavioral tasks. The striatum plays an essential role in control of goal-directed versus habitual behaviors through a rich interplay of the numerous neurotransmitters and neuromodulators to modify the input, processing and output functions of the striatum. The adenosine receptors (namely A_2AR and A₁R), with their high expression pattern in the striatum and abilities to interact and integrate dopamine, glutamate and cannabinoid signals in the striatum, may represent novel therapeutic targets for modulating instrumental behavior. In this study, we examined the effects of pharmacological blockade of the A_2ARs and A₁Rs on goal-directed versus habitual behaviors in different information processing phases of instrumental learning using a satiety-based instrumental behavior procedure. We found that A_2AR antagonist acts at the coding, consolidation and expression phases of instrumental learning to modulate animals’ sensitivity to goal-directed valuation without modifying action-outcome contingency. However, pharmacological blockade and genetic knockout of A₁Rs did not affect acquisition or sensitivity to goal-valuation of instrumental behavior. These findings provide pharmacological evidence for a potential therapeutic strategy to control abnormal instrumental behaviors associated with drug addiction and obsessive-compulsive disorder by targeting the A_2AR.

The Multifaceted Role of the Ventromedial Prefrontal Cortex in Emotion, Decision Making, Social Cognition, and Psychopathology

The ventromedial prefrontal cortex (vmPFC) has been implicated in a variety of social, cognitive, and affective functions that are commonly disrupted in mental illness. In this review, we summarize data from a diverse array of human and animal studies demonstrating that the vmPFC is a key node of cortical and subcortical networks that subserve at least three broad domains of psychological function linked to psychopathology. One track of research indicates that the vmPFC is critical for the representation of reward- and value-based decision making, through interactions with the ventral striatum and amygdala. A second track of research demonstrates that the vmPFC is critical for the generation and regulation of negative emotion, through its interactions with the amygdala, bed nucleus of the stria terminalis, periaqueductal gray, hippocampus, and dorsal anterior cingulate cortex. A third track of research shows the importance of the vmPFC in multiple aspects of social cognition, such as facial emotion recognition, theory-of-mind ability, and processing self-relevant information, through its interactions with the posterior cingulate cortex, precuneus, dorsomedial PFC, and amygdala. We then present meta-analytic data revealing distinct subregions within the vmPFC that correspond to each of these three functions, as well as the associations between these subregions and specific psychiatric disorders (depression, posttraumatic stress disorder, addiction, social anxiety disorder, bipolar disorder, schizophrenia, and attention-deficit/hyperactivity disorder). We conclude by describing several translational possibilities for clinical studies of vmPFC-based circuits, including neuropsychological assessment of transdiagnostic functions, anatomical targets for intervention, predictors of treatment response, markers of treatment efficacy, and subtyping within disorders.

The Crossed Projection to the Striatum in Two Species of Monkey and in Humans: Behavioral and Evolutionary Significance

The corpus callosum establishes the anatomical continuity between the 2 hemispheres and coordinates their activity. Using histological tracing, single axon reconstructions, and diffusion tractography, we describe a callosal projection to n caudatus and putamen in monkeys and humans. In both species, the origin of this projection is more restricted than that of the ipsilateral projection. In monkeys, it consists of thin axons (0.4-0.6 µm), appropriate for spatial and temporal dispersion of subliminal inputs. For prefrontal cortex, contralateral minus ipsilateral delays to striatum calculated from axon diameters and conduction distance are <2 ms in the monkey and, by extrapolation, <4 ms in humans. This delay corresponds to the performance in Poffenberger's paradigm, a classical attempt to estimate central conduction delays, with a neuropsychological task. In both species, callosal cortico-striatal projections originate from prefrontal, premotor, and motor areas. In humans, we discovered a new projection originating from superior parietal lobule, supramarginal, and superior temporal gyrus, regions engaged in language processing. This projection crosses in the isthmus the lesion of which was reported to dissociate syntax and prosody. The projection might originate from an overproduction of callosal projections in development, differentially pruned depending on species.

Basal ganglia mechanisms in action selection, plasticity, and dystonia

Basal ganglia circuits are organized to selected desired actions and to inhibit potentially competing unwanted actions. This is accomplished through a complex circuitry that is modified through development and learning. Mechanisms of neural plasticity underlying these modifications are increasingly understood, but new mechanisms continue to be discovered. Dystonia, a movement disorder characterized by involuntary muscle contractions that cause abnormal postures and movements. Emerging evidence points to important links between mechanisms of plasticity and the manifestations of dystonia. Investigation of these mechanisms has improved understanding of the action of currently used medication and is informing the development of new treatments.

Brain correlates of constituent structure in sign language comprehension

During sentence processing, areas of the left superior temporal sulcus, inferior frontal gyrus and left basal ganglia exhibit a systematic increase in brain activity as a function of constituent size, suggesting their involvement in the computation of syntactic and semantic structures. Here, we asked whether these areas play a universal role in language and therefore contribute to the processing of non-spoken sign language. Congenitally deaf adults who acquired French sign language as a first language and written French as a second language were scanned while watching sequences of signs in which the size of syntactic constituents was manipulated. An effect of constituent size was found in the basal ganglia, including the head of the caudate and the putamen. A smaller effect was also detected in temporal and frontal regions previously shown to be sensitive to constituent size in written language in hearing French subjects (Pallier et al., 2011). When the deaf participants read sentences versus word lists, the same network of language areas was observed. While reading and sign language processing yielded identical effects of linguistic structure in the basal ganglia, the effect of structure was stronger in all cortical language areas for written language relative to sign language. Furthermore, cortical activity was partially modulated by age of acquisition and reading proficiency. Our results stress the important role of the basal ganglia, within the language network, in the representation of the constituent structure of language, regardless of the input modality.

Inversely Active Striatal Projection Neurons and Interneurons Selectively Delimit Useful Behavioral Sequences

Understanding neural representations of behavioral routines is critical for understanding complex behavior in health and disease. We demonstrate here that accentuated activity of striatal projection neurons (SPNs) at the beginning and end of such behavioral repertoires is a supraordinate representation specifically marking previously rewarded behavioral sequences independent of the individual movements making up the behavior. We recorded spike activity in the striatum and primary motor cortex as individual rats learned specific rewarded lever-press sequences, each one unique to a given rat. Motor cortical neurons mainly responded in relation to specific movements regardless of their sequence of occurrence. By contrast, striatal SPN populations in each rat fired preferentially at the initiation and termination of its acquired sequence. Critically, the SPNs did not exhibit this bracketing signal when the same rats performed unreinforced sequences containing the same sub-movements that were present in their acquired sequence. Thus, the SPN activity was specifically related to a given repetitively reinforced movement sequence. This striatal beginning-and-end activity did not appear to be dependent on motor cortical inputs. However, strikingly, simultaneously recorded fast-spiking striatal interneurons (FSIs) showed equally selective but inverse firing patterns: they fired in between the initiation and termination of the acquired sequences. These findings suggest that the striatum contains networks of neurons representing acquired sequences of behavior at a level of abstraction higher than that of the individual movements making up the sequence. We propose that such SPN-FSI networks of the striatum could underlie the acquisition of chunked behavioral units.

Habitual Behavior Is Mediated by a Shift in Response-Outcome Encoding by Infralimbic Cortex

The ability to flexibly switch between goal-directed actions and habits is critical for adaptive behavior. The infralimbic prefrontal cortex (IfL-C) has been consistently identified as a crucial structure for the regulation of response strategies. To investigate the role of the IfL-C, the present study employed two validated reinforcement schedules that either promote habits or goal-directed actions in mice. The results reveal that information about action-outcome relationships is differentially encoded in the IfL-C during actions and habits as evidenced by encoding of behavioral outcomes during goal-directed actions that is lost during habits. Optogenetic inhibition of the IfL-C selectively at press during habitual behavior (when firing rates are reduced during unreinforced goal-directed actions) resulted in restoration of sensitivity to change of action-outcome contingency. These results reveal a novel functional mechanism by which IfL-C promotes habitual behavior, and provide insight into strategies for the treatment and prevention of pathological, inflexible behavior common in neuropsychiatric illness.

Two-photon imaging in mice shows striosomes and matrix have overlapping but differential reinforcement-related responses

Striosomes were discovered several decades ago as neurochemically identified zones in the striatum, yet technical hurdles have hampered the study of the functions of these striatal compartments. Here we used 2-photon calcium imaging in neuronal birthdate-labeled Mash1-CreER;Ai14 mice to image simultaneously the activity of striosomal and matrix neurons as mice performed an auditory conditioning task. With this method, we identified circumscribed zones of tdTomato-labeled neuropil that correspond to striosomes as verified immunohistochemically. Neurons in both striosomes and matrix responded to reward-predicting cues and were active during or after consummatory licking. However, we found quantitative differences in response strength: striosomal neurons fired more to reward-predicting cues and encoded more information about expected outcome as mice learned the task, whereas matrix neurons were more strongly modulated by recent reward history. These findings open the possibility of harnessing in vivo imaging to determine the contributions of striosomes and matrix to striatal circuit function.

Inhibiting Rho kinase promotes goal-directed decision making and blocks habitual responding for cocaine

The prelimbic prefrontal cortex is necessary for associating actions with their consequences, enabling goal-directed decision making. We find that the strength of action–outcome conditioning correlates with dendritic spine density in prelimbic cortex, suggesting that new action–outcome learning involves dendritic spine plasticity. To test this, we inhibited the cytoskeletal regulatory factor Rho kinase. We find that the inhibitor fasudil enhances action–outcome memory, resulting in goal-directed behavior in mice that would otherwise express stimulus-response habits. Fasudil transiently reduces prelimbic cortical dendritic spine densities during a period of presumed memory consolidation, but only when paired with new learning. Fasudil also blocks habitual responding for cocaine, an effect that persists over time, across multiple contexts, and depends on actin polymerization. We suggest that Rho kinase inhibition promotes goal-oriented action selection by augmenting the plasticity of prelimbic cortical dendritic spines during the formation of new action–outcome memories.

Action-outcome learning requires the prelimbic prefrontal cortex. Here the authors report that fasudil, a Rho kinase inhibitor, reduces dendritic spine densities on prelimbic neurons in an activity-dependent manner, stimulating goal-directed actions, and reducing habitual responding for cocaine.

Neural circuitry for behavioural arrest

The ability to stop ongoing movement is fundamental to animal survival. Behavioural arrest involves the hierarchical integration of information throughout the forebrain, which ultimately leads to the coordinated inhibition and activation of specific brainstem motor centres. Recent advances have shed light on multiple regions and pathways involved in this critical behavioural process. Here, we synthesize these new findings together with previous work to build a more complete understanding of the circuit mechanisms underlying suppression of ongoing action. We focus on three specific conditions leading to behavioural arrest: goal completion, fear and startle. We outline the circuitry responsible for the production of these behaviours and discuss their dysfunction in neurological disease.This article is part of the themed issue 'Movement suppression: brain mechanisms for stopping and stillness'.

Regulation of habit formation in the dorsal striatum

Habits are an essential and pervasive component of our daily lives that allow us to efficiently perform routine tasks. But their disruption contributes to the symptoms that underlie many psychiatric diseases. Emerging data are revealing the cellular and molecular mechanisms of habit formation in the dorsal striatum. New data suggest that in both the dorsolateral and dorsomedial striatum histone deacetylase (HDAC) activity acts as a critical negative regulator of the transcriptional processes underlying habit formation. In this review, we discuss this recent work and draw conclusions relevant to the treatment of diseases marked by maladaptive habits.

Ventromedial Prefrontal Cortex Damage Is Associated with Decreased Ventral Striatum Volume and Response to Reward

The ventral striatum and ventromedial prefrontal cortex (vmPFC) are two central nodes of the "reward circuit" of the brain. Human neuroimaging studies have demonstrated coincident activation and functional connectivity between these brain regions, and animal studies have demonstrated that the vmPFC modulates ventral striatum activity. However, there have been no comparable data in humans to address whether the vmPFC may be critical for the reward-related response properties of the ventral striatum. In this study, we used fMRI in five neurosurgical patients with focal vmPFC lesions to test the hypothesis that the vmPFC is necessary for enhancing ventral striatum responses to the anticipation of reward. In support of this hypothesis, we found that, compared with age- and gender-matched neurologically healthy subjects, the vmPFC-lesioned patients had reduced ventral striatal activity during the anticipation of reward. Furthermore, we observed that the vmPFC-lesioned patients had decreased volumes of the accumbens subregion of the ventral striatum. Together, these functional and structural neuroimaging data provide novel evidence for a critical role for the vmPFC in contributing to reward-related activity of the ventral striatum. These results offer new insight into the functional and structural interactions between key components of the brain circuitry underlying human affective function and decision-making.

SIGNIFICANCE STATEMENT

Maladaptive decision-making is a common problem across multiple mental health disorders. Developing new pathophysiologically based strategies for diagnosis and treatment thus requires a better understanding of the brain circuits responsible for adaptive decision-making and related psychological subprocesses (e.g., reward valuation, anticipation, and motivation). Animal studies provide evidence that these functions are mediated through direct interactions between two key nodes of a posited "reward circuit," the ventral striatum and the ventromedial prefrontal cortex (vmPFC). For the first time in humans, we demonstrate that damage to the vmPFC results in decreased ventral striatum activity during reward anticipation. These data provide unique evidence on the causal mechanisms by which the vmPFC and ventral striatum interact during the anticipation of rewards.

Circuit Mechanisms of Sensorimotor Learning

The relationship between the brain and the environment is flexible, forming the foundation for our ability to learn. Here we review the current state of our understanding of the modifications in the sensorimotor pathway related to sensorimotor learning. We divide the process into three hierarchical levels with distinct goals: (1) sensory perceptual learning, (2) sensorimotor associative learning, and (3) motor skill learning. Perceptual learning optimizes the representations of important sensory stimuli. Associative learning and the initial phase of motor skill learning are ensured by feedback-based mechanisms that permit trial-and-error learning. The later phase of motor skill learning may primarily involve feedback-independent mechanisms operating under the classic Hebbian rule. With these changes under distinct constraints and mechanisms, sensorimotor learning establishes dedicated circuitry for the reproduction of stereotyped neural activity patterns and behavior.

Neural systems mediating the inhibition of cocaine-seeking behaviors

Over the past decades, research has targeted the neurobiology regulating cocaine-seeking behaviors, largely in the hopes of identifying potential targets for the treatment of cocaine addiction. Although much of this work has focused on those systems driving cocaine seeking, recently, studies examining the inhibition of cocaine-related behaviors have made significant progress in uncovering the neural systems that attenuate cocaine seeking. Such systems include the infralimbic cortex, nucleus accumbens shell, and hypothalamus. Research in this field has focused largely on the infralimbic cortex, as activity in this region appears to attenuate cocaine seeking during reinstatement and contribute to extinction learning. However, an overarching theory of function for this region that includes its role in other types of reward seeking and learning remains to be determined. Furthermore, the precise relationship between other regions involved in attenuating cocaine-seeking behavior and the infralimbic cortex remains unclear. Recent advances in the use of viral vectors combined with optogenetics, chemogenetics, and other approaches have greatly affected our capacity to investigate those systems inhibiting behavior dependent on cocaine-associated memories. This review will present current understanding regarding the neurobiology underlying the inhibition of such behaviors, especially focusing on the extinction of such memories, and explore how viral-vector targeting of specific brain circuits has begun to alter, and will continue to enrich, our knowledge regarding this issue.

Opposing roles of primate areas 25 and 32 and their putative rodent homologs in the regulation of negative emotion

Disorders of dysregulated negative emotion such as depression and anxiety also feature increased cardiovascular mortality and decreased heart-rate variability (HRV). These disorders are correlated with dysfunction within areas 25 and 32 of the ventromedial prefrontal cortex (vmPFC), but a causal relationship between dysregulation of these areas and such symptoms has not been demonstrated. Furthermore, cross-species translation is limited by inconsistent findings between rodent fear extinction and human neuroimaging studies of negative emotion. To reconcile these literatures, we applied an investigative approach to the brain-body interactions at the core of negative emotional dysregulation. We show that, in marmoset monkeys (a nonhuman primate that has far greater vmPFC homology to humans than rodents), areas 25 and 32 have causal yet opposing roles in regulating the cardiovascular and behavioral correlates of negative emotion. In novel Pavlovian fear conditioning and extinction paradigms, pharmacological inactivation of area 25 decreased the autonomic and behavioral correlates of negative emotion expectation, whereas inactivation of area 32 increased them via generalization. Area 25 inactivation also increased resting HRV. These findings are inconsistent with current theories of rodent/primate prefrontal functional similarity, and provide insight into the role of these brain regions in affective disorders. They demonstrate that area 32 hypoactivity causes behavioral generalization relevant to anxiety, and that area 25 is a causal node governing the emotional and cardiovascular symptomatology relevant to anxiety and depression.

A neuronal activation correlate in striatum and prefrontal cortex of prolonged cocaine intake

Maladaptive changes in the involvement of striatal and frontal cortical regions in drug use are thought to underlie the progression to habitual drug use and loss of cognitive control over drug intake that occur with accumulating drug experience. The present experiments focus on changes in neuronal activity in these regions associated with short-term (10 days) and long-term (60 days) self-administration of cocaine. Quantitative in situ hybridization for the immediate early gene Mkp1 was combined with statistical parametric mapping to assess the distribution of neuronal activity. We hypothesized that neuronal activity in striatum would increase in its dorsal part and that activity in frontal cortex would decrease with prolonged cocaine self-administration experience. Expression of Mkp1 was profoundly increased after cocaine self-administration, and the magnitude of this effect was greater after short-term compared to long-term self-administration. Increased neuronal activity was seen in both dorsal and ventral sectors of the striatum after 10 days exposure to cocaine. However, enhanced activity was restricted to dorsomedial and dorsocentral striatum after 60 days cocaine self-administration. In virtually all medial prefrontal and most orbitofrontal areas, increased expression of Mkp1 was observed after 10 days of cocaine taking, whereas after 60 days, enhanced expression was restricted to caudal parts of medial prefrontal and caudomedial parts of orbitofrontal cortex. Our data reveal functional changes in cellular activity in striatum and frontal cortex with increasing cocaine self-administration experience. These changes might reflect the neural processes that underlie the descent from recreational drug taking to compulsive cocaine use.

Learning new sequential stepping patterns requires striatal plasticity during the earliest phase of acquisition

Animals including humans execute motor behavior to reach their goals. For this purpose, they must choose correct strategies according to environmental conditions and shape many parameters of their movements, including their serial order and timing. To investigate the neurobiology underlying such skills, we used a multi-sensor equipped, motor-driven running wheel with adjustable sequences of foothold pegs on which mice ran to obtain water reward. When the peg patterns changed from a familiar pattern to a new pattern, the mice had to learn and implement new locomotor strategies in order to receive reward. We found that the accuracy of stepping and the achievement of water reward improved with the new learning after changes in the peg-pattern, and c-Fos expression levels assayed after the first post-switch session were high in both dorsolateral striatum and motor cortex, relative to post-switch plateau levels. Combined in situ hybridization and immunohistochemistry of striatal sections demonstrated that both enkephalin-positive (indirect pathway) neurons and substance P-positive (direct pathway) neurons were recruited specifically after the pattern switches, as were interneurons expressing neuronal nitric oxide synthase. When we blocked N-methyl-D-aspartate (NMDA) receptors in the dorsolateral striatum by injecting the NMDA receptor antagonist, D-2-amino-5-phosphonopentanoic acid (AP5), we found delays in early post-switch improvement in performance. These findings suggest that the dorsolateral striatum is activated on detecting shifts in environment to adapt motor behavior to the new context via NMDA-dependent plasticity, and that this plasticity may underlie forming and breaking skills and habits as well as to behavioral difficulties in clinical disorders.

Habit formation

Habits, both good ones and bad ones, are pervasive in animal behavior. Important frameworks have been developed to understand habits through psychological and neurobiological studies. This work has given us a rich understanding of brain networks that promote habits, and has also helped us to understand what constitutes a habitual behavior as opposed to a behavior that is more flexible and prospective. Mounting evidence from studies using neural recording methods suggests that habit formation is not a simple process. We review this evidence and take the position that habits could be sculpted from multiple dissociable changes in neural activity. These changes occur across multiple brain regions and even within single brain regions. This strategy of classifying components of a habit based on different brain signals provides a potentially useful new way to conceive of disorders that involve overly fixed behaviors as arising from different potential dysfunctions within the brain's habit network.

Anomalous Putamen Volume in Children With Complex Motor Stereotypies

BACKGROUND

Complex motor stereotypies in children are repetitive rhythmic movements that have a predictable pattern and location, seem purposeful, but serve no obvious function, tend to be prolonged, and stop with distraction, e.g., arm or hand flapping, waving. They occur in both "primary" (otherwise typically developing) and secondary conditions. These movements are best defined as habitual behaviors and therefore pathophysiologically hypothesized to reside in premotor to posterior putamen circuits. This study sought to clarify the underlying neurobiologic abnormality in children with primary complex motor stereotypies using structural neuroimaging, emphasizing brain regions hypothesized to underlie these atypical behaviors.

METHODS

High-resolution anatomic magnetic resonance images, acquired at 3.0 T, were analyzed in children aged eight to twelve years (20 with primary complex motor stereotypies and 20 typically developing). Frontal lobe subregions and striatal structures were delineated for analysis.

RESULTS

Significant reductions (P = 0.045) in the stereotypies group were identified in total putamen volume but not in caudate, nucleus accumbens, or frontal subregions. There were no group differences in total cerebral volume.

CONCLUSIONS

Findings of a smaller putamen provide preliminary evidence suggesting the potential involvement of the habitual pathway as the underlying anatomic site in primary complex motor stereotypies.

Vicarious trial and error

When rats come to a decision point, they sometimes pause and look back and forth as if deliberating over the choice; at other times, they proceed as if they have already made their decision. In the 1930s, this pause-and-look behaviour was termed 'vicarious trial and error' (VTE), with the implication that the rat was 'thinking about the future'. The discovery in 2007 that the firing of hippocampal place cells gives rise to alternating representations of each of the potential path options in a serial manner during VTE suggested a possible neural mechanism that could underlie the representations of future outcomes. More-recent experiments examining VTE in rats suggest that there are direct parallels to human processes of deliberative decision making, working memory and mental time travel.

The mouse cortico-striatal projectome

Different cortical areas are organized into distinct intra-cortical subnetworks. How descending pathways from the entire cortex interact subcortically as a network remains unclear. Here, we report an open-access comprehensive mesoscale cortico-striatal projectome—a detailed connectivity projection map from the entire cerebral cortex to the dorsal striatum or caudoputamen (CP) in rodents. Based on these projections, we use novel computational neuroanatomical tools to identify 29 distinct functional striatal domains. Further, we characterize different cortico-striatal networks and how they reconfigure across the rostral-caudal extent of the CP. The workflow was also applied to select cortico-striatal connections in two different mouse models of disconnection syndromes to demonstrate its utility in characterizing circuitry-specific connectopathies. Together, this work provides the structural basis for studying the functional diversity of the dorsal striatum and disruptions of cortico-basal ganglia networks across a broad range of disorders.

Extended cocaine-seeking produces a shift from goal-directed to habitual responding in rats

Cocaine addiction is often characterized by a rigid pattern of behavior in which cocaine users continue seeking and taking drug despite negative consequences associated with its use. As such, full acquisition and relapse of drug-seeking behavior may be attributed to a shift away from goal-directed responding and a shift towards the maladaptive formation of rigid and habit-like responses. This rigid nature of habitual responding can be developed with extended training and is typically characterized by insensitivity to changes in outcome value. The present study determined whether cocaine (primary reinforcer) and cocaine associated cues (secondary reinforcer) could be devalued in rats with different histories of cocaine self-administration. Specifically, rats were trained on two schedules of cocaine self-administration (long-access vs. short-access). Following training the cocaine reinforcer was devalued through three separate pairings of lithium chloride with cocaine infusions. Cocaine history did not have an impact on devaluation of cocaine-associated cues. However, the reinforcing properties of cocaine were devalued only in rats on a short-access cocaine schedule but not those trained on a long-access schedule. Taken together this pattern of findings suggests that, in short access rats, devaluation is specific to the primary reinforcer and not associative stimuli such as cues. Importantly, rats that received extended training during self-administration displayed insensitivity to outcome devaluation of the primary reinforcer as well as all associative stimuli, thus displaying rigid behavioral responding similar to behavioral patterns found in addiction. Alternatively, long access cocaine exposure may have altered the devaluation threshold.

Emotional self-control and dysregulation: A dual-process analysis of pathways to externalizing/internalizing symptomatology and positive well-being in younger adolescents

OBJECTIVES

There is little knowledge about how emotional regulation contributes to vulnerability versus resilience to substance use disorder. With younger adolescents, we studied the pathways through which emotion regulation attributes are related to predisposing factors for disorder.

METHODS

A sample of 3561 adolescents (M age 12.5 years) was surveyed. Measures for emotional self-control (regulation of sadness and anger), emotional dysregulation (angerability, affective lability, and rumination about sadness or anger), and behavioral self-control (planfulness and problem solving) were obtained. A structural model was analyzed with regulation attributes related to six intermediate variables that are established risk or protective factors for adolescent substance use (e.g., academic involvement, stressful life events). Criterion variables were externalizing and internalizing symptomatology and positive well-being.

RESULTS

Indirect pathways were found from emotional regulation to symptomatology through academic competence, stressful events, and deviance-prone attitudes and cognitions. Direct effects were also found: from emotional dysregulation to externalizing and internalizing symptomatology; emotional self-control to well-being; and behavioral self-control (inverse) to externalizing symptomatology. Emotional self-control and emotional dysregulation had independent effects and different types of pathways.

CONCLUSIONS

Adolescents scoring high on emotional dysregulation are at risk for substance dependence because of more externalizing and internalizing symptomatology. Independently, youth with better behavioral and emotional self-control are at lower risk. This occurs partly through relations of regulation constructs to environmental variables that affect levels of symptomatology (e.g., stressful events, poor academic performance). Effects of emotion regulation were found at an early age, before the typical onset of substance disorder.

Endocannabinoid Modulation of Orbitostriatal Circuits Gates Habit Formation

Everyday function demands efficient and flexible decision-making that allows for habitual and goal-directed action control. An inability to shift has been implicated in disorders with impaired decision-making, including obsessive-compulsive disorder and addiction. Despite this, our understanding of the specific molecular mechanisms and circuitry involved in shifting action control remains limited. Here we identify an endogenous molecular mechanism in a specific cortical-striatal pathway that mediates the transition between goal-directed and habitual action strategies. Deletion of cannabinoid type 1 (CB1) receptors from cortical projections originating in the orbital frontal cortex (OFC) prevents mice from shifting from goal-directed to habitual instrumental lever pressing. Activity of OFC neurons projecting to dorsal striatum (OFC-DS) and, specifically, activity of OFC-DS terminals is necessary for goal-directed action control. Lastly, CB1 deletion from OFC-DS neurons prevents the shift from goal-directed to habitual action control. These data suggest that the emergence of habits depends on endocannabinoid-mediated attenuation of a competing circuit controlling goal-directed behaviors.

Nucleus accumbens functional connectivity discriminates medication-overuse headache

Medication-overuse headache (MOH) is a secondary form of headache related to the overuse of triptans, analgesics and other acute headache medications. It is believed that MOH and substance addiction share some similar pathophysiological mechanisms. In this study we examined the whole brain resting state functional connectivity of the dorsal and ventral striatum in 30 patients (15 MOH and 15 non-MOH patients) to investigate if classification algorithms can successfully discriminate between MOH and non-MOH patients on the basis of the spatial pattern of resting state functional connectivity of the dorsal and ventral striatal region of interest. Our results indicated that both nucleus accumbens and dorsal rostral putamen functional connectivity could discriminate between MOH and non-MOH patients, thereby providing possible support to two interpretations. First, that MOH patients show altered reward functionality in line with drug abusers (alterations in functional connectivity of the nucleus accumbens). Second, that MOH patients show inability to break habitual behavior (alterations in functional connectivity of the dorsal striatum). In conclusion, our data showed that MOH patients were characterized by an altered functional connectivity of motivational circuits at rest. These differences could permit the blind discrimination between the two conditions using classification algorithms. Considered overall, our findings might contribute to the development of novel diagnostic measures.

•

Nucleus accumbens functional connectivity could discriminate between MOH and non-MOH patients.

•

Dorsal rostral putamen functional connectivity could also discriminate between MOH and non-MOH patients.

•

Our data provide insights on possible pathophysiological mechanisms of medication abuse.

Dorsolateral neostriatum contribution to incentive salience: opioid or dopamine stimulation makes one reward cue more motivationally attractive than another

Pavlovian cues for rewards can become attractive incentives: approached and 'wanted' as the rewards themselves. The motivational attractiveness of a previously learned cue is not fixed, but can be dynamically amplified during re-encounter by simultaneous activation of brain limbic circuitry. Here it was reported that opioid or dopamine microinjections in the dorsolateral quadrant of the neostriatum (DLS) of rats selectively amplify attraction toward a previously learned Pavlovian cue in an individualized fashion, at the expense of a competing cue. In an autoshaping (sign-tracking vs. goal-tracking) paradigm, microinjection of the mu opioid receptor agonist (DAMGO) or dopamine indirect agonist (amphetamine) in the DLS of sign-tracker individuals selectively enhanced their sign-tracking attraction toward the reward-predictive lever cue. By contrast, DAMGO or amphetamine in the DLS of goal-trackers selectively enhanced prepotent attraction toward the reward-proximal cue of sucrose dish. Amphetamine also enhanced goal-tracking in some sign-tracker individuals (if they ever defected to the dish even once). That DLS enhancement of cue attraction was due to stronger motivation, not stronger habits, was suggested by: (i) sign-trackers flexibly followed their cue to a new location when the lever was suddenly moved after DLS DAMGO microinjection; and (ii) DAMGO in the DLS also made sign-trackers work harder on a new instrumental nose-poke response required to earn presentations of their Pavlovian lever cue (instrumental conditioned reinforcement). Altogether, the current results suggest that DLS circuitry can enhance the incentive salience of a Pavlovian reward cue, selectively making that cue a stronger motivational magnet.

Going and stopping: Dichotomies in behavioral control by the prefrontal cortex

The rodent dorsal medial prefrontal cortex (PFC), specifically the prelimbic cortex (PL), regulates the expression of conditioned fear and behaviors interpreted as reward-seeking. Meanwhile, the ventral medial PFC, namely the infralimbic cortex (IL), is essential to extinction conditioning in both appetitive and aversive domains. Here we review evidence that supports, or refutes, this "PL-go/IL-stop" dichotomy. We focus on the extinction of conditioned fear and the extinction and reinstatement of cocaine- or heroin-reinforced responding. We then synthesize evidence that the PL is essential for developing goal-directed response strategies, while the IL supports habit behavior. Finally, we propose that some functions of the orbital PFC parallel those of the medial PFC in the regulation of response selection. Integration of these discoveries may provide points of intervention for inhibiting untethered drug seeking in drug use disorders, failures in extinction in Post-traumatic Stress Disorder, or co-morbidities between the two.

Addiction History Associates with the Propensity to Form Habits

Learned habitual responses to environmental stimuli allow efficient interaction with the environment, freeing cognitive resources for more demanding tasks. However, when the outcome of such actions is no longer a desired goal, established stimulus-response (S-R) associations or habits must be overcome. Among people with substance use disorders (SUDs), difficulty in overcoming habitual responses to stimuli associated with their addiction in favor of new, goal-directed behaviors contributes to relapse. Animal models of habit learning demonstrate that chronic self-administration of drugs of abuse promotes habitual responding beyond the domain of compulsive drug seeking. However, whether a similar propensity toward domain-general habitual responding occurs in humans with SUDs has remained unclear. To address this question, we used a visuomotor S-R learning and relearning task, the Hidden Association between Images Task, which employs abstract visual stimuli and manual responses. This task allows us to measure new S-R association learning and well-learned S-R association execution and includes a response contingency change manipulation to quantify the degree to which responding is habit-based, rather than goal-directed. We find that people with SUDs learn new S-R associations as well as healthy control participants do. Moreover, people with an SUD history slightly outperform controls in S-R execution. In contrast, people with SUDs are specifically impaired in overcoming well-learned S-R associations; those with SUDs make a significantly greater proportion of perseverative errors during well-learned S-R replacement, indicating the more habitual nature of their responses. Thus, with equivalent training and practice, people with SUDs appear to show enhanced domain-general habit formation.

Enhancement of striatum-dependent memory by conditioned fear is mediated by beta-adrenergic receptors in the basolateral amygdala

Emotional arousal can have a profound impact on various learning and memory processes. For example, unconditioned emotional stimuli (e.g., predator odor or anxiogenic drugs) enhance dorsolateral striatum (DLS)-dependent habit memory. These effects critically depend on a modulatory role of the basolateral complex of the amygdala (BLA). Recent work indicates that, like unconditioned emotional stimuli, exposure to an aversive conditioned stimulus (CS) (i.e., a tone previously paired with shock) can also enhance consolidation of DLS-dependent habit memory. The present experiments examined whether noradrenergic activity, particularly within the BLA, is required for a fear CS to enhance habit memory consolidation. First, rats underwent a fear conditioning procedure in which a tone CS was paired with an aversive unconditioned stimulus. Over the course of the next five days, rats received training in a DLS-dependent water plus-maze task, in which rats were reinforced to make a consistent body-turn response to reach a hidden escape platform. Immediately after training on days 1–3, rats received post-training systemic (Experiment 1) or intra-BLA (Experiment 2) administration of the β-adrenoreceptor antagonist, propranolol. Immediately after drug administration, half of the rats were re-exposed to the tone CS in the conditioning context (without shock). Post-training CS exposure enhanced consolidation of habit memory in vehicle-treated rats, and this effect was blocked by peripheral (Experiment 1) or intra-BLA (Experiment 2) propranolol administration. The present findings reveal that noradrenergic activity within the BLA is critical for the enhancement of DLS-dependent habit memory as a result of exposure to conditioned emotional stimuli.

Translational Research on Habit and Alcohol

Habitual actions enable efficient daily living, but they can also contribute to pathological behaviors that resistant change, such as alcoholism. Habitual behaviors are learned actions that appear goal-directed but are in fact no longer under the control of the action's outcome. Instead, these actions are triggered by stimuli, which may be exogenous or interoceptive, discrete or contextual. A major hallmark characteristic of alcoholism is continued alcohol use despite serious negative consequences. In essence, although the outcome of alcohol seeking and drinking is dramatically devalued, these actions persist, often triggered by environmental cues associated with alcohol use. Thus, alcoholism meets the definition of an initially goal-directed behavior that converts to a habit-based process. Habit and alcohol have been well investigated in rodent models, with comparatively less research in non-human primates and people. This review focuses on translational research on habit and alcohol with an emphasis on cross-species methodology and neural circuitry.

The role of NPY in learning and memory

High levels of NPY expression in brain regions important for learning and memory together with its neuromodulatory and neurotrophic effects suggest a regulatory role for NPY in memory processes. Therefore it is not surprising that an increasing number of studies have provided evidence for NPY acting as a modulator of neuroplasticity, neurotransmission, and memory. Here these results are presented in relation to the types of memory affected by NPY and its receptors. NPY can exert both inhibitory and stimulatory effects on memory, depending on memory type and phase, dose applied, brain region, and NPY receptor subtypes. Thus NPY act as a resilience factor by impairing associative implicit memory after stressful and aversive events, as evident in models of fear conditioning, presumably via Y1 receptors in the amygdala and prefrontal cortex. In addition, NPY impairs acquisition but enhances consolidation and retention in models depending on spatial and discriminative types of associative explicit memory, presumably involving Y2 receptor-mediated regulations of hippocampal excitatory transmission. Moreover, spatial memory training leads to increased hippocampal NPY gene expression that together with Y1 receptor-mediated neurogenesis could constitute necessary steps in consolidation and long-term retention of spatial memory. Altogether, NPY-induced effects on learning and memory seem to be biphasic, anatomically and temporally differential, and in support of a modulatory role of NPY at keeping the system in balance. Obtaining further insight into memory-related effects of NPY could inspire the engineering of new therapeutics targeting diseases where impaired learning and memory are central elements.

Nucleus Accumbens Core and Shell Differentially Encode Reward-Associated Cues after Reinforcer Devaluation

Nucleus accumbens (NAc) neurons encode features of stimulus learning and action selection associated with rewards. The NAc is necessary for using information about expected outcome values to guide behavior after reinforcer devaluation. Evidence suggests that core and shell subregions may play dissociable roles in guiding motivated behavior. Here, we recorded neural activity in the NAc core and shell during training and performance of a reinforcer devaluation task. Long-Evans male rats were trained that presses on a lever under an illuminated cue light delivered a flavored sucrose reward. On subsequent test days, each rat was given free access to one of two distinctly flavored foods to consume to satiation and were then immediately tested on the lever pressing task under extinction conditions. Rats decreased pressing on the test day when the reinforcer earned during training was the sated flavor (devalued) compared with the test day when the reinforcer was not the sated flavor (nondevalued), demonstrating evidence of outcome-selective devaluation. Cue-selective encoding during training by NAc core (but not shell) neurons reliably predicted subsequent behavioral performance; that is, the greater the percentage of neurons that responded to the cue, the better the rats suppressed responding after devaluation. In contrast, NAc shell (but not core) neurons significantly decreased cue-selective encoding in the devalued condition compared with the nondevalued condition. These data reveal that NAc core and shell neurons encode information differentially about outcome-specific cues after reinforcer devaluation that are related to behavioral performance and outcome value, respectively.

SIGNIFICANCE STATEMENT

Many neuropsychiatric disorders are marked by impairments in behavioral flexibility. Although the nucleus accumbens (NAc) is required for behavioral flexibility, it is not known how NAc neurons encode this information. Here, we recorded NAc neurons during a training session in which rats learned that a cue predicted a specific reward and during a test session when that reward value was changed. Although encoding in the core during training predicted the ability of rats to change behavior after the reward value was altered, the NAc shell encoded information about the change in reward value during the test session. These findings suggest differential roles of the core and shell in behavioral flexibility.