Endocannabinoid Modulation of Orbitostriatal Circuits Gates Habit Formation

Alcohol consumption does not impact delta and kappa opioid receptor-mediated synaptic depression in dorsolateral striatum of adult male mice

Many drugs of abuse, including alcohol, disrupt long-term synaptic depression (LTD) at dorsal striatal glutamate synapses. This disruption is common to many forms of LTD that are mediated by G protein coupled receptors (GPCRs) that signal through the inhibitory Gi/o class of G proteins. A loss of LTD is thought to mediate behavioral changes associated with the development of substance use disorders. We have previously shown in multiple studies that LTD mediated by the Gi/o-coupled mu opioid receptor is disrupted by in vivo opioid and alcohol exposure in adolescent and adult mice. One of our previous studies suggested that LTD mediated by delta and kappa opioid receptors was resistant to the LTD-disrupting properties of in vivo opioid exposure. We hypothesized that delta and kappa opioid receptor-mediated LTD would be exceptions to the generalizable observation that forms of dorsal striatal Gi/o-coupled receptor LTD are disrupted by drugs of abuse. Specifically, we predicted that these forms of LTD would be resistant to the deleterious effects of alcohol consumption, just as they were resistant to opioid exposure. Indeed, in adult male mice that drank alcohol for 3 weeks, delta and kappa opioid receptor-mediated LTD at glutamatergic inputs to direct pathway and indirect pathway medium spiny neurons in the dorsolateral striatum was unaffected by alcohol. These data demonstrate that alcohol effects on GPCR-mediated LTD are not generalizable across all types of Gi/o-coupled GPCRs.

Whole-brain Mapping of Inputs and Outputs of Specific Orbitofrontal Cortical Neurons in Mice

The orbitofrontal cortex (ORB), a region crucial for stimulus-reward association, decision-making, and flexible behaviors, extensively connects with other brain areas. However, brain-wide inputs to projection-defined ORB neurons and the distribution of inhibitory neurons postsynaptic to neurons in specific ORB subregions remain poorly characterized. Here we mapped the inputs of five types of projection-specific ORB neurons and ORB outputs to two types of inhibitory neurons. We found that different projection-defined ORB neurons received inputs from similar cortical and thalamic regions, albeit with quantitative variations, particularly in somatomotor areas and medial groups of the dorsal thalamus. By counting parvalbumin (PV) or somatostatin (SST) interneurons innervated by neurons in specific ORB subregions, we found a higher fraction of PV neurons in sensory cortices and a higher fraction of SST neurons in subcortical regions targeted by medial ORB neurons. These results provide insights into understanding and investigating the function of specific ORB neurons.

Conditional deletion of neurexin-2 impaired behavioral flexibility to alterations in action-outcome contingency

Neurexins (Nrxns) are critical for synapse organization and their mutations have been documented in autism spectrum disorder, schizophrenia, and epilepsy. We recently reported that conditional deletion of Nrxn2, under the control of Emx1Cre promoter, predominately expressed in the neocortex and hippocampus (Emx1-Nrxn2 cKO mice) induced stereotyped patterns of behavior in mice, suggesting behavioral inflexibility. In this study, we investigated the effects of Nrxn2 deletion through two different conditional approaches targeting presynaptic cortical neurons projecting to dorsomedial striatum on the flexibility between goal-directed and habitual actions in response to devaluation of action-outcome (A-O) contingencies in an instrumental learning paradigm or upon reversal of A-O contingencies in a water T-maze paradigm. Nrxn2 deletion through both the conditional approaches induced an inability of mice to discriminate between goal-directed and habitual action strategies in their response to devaluation of A-O contingency. Emx1-Nrxn2 cKO mice exhibited reversal learning deficits, indicating their inability to adopt new action strategies. Overall, our studies showed that Nrxn2 deletion through two distinct conditional deletion approaches impaired flexibility in response to alterations in A-O contingencies. These investigations can lay the foundation for identification of novel genetic factors underlying behavioral inflexibility.

Sex specific effects of dorsomedial striatal cannabinoid receptor-1 signaling on Pavlovian outcome devaluation

Cannabinoid-1 receptor (CB1R) signaling in the dorsal striatum regulates the shift from flexible to habitual behavior in instrumental outcome devaluation. Based on prior work establishing individual, sex, and experience-dependent differences in Pavlovian behaviors, we predicted a role for dorsomedial striatum CB1R signaling in driving sign-tracking and rigid responding in Pavlovian outcome devaluation. We trained male and female rats in Pavlovian Lever Autoshaping to determine sign-, or goal- or intermediate tracking groups. After extended Pavlovian training, we gave intra-DMS infusions of the CB1R inverse agonist, rimonabant, before satiety-induced outcome devaluation test sessions, in which we sated rats on training pellets (devalued) or home cage chow (valued) and examined responding to cues in brief nonreinforced Pavlovian Lever Autoshaping sessions. Overall, DMS CB1R signaling inhibition blocked Pavlovian outcome devaluation. After extended training, male rats were sensitive to devaluation while female rats were not. Inhibition of DMS CB1R signaling impaired Pavlovian outcome devaluation in male sign-tracking rats making their behavior more rigid but had no effects in female sign-tracking rats. Intra-DMS rimonabant infusions before reinforced sessions had no effects on Pavlovian sign- or goal-tracking in either sex. The sex-specific and CB1R-dependent effects were specific to outcome devaluation and were consistent between sign- and goal-tracking groups. Our results demonstrate that DMS endocannabinoid receptor signaling regulates behavioral flexibility in a sex-specific manner, suggesting differences in CB1R signaling in DMS between male and female rats.

Characterizing the dynamic learning process: Implications of a quantitative analysis

Understanding the neural mechanisms involved in learning processes is crucial for unraveling the complexities of behavior and cognition. Sudden change from the untrained level to the fully-learned level is a pivotal feature of instrumental learning. However, the concept of change point and suitable methods to conveniently analyze the characteristics of sudden change in groups remain elusive, which might hinder a fuller understanding of the neural mechanism underlying dynamic leaning process. In the current study, we investigated the learning processes of mice that were trained in an aversive instrumental learning task, and introduced a novel strategy to analyze behavioral variations in instrumental learning, leading to improved clarity on the concept of sudden change and enabling comprehensive group analysis. By applying this novel strategy, we examined the effects of cocaine and a cannabinoid receptor agonist on instrumental learning. Intriguingly, our analysis revealed significant differences in timing and occurrence of sudden changes that were previously overlooked using traditional analysis. Overall, our research advances understanding of behavioral variation during instrumental learning and the interplay between learning behaviors and neurotransmitter systems, contributing to a deeper comprehension of learning processes and informing future investigations and therapeutic interventions.

Circadian Influence on Acute Stress-induced Changes in Cortico-limbic Endocannabinoid Levels in Adult Male Rats

The endocannabinoid (eCB) system plays an important role in regulating the stress response, including glucocorticoid release and the generation of avoidance behaviour. Its two major ligands, 2-arachidonoylglycerol (2-AG) and N-arachidonoylethanolamine (anandamide; AEA), are dynamically influenced by psychological stress to gate the generation of the stress response and facilitate recovery upon stress termination. Many biological systems exhibit circadian "daily" rhythms, including glucocorticoids and endocannabinoids, and the behavioural and endocrine impact of stress is modulated by the time of day. Nonetheless, most preclinical experiments investigating the interaction between stress and endocannabinoids occur in the light, "inactive" phase. We therefore tested if circadian phase influences stress-induced changes in eCB levels in the hippocampus (HIP), prefrontal cortex (PFC), and amygdala (AMY). Adult male rats were exposed to 15 min swim stress or immediately euthanized, and brains were collected. Testing occurred either early in the light or early in the dark phase of their cycle to compare circadian effects. We found that overall, stress decreased AEA in the AMY and HIP, with an effect in the PFC dependent on the time of day. Conversely, stress increased 2-AG in the AMY, with an effect in the PFC and HIP dependent on the time of day. This suggests that stress has a similar overall impact on eCB levels regardless of circadian phase, but that subtle differences may occur depending on the brain region, especially the PFC.

Astrocytic Regulation of Endocannabinoid-Dependent Synaptic Plasticity in the Dorsolateral Striatum

Astrocytes are pivotal for synaptic transmission and may also play a role in the induction and expression of synaptic plasticity, including endocannabinoid-mediated long-term depression (eCB-LTD). In the dorsolateral striatum (DLS), eCB signaling plays a major role in balancing excitation and inhibition and promoting habitual learning. The aim of this study was to outline the role of astrocytes in regulating eCB signaling in the DLS. To this end, we employed electrophysiological slice recordings combined with metabolic, chemogenetic and pharmacological approaches in an attempt to selectively suppress astrocyte function. High-frequency stimulation induced eCB-mediated LTD (HFS-LTD) in brain slices from both male and female rats. The metabolic uncoupler fluorocitrate (FC) reduced the probability of transmitter release and depressed synaptic output in a manner that was independent on cannabinoid 1 receptor (CB1R) activation. Fluorocitrate did not affect the LTD induced by the CB1R agonist WIN55,212-2, but enhanced CB1R-dependent HFS-LTD. Reduced neurotransmission and facilitated HFS-LTD were also observed during chemogenetic manipulation using Gi-coupled DREADDs targeting glial fibrillary acidic protein (GFAP)-expressing cells, during the pharmacological inhibition of connexins using carbenoxolone disodium, or during astrocytic glutamate uptake using TFB-TBOA. While pretreatment with the N-methyl-D-aspartate (NMDA) receptor antagonist 2-amino-5-phosphonopentanoic acid (APV) failed to prevent synaptic depression induced by FC, it blocked the facilitation of HFS-LTD. While the lack of tools to disentangle astrocytes from neurons is a major limitation of this study, our data collectively support a role for astrocytes in modulating basal neurotransmission and eCB-mediated synaptic plasticity.

Multiple Memory Subsystems: Reconsidering Memory in the Mind and Brain

The multiple-memory-systems framework-that distinct types of memory are supported by distinct brain systems-has guided learning and memory research for decades. However, recent work challenges the one-to-one mapping between brain structures and memory types central to this taxonomy, with key memory-related structures supporting multiple functions across substructures. Here we integrate cross-species findings in the hippocampus, striatum, and amygdala to propose an updated framework of multiple memory subsystems (MMSS). We provide evidence for two organizational principles of the MMSS theory: First, opposing memory representations are colocated in the same brain structures; second, parallel memory representations are supported by distinct structures. We discuss why this burgeoning framework has the potential to provide a useful revision of classic theories of long-term memory, what evidence is needed to further validate the framework, and how this novel perspective on memory organization may guide future research.

Characterization of striatal dopamine projections across striatal subregions in behavioral flexibility

Behavioural flexibility is key to survival in a dynamic environmentWhile flexible, goal-directed behaviours are initially dependent on dorsomedial striatum, they become dependent on lateral striatum as behaviours become inflexible. Similarly, lesions of dopamine terminals in lateral striatum disrupt the development of inflexible habits. This work suggests that dopamine release in lateral striatum may drive inflexible behaviours, though few studies have investigated a causative role of subpopulations of striatal dopamine terminals in reversal learning, a measure of flexibility. Here, we performed two optogenetic experiments to activate dopamine terminals in dorsomedial (DMS), dorsolateral (DLS) or ventral (nucleus accumbens [NAc]) striatum in DAT-Cre mice that expressed channelrhodopsin-2 via viral injection (Experiment I) or through transgenic breeding with an Ai32 reporter line (Experiment II) to determine how specific dopamine subpopulations impact reversal learning. Mice performed a reversal task in which they self-stimulated DMS, DLS, or NAc dopamine terminals by pressing one of two levers before action-outcome lever contingencies were reversed. Largely consistent with presumed ventromedial/lateral striatal function, we found that mice self-stimulating medial dopamine terminals reversed lever preference following contingency reversal, while mice self-stimulating NAc showed parial flexibility, and DLS self-stimulation resulted in impaired reversal. Impairments in DLS mice were characterized by more regressive errors and reliance on lose-stay strategies following reversal, as well as reduced within-session learning, suggesting reward insensitivity and overreliance on previously learned actions. This study supports a model of striatal function in which DMS and ventral dopamine facilitate goal-directed responding, and DLS dopamine supports more inflexible responding.

Striatonigral direct pathway 2-arachidonoylglycerol contributes to ethanol effects on synaptic transmission and behavior

Endocannabinoids (eCB) and cannabinoid receptor 1 (CB1) play important roles in mediating short- and long-term synaptic plasticity in many brain regions involved in learning and memory, as well as the reinforcing effects of misused substances. Ethanol-induced plasticity and neuroadaptations predominantly occur in striatal direct pathway projecting medium spiny neurons (dMSNs). It is hypothesized that alterations in eCB neuromodulation may be involved. Recent work has implicated a role of eCB 2-arachidonoylglycerol (2-AG) in the rewarding effects of ethanol. However, there is insufficient research to answer which cellular subtype is responsible for mediating the 2-AG eCB signal that might be involved in the rewarding properties of ethanol and the mechanisms by which that occurs. To examine the role of dMSN mediated 2-AG signaling in ethanol related synaptic transmission and behaviors, we used conditional knockout mice in which the 2-AG-synthesizing enzyme diacylglycerol lipase α (DGLα) was deleted in dMSNs, DGLαD1-Cre+. Using acute brain slice photometry and a genetically encoded fluorescent eCB sensor, GRABeCB2.0, to assess real-time eCB mediated activity of sensorimotor inputs from primary motor cortices (M1/M2) to the dorsolateral striatum, we showed that DGLαD1-Cre+ mice had blunted evoked eCB-mediated presynaptic eCB signaling compared to littermate controls. Furthermore, ethanol induced eCB inhibition was significantly reduced in DGLαD1-Cre+ deficient mice. Additionally, there was a reduction in the duration of loss of righting reflex (LORR) to a high dose of ethanol in the DGLαD1-Cre+ mice compared to controls. These mice also showed a male-specific decrease in ethanol preference accompanied by an increase in ethanol-induced water consumption in a voluntary drinking paradigm. There were no significant differences observed in sucrose and quinine consumption between the genotypes. These findings reveal a novel role for dMSN mediated 2-AG signaling in modulating ethanol effects on presynaptic function and behavior.

A novel inhibitory corticostriatal circuit that expresses mu opioid receptor-mediated synaptic plasticity

Corticostriatal circuits are generally characterized by the release of glutamate neurotransmitter from cortical terminals within the striatum. It is well known that cortical excitatory input to the dorsal striatum regulates addictive drug-related behaviors. We previously reported that anterior insular cortex (AIC) synaptic inputs to the dorsolateral striatum (DLS) control binge alcohol drinking in mice. These AIC-DLS glutamate synapses are also the sole sites of corticostriatal mu opioid receptor-mediated excitatory long-term depression (MOR-LTD) in the DLS. Recent work demonstrates that some regions of cortex send long-range, direct inhibitory inputs into the dorsal striatum. Nothing is known about the existence and regulation of AIC-DLS inhibitory synaptic transmission. Here, using a combination of patch clamp electrophysiology and optogenetics, we characterized a novel AIC-DLS corticostriatal inhibitory circuit and its regulation by MOR-mediated inhibitory LTD (MOR-iLTD). First, we found that the activation of presynaptic MORs produces MOR-iLTD in the DLS and dorsomedial striatum. Then, we showed that medium spiny neurons within the DLS receive direct inhibitory synaptic input from the cortex, specifically from the motor cortex and AIC. Using transgenic mice that express cre-recombinase within parvalbumin-expressing inhibitory neurons, we determined that this specific cortical neuron subtype sends direct GABAergic projections to the DLS. Moreover, these AIC-DLS inhibitory synaptic input subtypes express MOR-iLTD. These data suggest a novel GABAergic corticostriatal circuit that could be involved in the regulation of drug and alcohol consumption-related behaviors.

A cortico-amygdala neural substrate for endocannabinoid modulation of fear extinction

Preclinical and clinical studies implicate endocannabinoids (eCBs) in fear extinction, but the underlying neural circuit basis of these actions is unclear. Here, we employed in vivo optogenetics, eCB biosensor imaging, ex vivo electrophysiology, and CRISPR-Cas9 gene editing in mice to examine whether basolateral amygdala (BLA)-projecting medial prefrontal cortex (mPFC) neurons represent a neural substrate for the effects of eCBs on extinction. We found that photoexcitation of mPFC axons in BLA during extinction mobilizes BLA eCBs. eCB biosensor imaging showed that eCBs exhibit a dynamic stimulus-specific pattern of activity at mPFC→BLA neurons that tracks extinction learning. Furthermore, using CRISPR-Cas9-mediated gene editing, we demonstrated that extinction memory formation involves eCB activity at cannabinoid CB1 receptors expressed at vmPFC→BLA synapses. Our findings reveal the temporal characteristics and a neural circuit basis of eCBs' effects on fear extinction and inform efforts to target the eCB system as a therapeutic approach in extinction-deficient neuropsychiatric disorders.

A dual-pathway architecture enables chronic stress to promote habit formation

Chronic stress can change how we learn and, thus, how we make decisions by promoting the formation of inflexible, potentially maladaptive, habits. Here we investigated the neuronal circuit mechanisms that enable this. Using a multifaceted approach in male and female mice, we reveal a dual pathway, amygdala-striatal, neuronal circuit architecture by which a recent history of chronic stress shapes learning to disrupt flexible goal-directed behavior in favor of inflexible habits. Chronic stress inhibits activity of basolateral amygdala projections to the dorsomedial striatum to impede the action-outcome learning that supports flexible, goal-directed decisions. Stress also increases activity in direct central amygdala projections to the dorsomedial striatum to promote the formation of rigid, inflexible habits. Thus, stress exerts opposing effects on two amygdala-striatal pathways to promote premature habit formation. These data provide neuronal circuit insights into how chronic stress shapes learning and decision making, and help understand how stress can lead to the disrupted decision making and pathological habits that characterize substance use disorders and other psychiatric conditions.

Neuronal and astrocytic CB1R signaling differentially modulates goal-directed behavior and working memory by distinct temporal mechanisms

Several cognitive processes, including instrumental behavior and working memory, are controlled by endocannabinoids acting on cannabinoid receptor 1 (CB1R) in the brain through retrograde and presynaptic inhibition of GABA or glutamate release. However, the temporal mechanisms underlying the control of these cognitive processes by CB1Rs remain largely unknown. Here, we have developed a light-sensitive CB1R chimera (optoCB1R) by replacing the intracellular domains of bovine rhodopsin with those of human CB1R. We demonstrated that light stimulation of optoCB1R triggered canonical CB1R signaling by inhibiting cAMP (but not cGMP or IP1) signaling and activating the MAPK pathway in vitro or in vivo. Moreover, light stimulation of optoCB1R in corticostriatal glutamatergic neurons could temporally inhibit excitatory postsynaptic currents (EPSCs) at the level of seconds. Importantly, transient (3 s) and "time-locked", but not random, activation of optoCB1R signaling in corticostriatal neurons at the time of reward affected animal sensitivity to outcome devaluation and inhibited goal-directed behavior. However, prolonged (~30 min) but not transient (10 or 30 s) activation of astrocytic CB1R signaling in the hippocampus impaired working memory. Consequently, neuronal and astrocytic CB1R signaling differentially regulate working memory and goal-directed behavior through distinct temporal and cellular mechanisms. Ultimately, the pharmacological blockade of adenosine A2AR improved the neuronal and astrocytic CB1R-induced impairments in goal-directed behavior and working memory, possibly through modulation of EPSCs and c-Fos, respectively. Therefore, A2AR may represent a promising target for managing cognitive dysfunction resulting from the use of CB1R drugs.

Orbitofrontal cortex control of striatum leads economic decision-making

Animals must continually evaluate stimuli in their environment to decide which opportunities to pursue, and in many cases these decisions can be understood in fundamentally economic terms. Although several brain regions have been individually implicated in these processes, the brain-wide mechanisms relating these regions in decision-making are unclear. Using an economic decision-making task adapted for rats, we find that neural activity in both of two connected brain regions, the ventrolateral orbitofrontal cortex (OFC) and the dorsomedial striatum (DMS), was required for economic decision-making. Relevant neural activity in both brain regions was strikingly similar, dominated by the spatial features of the decision-making process. However, the neural encoding of choice direction in OFC preceded that of DMS, and this temporal relationship was strongly correlated with choice accuracy. Furthermore, activity specifically in the OFC projection to the DMS was required for appropriate economic decision-making. These results demonstrate that choice information in the OFC is relayed to the DMS to lead accurate economic decision-making.

Intersectin deficiency impairs cortico-striatal neurotransmission and causes obsessive-compulsive behaviors in mice

The generation of appropriate behavioral responses involves dedicated neuronal circuits. The cortico-striatal-thalamo-cortical loop is especially important for the expression of motor routines and habits. Defects in this circuitry are closely linked to obsessive stereotypic behaviors, hallmarks of neuropsychiatric diseases including autism spectrum disorders (ASDs) and obsessive-compulsive disorders (OCDs). However, our knowledge of the essential synaptic machinery required to maintain balanced neurotransmission and plasticity within the cortico-striatal circuitry remains fragmentary. Mutations in the large synaptic scaffold protein intersectin1 (ITSN1) have been identified in patients presenting with ASD symptoms including stereotypic behaviors, although a causal relationship between stereotypic behavior and intersectin function has not been established. We report here that deletion of the two closely related proteins ITSN1 and ITSN2 leads to severe ASD/OCD-like behavioral alterations and defective cortico-striatal neurotransmission in knockout (KO) mice. Cortico-striatal function was compromised at multiple levels in ITSN1/2-depleted animals. Morphological analyses showed that the striatum of intersectin KO mice is decreased in size. Striatal neurons exhibit reduced complexity and an underdeveloped dendritic spine architecture. These morphological abnormalities correlate with defects in cortico-striatal neurotransmission and plasticity as well as reduced N-methyl-D-aspartate (NMDA) receptor currents as a consequence of postsynaptic NMDA receptor depletion. Our findings unravel a physiological role of intersectin in cortico-striatal neurotransmission to counteract ASD/OCD. Moreover, we delineate a molecular pathomechanism for the neuropsychiatric symptoms of patients carrying intersectin mutations that correlates with the observation that NMDA receptor dysfunction is a recurrent feature in the development of ASD/OCD-like symptoms.

Dopaminergic D2 receptor modulation of striatal cholinergic interneurons contributes to sequence learning

Learning action sequences is necessary for normal daily activities. Medium spiny neurons (MSNs) in the dorsal striatum (dStr) encode action sequences through changes in firing at the start and/or stop of action sequences or sustained changes in firing throughout the sequence. Acetylcholine (ACh), released from cholinergic interneurons (ChIs), regulates striatal function by modulating MSN and interneuron excitability, dopamine and glutamate release, and synaptic plasticity. Cholinergic neurons in dStr pause their tonic firing during the performance of learned action sequences. Activation of dopamine type-2 receptors (D2Rs) on ChIs is one mechanism of ChI pausing. In this study we show that deleting D2Rs from ChIs by crossing D2-floxed with ChAT-Cre mice (D2Flox-ChATCre), which inhibits dopamine-mediated ChI pausing and leads to deficits in an operant action sequence task and lower breakpoints in a progressive ratio task. These data suggest that D2Flox-ChATCre mice have reduced motivation to work for sucrose reward, but show no generalized motor skill deficits. D2Flox-ChATCre mice perform similarly to controls in a simple reversal learning task, indicating normal behavioral flexibility, a cognitive function associated with ChIs. In vivo electrophysiological recordings show that D2Flox-ChatCre mice have deficits in sequence encoding, with fewer dStr MSNs encoding entire action sequences compared to controls. Thus, ChI D2R deletion appears to impair a neural substrate of action chunking. Virally replacing D2Rs in dStr ChIs in adult mice improves action sequence learning, but not the lower breakpoints, further suggesting that D2Rs on ChIs in the dStr are critical for sequence learning, but not for driving the motivational aspects of the task.

The Dysfunctional Mechanisms Throwing Tics: Structural and Functional Changes in Tourette Syndrome

Tourette Syndrome (TS) is a high-incidence multifactorial neuropsychiatric disorder characterized by motor and vocal tics co-occurring with several diverse comorbidities, including obsessive-compulsive disorder and attention-deficit hyperactivity disorder. The origin of TS is multifactorial, with strong genetic, perinatal, and immunological influences. Although almost all neurotransmettitorial systems have been implicated in TS pathophysiology, a comprehensive neurophysiological model explaining the dynamics of expression and inhibition of tics is still lacking. The genesis and maintenance of motor and non-motor aspects of TS are thought to arise from functional and/or structural modifications of the basal ganglia and related circuitry. This complex wiring involves several cortical and subcortical structures whose concerted activity controls the selection of the most appropriate reflexive and habitual motor, cognitive and emotional actions. Importantly, striatal circuits exhibit bidirectional forms of synaptic plasticity that differ in many respects from hippocampal and neocortical plasticity, including sensitivity to metaplastic molecules such as dopamine. Here, we review the available evidence about structural and functional anomalies in neural circuits which have been found in TS patients. Finally, considering what is known in the field of striatal plasticity, we discuss the role of exuberant plasticity in TS, including the prospect of future pharmacological and neuromodulation avenues.

Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks

A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.

Early-life seizures alter habit behavior formation and fronto-striatal circuit dynamics

Obsessive compulsive disorder (OCD) can occur comorbidly with epilepsy; both are complex, disruptive disorders that lower quality of life. Both OCD and epilepsy are disorders of hyperexcitable circuits, but it is unclear whether common circuit pathology may underlie the co-occurrence of these two neuropsychiatric disorders. Here, we induced early-life seizures (ELS) in rats to examine habit formation as a model for compulsive behaviors. Compulsive, repetitive behaviors in OCD utilize the same circuitry as habit formation. We hypothesized that rats with ELS could be more susceptible to habit formation than littermate controls, and that altered behavior would correspond to altered signaling in fronto-striatal circuits that underlie decision-making and action initiation. Here, we show instead that rats with ELS were significantly less likely to form habit behaviors compared with control rats. This behavioral difference corresponded with significant alterations to temporal coordination within and between brain regions that underpin the action to habit transition: 1) phase coherence between the lateral orbitofrontal cortex and dorsomedial striatum (DMS) and 2) theta-gamma coupling within DMS. Finally, we used cortical electrical stimulation as a model of transcranial magnetic stimulation (TMS) to show that temporal coordination of fronto-striatal circuits in control and ELS rats are differentially susceptible to potentiating and suppressive stimulation, suggesting that altered underlying circuit physiology may lead to altered response to therapeutic interventions such as TMS.

Pathways to the persistence of drug use despite its adverse consequences

The persistence of drug taking despite its adverse consequences plays a central role in the presentation, diagnosis, and impacts of addiction. Eventual recognition and appraisal of these adverse consequences is central to decisions to reduce or cease use. However, the most appropriate ways of conceptualizing persistence in the face of adverse consequences remain unclear. Here we review evidence that there are at least three pathways to persistent use despite the negative consequences of that use. A cognitive pathway for recognition of adverse consequences, a motivational pathway for valuation of these consequences, and a behavioral pathway for responding to these adverse consequences. These pathways are dynamic, not linear, with multiple possible trajectories between them, and each is sufficient to produce persistence. We describe these pathways, their characteristics, brain cellular and circuit substrates, and we highlight their relevance to different pathways to self- and treatment-guided behavior change.

Postsynaptic synucleins mediate endocannabinoid signaling

Endocannabinoids are among the most powerful modulators of synaptic transmission throughout the nervous system, and yet little is understood about the release of endocannabinoids from postsynaptic compartments. Here we report an unexpected finding that endocannabinoid release requires synucleins, key contributors to Parkinson's disease. We show that endocannabinoids are released postsynaptically by a synuclein-dependent and SNARE-dependent mechanism. Specifically, we found that synuclein deletion blocks endocannabinoid-dependent synaptic plasticity; this block is reversed by postsynaptic expression of wild-type but not of mutant α-synuclein. Whole-cell recordings and direct optical monitoring of endocannabinoid signaling suggest that the synuclein deletion specifically blocks endocannabinoid release. Given the presynaptic role of synucleins in regulating vesicle lifecycle, we hypothesize that endocannabinoids are released via a membrane interaction mechanism. Consistent with this hypothesis, postsynaptic expression of tetanus toxin light chain, which cleaves synaptobrevin SNAREs, also blocks endocannabinoid-dependent signaling. The unexpected finding that endocannabinoids are released via a synuclein-dependent mechanism is consistent with a general function of synucleins in membrane trafficking and adds a piece to the longstanding puzzle of how neurons release endocannabinoids to induce synaptic plasticity.

Identification of a GABAergic neural circuit governing leptin signaling deficiency-induced obesity

The hormone leptin is known to robustly suppress food intake by acting upon the leptin receptor (LepR) signaling system residing within the agouti-related protein (AgRP) neurons of the hypothalamus. However, clinical studies indicate that leptin is undesirable as a therapeutic regiment for obesity, which is at least partly attributed to the poorly understood complex secondary structure and key signaling mechanism of the leptin-responsive neural circuit. Here, we show that the LepR-expressing portal neurons send GABAergic projections to a cohort of α3-GABAA receptor expressing neurons within the dorsomedial hypothalamic nucleus (DMH) for the control of leptin-mediated obesity phenotype. We identified the DMH as a key brain region that contributes to the regulation of leptin-mediated feeding. Acute activation of the GABAergic AgRP-DMH circuit promoted food intake and glucose intolerance, while activation of post-synaptic MC4R neurons in the DMH elicited exactly opposite phenotypes. Rapid deletion of LepR from AgRP neurons caused an obesity phenotype which can be rescued by blockage of GABAA receptor in the DMH. Consistent with behavioral results, these DMH neurons displayed suppressed neural activities in response to hunger or hyperglycemia. Furthermore, we identified that α3-GABAA receptor signaling within the DMH exerts potent bi-directional regulation of the central effects of leptin on feeding and body weight. Together, our results demonstrate a novel GABAergic neural circuit governing leptin-mediated feeding and energy balance via a unique α3-GABAA signaling within the secondary leptin-responsive neural circuit, constituting a new avenue for therapeutic interventions in the treatment of obesity and associated comorbidities.

Early life adversity impaired dorsal striatal synaptic transmission and behavioral adaptability to appropriate action selection in a sex-dependent manner

Early life adversity (ELA) is a major health burden in the United States, with 62% of adults reporting at least one adverse childhood experience. These experiences during critical stages of brain development can perturb the development of neural circuits that mediate sensory cue processing and behavioral regulation. Recent studies have reported that ELA impaired the maturation of dendritic spines on neurons in the dorsolateral striatum (DLS) but not in the dorsomedial striatum (DMS). The DMS and DLS are part of two distinct corticostriatal circuits that have been extensively implicated in behavioral flexibility by regulating and integrating action selection with the reward value of those actions. To date, no studies have investigated the multifaceted effects of ELA on aspects of behavioral flexibility that require alternating between different action selection strategies or higher-order cognitive processes, and the underlying synaptic transmission in corticostriatal circuitries. To address this, we employed whole-cell patch-clamp electrophysiology to assess the effects of ELA on synaptic transmission in the DMS and DLS. We also investigated the effects of ELA on the ability to update action control in response to outcome devaluation in an instrumental learning paradigm and reversal of action-outcome contingency in a water T-maze paradigm. At the circuit level, ELA decreased corticostriatal glutamate transmission in male but not in female mice. Interestingly, in DMS, glutamate transmission is decreased in male ELA mice, but increased in female ELA mice. ELA impaired the ability to update action control in response to reward devaluation in a context that promotes goal-directedness in male mice and induced deficits in reversal learning. Overall, our findings demonstrate the sex- and region-dependent effects of ELA on behavioral flexibility and underlying corticostriatal glutamate transmission. By establishing a link between ELA and circuit mechanisms underlying behavioral flexibility, our findings will begin to identify novel molecular mechanisms that can represent strategies for treating behavioral inflexibility in individuals who experienced early life traumatic incidents.

Chemogenetic inhibition of a monosynaptic projection from the basolateral amygdala to the ventral hippocampus selectively reduces appetitive, but not consummatory, alcohol drinking-related behaviours

Alcohol use disorder (AUD) and anxiety/stressor disorders frequently co-occur and this dual diagnosis represents a major health and economic problem worldwide. The basolateral amygdala (BLA) is a key brain region that is known to contribute to the aetiology of both disorders. Although many studies have implicated BLA hyperexcitability in the pathogenesis of AUD and comorbid conditions, relatively little is known about the specific efferent projections from this brain region that contribute to these disorders. Recent optogenetic studies have shown that the BLA sends a strong monosynaptic excitatory projection to the ventral hippocampus (vHC) and that this circuit modulates anxiety- and fear-related behaviours. However, it is not known if this pathway influences alcohol drinking-related behaviours. Here, we employed a rodent operant self-administration regimen that procedurally separates appetitive (e.g. seeking) and consummatory (e.g., drinking) behaviours, chemogenetics and brain region-specific microinjections, to determine if BLA-vHC circuitry influences alcohol and sucrose drinking-related measures. We first confirmed prior optogenetic findings that silencing this circuit reduced anxiety-like behaviours on the elevated plus maze. We then demonstrated that inhibiting the BLA-vHC pathway significantly reduced appetitive drinking-related behaviours for both alcohol and sucrose while having no effect on consummatory measures. Taken together, these findings provide the first indication that the BLA-vHC circuit may regulate appetitive reward seeking directed at alcohol and natural rewards and add to a growing body of evidence suggesting that dysregulation of this pathway may contribute to the pathophysiology of AUD and anxiety/stressor-related disorders.

Translational Neuroscience Approaches to Understanding Autism

While autism spectrum disorder affects nearly 2% of children in the United States, little is known with certainty concerning the etiologies and brain systems involved. This is due, in part, to the substantial heterogeneity in the presentation of the core symptoms of autism as well as the great number of co-occurring conditions that are common in autistic individuals. Understanding the neurobiology of autism is further hampered by the limited availability of postmortem brain tissue to determine the cellular and molecular alterations that take place in the autistic brain. Animal models therefore provide great translational value in helping to define the neural systems that constitute the social brain and mediate repetitive behaviors or interests. If they are based on genetic or environmental factors that contribute to autism, organisms from flies to nonhuman primates may serve as models of the neural structure or function of the autistic brain. Ultimately, successful models can also be employed to test the safety and effectiveness of potential therapeutics. This is an overview of the major animal species that are currently used as models of autism, including an appraisal of the advantages and limitations of each.

Human habit neural circuitry may be perturbed in eating disorders

Circuit-based mechanisms mediating the development and execution of habitual behaviors involve complex cortical-striatal interactions that have been investigated in animal models and more recently in humans. However, how human brain circuits implicated in habit formation may be perturbed in psychiatric disorders remains unclear. First, we identified the locations of the sensorimotor putamen and associative caudate in the human brain using probabilistic tractography from Human Connectome Project data. We found that multivariate connectivity of the sensorimotor putamen was altered in humans with binge eating disorder and bulimia nervosa and that the degree of alteration correlated with severity of disordered eating behavior. Furthermore, the extent of this circuit aberration correlated with mean diffusivity in the sensorimotor putamen and decreased basal dopamine D2/3 receptor binding potential in the striatum, consistent with previously reported microstructural changes and dopamine signaling mediating habit learning in animal models. Our findings suggest a neural circuit that links habit learning and binge eating behavior in humans, which could, in part, explain the treatment-resistant behavior common to eating disorders and other psychiatric conditions.

Adolescent social isolation shifts the balance of decision-making strategy from goal-directed action to habitual response in adulthood via suppressing the excitatory neurotransmission onto the direct pathway of the dorsomedial striatum

Adverse experience, such as social isolation, during adolescence is one of the major causes of neuropsychiatric disorders that extend from adolescence into adulthood, such as substance addiction, obsessive-compulsive disorder, and eating disorders leading to obesity. A common behavioral feature of these neuropsychiatric disorders is a shift in the balance of decision-making strategy from goal-directed action to habitual response. This study has verified that adolescent social isolation directly shifts the balance of decision-making strategy from goal-directed action to habitual response, and that it cannot be reversed by simple regrouping. This study has further revealed that adolescent social isolation induces a suppression in the excitatory neurotransmission onto the direct-pathway medium spiny neurons of the dorsomedial striatum (DMS), and that chemogenetically compensating this suppression effect shifts the balance of decision-making strategy from habitual response back to goal-directed action. These findings suggest that the plasticity in the DMS causes the shift in the balance of decision-making strategy, which would potentially help to develop a general therapy to treat the various neuropsychiatric disorders caused by adolescent social isolation. Such a study is especially necessary under the circumstances that social distancing and lockdown have caused during times of world-wide, society-wide pandemic.

Inhibition of noradrenergic signalling in rodent orbitofrontal cortex impairs the updating of goal-directed actions

In a constantly changing environment, organisms must track the current relationship between actions and their specific consequences and use this information to guide decision-making. Such goal-directed behaviour relies on circuits involving cortical and subcortical structures. Notably, a functional heterogeneity exists within the medial prefrontal, insular, and orbitofrontal cortices (OFC) in rodents. The role of the latter in goal-directed behaviour has been debated, but recent data indicate that the ventral and lateral subregions of the OFC are needed to integrate changes in the relationships between actions and their outcomes. Neuromodulatory agents are also crucial components of prefrontal functions and behavioural flexibility might depend upon the noradrenergic modulation of the prefrontal cortex. Therefore, we assessed whether noradrenergic innervation of the OFC plays a role in updating action-outcome relationships in male rats. We used an identity-based reversal task and found that depletion or chemogenetic silencing of noradrenergic inputs within the OFC rendered rats unable to associate new outcomes with previously acquired actions. Silencing of noradrenergic inputs in the prelimbic cortex or depletion of dopaminergic inputs in the OFC did not reproduce this deficit. Together, our results suggest that noradrenergic projections to the OFC are required to update goal-directed actions.

Synaptic Effects Induced by Alcohol

Ethanol (EtOH) has effects on numerous cellular molecular targets, and alterations in synaptic function are prominent among these effects. Acute exposure to EtOH activates or inhibits the function of proteins involved in synaptic transmission, while chronic exposure often produces opposing and/or compensatory/homeostatic effects on the expression, localization, and function of these proteins. Interactions between different neurotransmitters (e.g., neuropeptide effects on release of small molecule transmitters) can also influence both acute and chronic EtOH actions. Studies in intact animals indicate that the proteins affected by EtOH also play roles in the neural actions of the drug, including acute intoxication, tolerance, dependence, and the seeking and drinking of EtOH. The present chapter is an update of our previous Lovinger and Roberto (Curr Top Behav Neurosci 13:31-86, 2013) chapter and reviews the literature describing these acute and chronic synaptic effects of EtOH with a focus on adult animals and their relevance for synaptic transmission, plasticity, and behavior.

Disinhibition of the orbitofrontal cortex biases decision-making in obesity

The lateral orbitofrontal cortex (lOFC) receives sensory information about food and integrates these signals with expected outcomes to guide future actions, and thus may play a key role in a distributed network of neural circuits that regulate feeding behavior. Here, we reveal a new role for the lOFC in the cognitive control of behavior in obesity. Food-seeking behavior is biased in obesity such that in male obese mice, behaviors are less flexible to changes in the perceived value of the outcome. Obesity is associated with reduced lOFC inhibitory drive and chemogenetic reduction in GABAergic neurotransmission in the lOFC induces obesity-like impairments in goal-directed behavior. Conversely, pharmacological or optogenetic restoration of inhibitory neurotransmission in the lOFC of obese mice reinstates flexible behavior. Our results indicate that obesity-induced disinhibition of the lOFC leads to a failure to update changes in the value of food with satiety, which in turn may influence how individuals make decisions in an obesogenic environment.

Orbitofrontal cortex populations are differentially recruited to support actions

The ability to use information from one's prior actions is necessary for decision-making. While orbitofrontal cortex (OFC) has been hypothesized as key for inferences made using cue and value-related information, whether OFC populations contribute to the use of information from volitional actions to guide behavior is not clear. Here, we used a self-paced lever-press hold-down task in which mice infer prior lever-press durations to guide subsequent action performance. We show that the activity of genetically identified lateral OFC (lOFC) subpopulations differentially instantiate current and prior action information during ongoing action execution. Transient state-dependent lOFC circuit disruptions of specified subpopulations reduced the encoding of ongoing press durations but did not disrupt the use of prior action information to guide future action performance. In contrast, a chronic functional loss of lOFC circuit activity resulted in increased reliance on recently executed lever-press durations and impaired contingency reversal, suggesting the recruitment of compensatory mechanisms that resulted in repetitive action control. Our results identify a novel role for lOFC in the integration of action information to guide adaptive behavior.

The role of dopamine and endocannabinoid systems in prefrontal cortex development: Adolescence as a critical period

The prefrontal cortex plays a central role in the control of complex cognitive processes including action control and decision making. It also shows a specific pattern of delayed maturation related to unique behavioral changes during adolescence and allows the development of adult cognitive processes. The adolescent brain is extremely plastic and critically vulnerable to external insults. Related to this vulnerability, adolescence is also associated with the emergence of numerous neuropsychiatric disorders involving alterations of prefrontal functions. Within prefrontal microcircuits, the dopamine and the endocannabinoid systems have widespread effects on adolescent-specific ontogenetic processes. In this review, we highlight recent advances in our understanding of the maturation of the dopamine system and the endocannabinoid system in the prefrontal cortex during adolescence. We discuss how they interact with GABA and glutamate neurons to modulate prefrontal circuits and how they can be altered by different environmental events leading to long-term neurobiological and behavioral changes at adulthood. Finally, we aim to identify several future research directions to help highlight gaps in our current knowledge on the maturation of these microcircuits.

HCN1 channels mediate mu opioid receptor long-term depression at insular cortex inputs to the dorsal striatum

Mu opioid receptors (MORs) are expressed in the dorsal striatum, a brain region that mediates goal-directed (via the dorsomedial striatum) and habitual (via the dorsolateral striatum, DLS) behaviours. Our previous work indicates that glutamate transmission is depressed when MORs are activated in the dorsal striatum, inducing MOR-mediated long-term synaptic depression (MOR-LTD) or short-term depression (MOR-STD), depending on the input. In the DLS, MOR-LTD is produced by MORs on anterior insular cortex (AIC) inputs and MOR-STD occurs at thalamic inputs, suggesting input-specific MOR plasticity mechanisms. Here, we evaluated the mechanisms of induction of MOR-LTD and MOR-STD in the DLS using pharmacology and optogenetics combined with patch-clamp electrophysiology. We found that cAMP/PKA signalling and protein synthesis are necessary for MOR-LTD expression, similar to previous studies of cannabinoid-mediated LTD in DLS. MOR-STD does not utilize these same mechanisms. We also demonstrated that cannabinoid-LTD occurs at AIC inputs to DLS. However, while cannabinoid-LTD requires mTOR signalling in DLS, MOR-LTD does not. We characterized the role of presynaptic HCN1 channels in MOR-LTD induction as HCN1 channels expressed in AIC are necessary for MOR-LTD expression in the DLS. These results suggest a mechanism in which MOR activation requires HCN1 to induce MOR-LTD, suggesting a new target for pharmacological modulation of synaptic plasticity, providing new opportunities to develop novel drugs to treat alcohol and opioid use disorders. KEY POINTS: Mu opioid receptor-mediated long-term depression at anterior insular cortex inputs to dorsolateral striatum involves presynaptic cAMP/PKA signalling and protein translation, similar to known mechanisms of cannabinoid long-term depression. Dorsal striatal cannabinoid long-term depression also occurs at anterior insular cortex inputs to the dorsolateral striatum. Dorsal striatal cannabinoid long-term depression requires mTOR signalling, similar to hippocampal cannabinoid long-term depression, but dorsal striatal mu opioid long-term depression does not require mTOR signalling. Mu opioid long-term depression requires presynaptic HCN1 channels at anterior insular cortex inputs to dorsolateral striatum.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.

Chemogenetic Disconnection between the Orbitofrontal Cortex and the Rostromedial Caudate Nucleus Disrupts Motivational Control of Goal-Directed Action

The orbitofrontal cortex (OFC) and its major downstream target within the basal ganglia-the rostromedial caudate nucleus (rmCD)-are involved in reward-value processing and goal-directed behavior. However, a causal contribution of the pathway linking these two structures to goal-directed behavior has not been established. Using the chemogenetic technology of designer receptors exclusively activated by designer drugs with a crossed inactivation design, we functionally and reversibly disrupted interactions between the OFC and rmCD in two male macaque monkeys. We injected an adeno-associated virus vector expressing an inhibitory designer receptor, hM4Di, into the OFC and contralateral rmCD, the expression of which was visualized in vivo by positron emission tomography and confirmed by postmortem immunohistochemistry. Functional disconnection of the OFC and rmCD resulted in a significant and reproducible loss of sensitivity to the cued reward value for goal-directed action. This decreased sensitivity was most prominent when monkeys had accumulated a certain amount of reward. These results provide causal evidence that the interaction between the OFC and the rmCD is needed for motivational control of action on the basis of the relative reward value and internal drive. This finding extends the current understanding of the physiological basis of psychiatric disorders in which goal-directed behavior is affected, such as obsessive-compulsive disorder.SIGNIFICANCE STATEMENT In daily life, we routinely adjust the speed and accuracy of our actions on the basis of the value of expected reward. Abnormalities in these kinds of motivational adjustments might be related to behaviors seen in psychiatric disorders such as obsessive-compulsive disorder. In the current study, we show that the connection from the orbitofrontal cortex to the rostromedial caudate nucleus is essential for motivational control of action in monkeys. This finding expands our knowledge about how the primate brain controls motivation and behavior and provides a particular insight into disorders like obsessive-compulsive disorder in which altered connectivity between the orbitofrontal cortex and the striatum has been implicated.

Cannabidiol (CBD) drives sex-dependent impairments in omission, but has no effect on reinforcer devaluation

Habits are inflexible behaviors that persist despite changes in outcome value. While habits allow for efficient responding, neuropsychiatric diseases such as drug addiction and obsessive-compulsive disorder are characterized by overreliance on habits. Recently, the commercially popular drug cannabidiol (CBD) has emerged as a potential treatment for addictive behaviors, though it is not entirely clear how it exerts this therapeutic effect. As brain endocannabinoids play a key role in habit formation, we sought to determine how CBD modifies goal-directed behaviors and habit formation. To explore this, mice were administered CBD (20mg/kg i.p.) or vehicle as a control and trained on random interval (RI30/60) or random ratio (RR10/20) schedules designed to elicit habitual or goal-directed lever pressing, respectively. Mice were tested for habitual responding using probe trials following reinforcer-specific devaluation as well as omission trials, where mice had to withhold responding to earn rewards. We found that while CBD had little effect on operant behaviors or reward devaluation, CBD inhibited goal-directed behavior in a sex-specific and contextdependent manner during the omission task. Beyond drug treatment, we found an effect of sex throughout training, reward devaluation, and omission. This work provides evidence that CBD has no effect on habit formation in a reward devaluation paradigm. However, the omission results suggest that CBD may slow learning of novel actionoutcome contingencies or decrease goal-directed behavior. This work calls for further examination of sex-dependent outcomes of CBD treatment and highlights the importance of investigating sex effects in habit-related experiments.

Effects of chronic alcohol exposure on motivation-based value updating

Dysfunctional decision-making has been observed in alcohol dependence. However, the specific underlying processes disrupted have yet to be identified. Important to goal-directed decision-making is one's motivational state, which is used to update the value of actions. As ethanol dependence disrupts decision-making processes, we hypothesized that ethanol dependence could alter sensitivity to motivational state and/or value updating, thereby reducing the capability for adaptive behavior. Here we employed a sequential instrumental learning task to examine this hypothesis. In two experiments, mice underwent chronic intermittent ethanol (CIE) or air (Air) vapor exposure and repeated withdrawal procedures to induce ethanol dependence. Mice were then trained on a sequence of distal and proximal lever pressing for sucrose under either mild or more severe food restriction. Half of all Air and CIE mice then underwent a motivational shift to a less hungry state and effects of this motivational shift were evaluated across three days. First, mice were re-exposed to sucrose, and effects of food restriction state and CIE exposure on lick and consummatory behavior were examined in the absence of lever pressing. Over the next two days, mice underwent a brief non-rewarded test and then a rewarded test where the ability to retrieve and infer sucrose value to guide lever pressing was measured. In the sucrose re-exposure session, prior CIE exposure altered sucrose-seeking in mice with a history of mild but not more severe food restriction, suggesting altered motivational sensitivity. During lever press testing, CIE mice were insensitive to decreases in motivational state and did not reduce proximal lever pressing regardless of food restriction state. Mildly restricted CIE mice, but not severely restricted CIE mice, also did not reduce distal pressing to the same degree as Air mice following a downshift in motivational state. Our findings suggest that ethanol dependence may disrupt motivational processes supporting value updating that are important for decision-making.

Dissecting the Prefrontal Network With Pathway-Selective Manipulation in the Macaque Brain—A Review

Macaque monkeys are prime animal models for studying the neural mechanisms of decision-making because of their close kinship with humans. Manipulation of neural activity during decision-making tasks is essential for approaching the causal relationship between the brain and its functions. Conventional manipulation methods used in macaque studies are coarse-grained, and have worked indiscriminately on mutually intertwined neural pathways. To systematically dissect neural circuits responsible for a variety of functions, it is essential to analyze changes in behavior and neural activity through interventions in specific neural pathways. In recent years, an increasing number of studies have applied optogenetics and chemogenetics to achieve fine-grained pathway-selective manipulation in the macaque brain. Here, we review the developments in macaque studies involving pathway-selective operations, with a particular focus on applications to the prefrontal network. Pathway selectivity can be achieved using single viral vector transduction combined with local light stimulation or ligand administration directly into the brain or double-viral vector transduction combined with systemic drug administration. We discuss the advantages and disadvantages of these methods. We also highlight recent technological developments in viral vectors that can effectively infect the macaque brain, as well as the development of methods to deliver photostimulation or ligand drugs to a wide area to effectively manipulate behavior. The development and dissemination of such pathway-selective manipulations of macaque prefrontal networks will enable us to efficiently dissect the neural mechanisms of decision-making and innovate novel treatments for decision-related psychiatric disorders.

Local modulation by presynaptic receptors controls neuronal communication and behaviour

Central nervous system neurons communicate via fast synaptic transmission mediated by ligand-gated ion channel (LGIC) receptors and slower neuromodulation mediated by G protein-coupled receptors (GPCRs). These receptors influence many neuronal functions, including presynaptic neurotransmitter release. Presynaptic LGIC and GPCR activation by locally released neurotransmitters influences neuronal communication in ways that modify effects of somatic action potentials. Although much is known about presynaptic receptors and their mechanisms of action, less is known about when and where these receptor actions alter release, especially in vivo. This Review focuses on emerging evidence for important local presynaptic receptor actions and ideas for future studies in this area.

Prenatal Opioid Exposure Impairs Endocannabinoid and Glutamate Transmission in the Dorsal Striatum

The opioid crisis has contributed to a growing population of children exposed to opioids during fetal development; however, many of the long-term effects of opioid exposure on development are unknown. We previously demonstrated that opioids have deleterious effects on endocannabinoid plasticity at glutamate synapses in the dorsal striatum of adolescent rodents, but it is unclear whether prenatal opioid exposure produces similar neuroadaptations. Using a mouse model of prenatal methadone exposure (PME), we performed proteomics, phosphoproteomics, and patch-clamp electrophysiology in the dorsolateral striatum (DLS) and dorsomedial striatum (DMS) to examine synaptic functioning in adolescent PME offspring. PME impacted the proteome and phosphoproteome in a region- and sex-dependent manner. Many proteins and phosphorylated proteins associated with glutamate transmission were differentially abundant in PME offspring, which was associated with reduced glutamate release in the DLS and altered the rise time of excitatory events in the DMS. Similarly, the intrinsic excitability properties of DMS neurons were significantly affected by PME. Last, pathway analyses revealed an enrichment in retrograde endocannabinoid signaling in the DLS, but not in the DMS, of males. Electrophysiology studies confirmed that endocannabinoid-mediated synaptic depression was impaired in the DLS, but not DMS, of PME-males. These results indicate that PME induces persistent neuroadaptations in the dorsal striatum and could contribute to the aberrant behavioral development described in offspring with prenatal opioid exposure.

Inter-individual variability amplified through breeding reveals control of reward-related action strategies by Melanocortin-4 Receptor in the dorsomedial striatum

In day-to-day life, we often must choose between pursuing familiar behaviors or adjusting behaviors when new strategies might be more fruitful. The dorsomedial striatum (DMS) is indispensable for arbitrating between old and new action strategies. To uncover molecular mechanisms, we trained mice to generate nose poke responses for food, then uncoupled the predictive relationship between one action and its outcome. We then bred the mice that failed to rapidly modify responding. This breeding created offspring with the same tendencies, failing to inhibit behaviors that were not reinforced. These mice had less post-synaptic density protein 95 in the DMS. Also, densities of the melanocortin-4 receptor (MC4R), a high-affinity receptor for α-melanocyte-stimulating hormone, predicted individuals' response strategies. Specifically, high MC4R levels were associated with poor response inhibition. We next found that reducing Mc4r in the DMS in otherwise typical mice expedited response inhibition, allowing mice to modify behavior when rewards were unavailable or lost value. This process required inputs from the orbitofrontal cortex, a brain region canonically associated with response strategy switching. Thus, MC4R in the DMS appears to propel reward-seeking behavior, even when it is not fruitful, while moderating MC4R presence increases the capacity of mice to inhibit such behaviors.

Thunderstruck: The ACDC model of flexible sequences and rhythms in recurrent neural circuits

Adaptive sequential behavior is a hallmark of human cognition. In particular, humans can learn to produce precise spatiotemporal sequences given a certain context. For instance, musicians can not only reproduce learned action sequences in a context-dependent manner, they can also quickly and flexibly reapply them in any desired tempo or rhythm without overwriting previous learning. Existing neural network models fail to account for these properties. We argue that this limitation emerges from the fact that sequence information (i.e., the position of the action) and timing (i.e., the moment of response execution) are typically stored in the same neural network weights. Here, we augment a biologically plausible recurrent neural network of cortical dynamics to include a basal ganglia-thalamic module which uses reinforcement learning to dynamically modulate action. This “associative cluster-dependent chain” (ACDC) model modularly stores sequence and timing information in distinct loci of the network. This feature increases computational power and allows ACDC to display a wide range of temporal properties (e.g., multiple sequences, temporal shifting, rescaling, and compositionality), while still accounting for several behavioral and neurophysiological empirical observations. Finally, we apply this ACDC network to show how it can learn the famous “Thunderstruck” song intro and then flexibly play it in a “bossa nova” rhythm without further training.

How do humans flexibly adapt action sequences? For instance, musicians can learn a song and quickly speed up or slow down the tempo, or even play the song following a completely different rhythm (e.g., a rock song using a bossa nova rhythm). In this work, we build a biologically plausible network of cortico-basal ganglia interactions that explains how this temporal flexibility may emerge in the brain. Crucially, our model factorizes sequence order and action timing, respectively represented in cortical and basal ganglia dynamics. This factorization allows full temporal flexibility, i.e. the timing of a learned action sequence can be recomposed without interfering with the order of the sequence. As such, our model is capable of learning asynchronous action sequences, and flexibly shift, rescale, and recompose them, while accounting for biological data.

Chronic alcohol drinking persistently suppresses thalamostriatal excitation of cholinergic neurons to impair cognitive flexibility

Exposure to addictive substances impairs flexible decision making. Cognitive flexibility is mediated by striatal cholinergic interneurons (CINs). However, how chronic alcohol drinking alters cognitive flexibility through CINs remains unclear. Here, we report that chronic alcohol consumption and withdrawal impaired reversal of instrumental learning. Chronic alcohol consumption and withdrawal also caused a long-lasting (21 days) reduction of excitatory thalamic inputs onto CINs and reduced pause responses of CINs in the dorsomedial striatum (DMS). CINs are known to inhibit glutamatergic transmission in dopamine D1 receptor–expressing medium spiny neurons (D1-MSNs) but facilitate this transmission in D2-MSNs, which may contribute to flexible behavior. We discovered that chronic alcohol drinking impaired CIN-mediated inhibition in D1-MSNs and facilitation in D2-MSNs. Importantly, in vivo optogenetic induction of long-term potentiation of thalamostriatal transmission in DMS CINs rescued alcohol-induced reversal learning deficits. These results demonstrate that chronic alcohol drinking reduces thalamic excitation of DMS CINs, compromising their regulation of glutamatergic transmission in MSNs, which may contribute to alcohol-induced impairment of cognitive flexibility. These findings provide a neural mechanism underlying inflexible drinking in alcohol use disorder.

Pyk2 Stabilizes Striatal Medium Spiny Neuron Structure and Striatal-Dependent Action

In day-to-day life, we often choose between pursuing familiar behaviors that have been rewarded in the past or adjusting behaviors when new strategies might be more fruitful. The dorsomedial striatum (DMS) is indispensable for flexibly arbitrating between old and new behavioral strategies. The way in which DMS neurons host stable connections necessary for sustained flexibility is still being defined. An entry point to addressing this question may be the structural scaffolds on DMS neurons that house synaptic connections. We find that the non-receptor tyrosine kinase Proline-rich tyrosine kinase 2 (Pyk2) stabilizes both dendrites and spines on striatal medium spiny neurons, such that Pyk2 loss causes dendrite arbor and spine loss. Viral-mediated Pyk2 silencing in the DMS obstructs the ability of mice to arbitrate between rewarded and non-rewarded behaviors. Meanwhile, the overexpression of Pyk2 or the closely related focal adhesion kinase (FAK) enhances this ability. Finally, experiments using combinatorial viral vector strategies suggest that flexible, Pyk2-dependent action involves inputs from the medial prefrontal cortex (mPFC), but not the ventrolateral orbitofrontal cortex (OFC). Thus, Pyk2 stabilizes the striatal medium spiny neuron structure, likely providing substrates for inputs, and supports the capacity of mice to arbitrate between novel and familiar behaviors, including via interactions with the medial-prefrontal cortex.

Relative salience signaling within a thalamo-orbitofrontal circuit governs learning rate

Learning to predict rewards is essential for the sustained fitness of animals. Contemporary views suggest that such learning is driven by a reward prediction error (RPE)-the difference between received and predicted rewards. The magnitude of learning induced by an RPE is proportional to the product of the RPE and a learning rate. Here we demonstrate using two-photon calcium imaging and optogenetics in mice that certain functionally distinct subpopulations of ventral/medial orbitofrontal cortex (vmOFC) neurons signal learning rate control. Consistent with learning rate control, trial-by-trial fluctuations in vmOFC activity positively correlate with behavioral updating when the RPE is positive, and negatively correlates with behavioral updating when the RPE is negative. Learning rate is affected by many variables including the salience of a reward. We found that the average reward response of these neurons signals the relative salience of a reward, because it decreases after reward prediction learning or the introduction of another highly salient aversive stimulus. The relative salience signaling in vmOFC is sculpted by medial thalamic inputs. These results support emerging theoretical views that prefrontal cortex encodes and controls learning parameters.

The learning of prospective and retrospective cognitive maps within neural circuits

Brain circuits are thought to form a "cognitive map" to process and store statistical relationships in the environment. A cognitive map is commonly defined as a mental representation that describes environmental states (i.e., variables or events) and the relationship between these states. This process is commonly conceptualized as a prospective process, as it is based on the relationships between states in chronological order (e.g., does reward follow a given state?). In this perspective, we expand this concept on the basis of recent findings to postulate that in addition to a prospective map, the brain forms and uses a retrospective cognitive map (e.g., does a given state precede reward?). In doing so, we demonstrate that many neural signals and behaviors (e.g., habits) that seem inflexible and non-cognitive can result from retrospective cognitive maps. Together, we present a significant conceptual reframing of the neurobiological study of associative learning, memory, and decision making.

Δ 9-Tetrahydrocannabinol During Adolescence Reprograms the Nucleus Accumbens Transcriptome, Affecting Reward Processing, Impulsivity, and Specific Aspects of Cocaine Addiction-Like Behavior in a Sex-Dependent Manner

BACKGROUND

Cannabis exposure during adolescence is associated with emotional and motivational alterations that may entail an enhanced risk of developing psychiatric disorders. In rodent models, exposure to cannabinoids during adolescence leads to increased self-administration of opiates and cocaine, however, the psychological and neural mechanisms and the sex-specificity of this phenomenon are largely unknown.

METHODS

We exposed male and female adolescent rats to Δ9-tetrahydrocannabinol (THC) and studied at adulthood the effects of such treatment on psychological processes related to reward, such as Pavlovian conditioned approach, Pavlovian to instrumental transfer, habit formation and waiting impulsivity. In the light of these data and given the involvement of the nucleus accumbens in the processes examined, we performed an RNASeq transcriptomic study and assessed cocaine addiction-like behavior.

RESULTS

THC exposure increased goal-tracking (in males and females) and enhanced Pavlovian to instrumental transfer (especially in males) but did not affect habit formation. THC-exposed rats exhibited subtle, state-dependent changes in premature responding in the 2-CSRTT task. RNASeq data showed gene expression alterations in a marked sex-specific manner. While no effects were found on the acquisition of cocaine self-administration or punished drug-seeking, rats exposed to THC self-administered more cocaine under a progressive ratio schedule (males), had a higher rebound upon returning to continuous access to the drug (females) and showed reduced drug-seeking after 30 days of withdrawal (females).

CONCLUSIONS

Adolescent THC affects specific aspects of reward- (and cocaine-) guided behavior and the function of a key brain region mediating these effects, in a remarkable sex-specific manner.