Dissociable roles for the nucleus accumbens core and shell in regulating set shifting

Distinct Action Signals by Subregions in the Nucleus Accumbens during STOP-Change Performance

The nucleus accumbens (NAc) is thought to contribute to motivated behavior by signaling the value of reward-predicting cues and the delivery of anticipated reward. The NAc is subdivided into core and shell, with each region containing different populations of neurons that increase or decrease firing to rewarding events. While there are numerous theories of functions pertaining to these subregions and cell types, most are in the context of reward processing, with fewer considering that the NAc might serve functions related to action selection more generally. We recorded from single neurons in the NAc as rats of both sexes performed a STOP-change task that is commonly used to study motor control and impulsivity. In this task, rats respond quickly to a spatial cue on 80% of trials (GO) and must stop and redirect planned movement on 20% of trials (STOP). We found that the activity of reward-excited neurons signaled accurate response direction on GO, but not STOP, trials and that these neurons exhibited higher precue firing after correct trials. In contrast, reward-inhibited neurons significantly represented response direction on STOP trials at the time of the instrumental response. Finally, the proportion of reward-excited to reward-inhibited neurons and the strength of precue firing decreased as the electrode traversed the NAc. We conclude that reward-excited cells (more common in core) promote proactive action selection, while reward-inhibited cells (more common in shell) contribute to accurate responding on STOP trials that require reactive suppression and redirection of behavior.

Boosting decision-making in rat models of early-life adversity with environmental enrichment and intranasal oxytocin

Impaired decision-making constitutes a fundamental issue in numerous psychiatric disorders. Extensive research has established that early life adversity (ELA) increases vulnerability to psychiatric disorders later in life. ELA in human neonates is associated with changes in cognitive, emotional, as well as reward-related processing. Maternal separation (MS) is an established animal model of ELA and has been shown to be associated with decision-making deficits. On the other hand, enriched environment (EE) and intranasal oxytocin (OT) administration have been demonstrated to have beneficial effects on decision-making in humans or animals. Given these considerations, our investigation sought to explore the impact of brief exposure to EE and intranasal OT administration on the decision-making abilities of adolescent rats that had experienced MS during infancy. The experimental protocol involved subjecting rat pups to the MS regimen for 180 min per day from postnatal day (PND) 1 to PND 21. Then, from PND 22 to PND 34, the rats were exposed to EE and/or received intranasal OT (2 μg/μl) for seven days. The assessment of decision-making abilities, using a rat gambling task (RGT), commenced during adolescence. Our findings revealed that MS led to impaired decision-making and a decreased percentage of advantageous choices. However, exposure to brief EE or intranasal OT administration mitigated the deficits induced by MS and improved the decision-making skills of maternally-separated rats. Furthermore, combination of these treatments did not yield additional benefits. These results suggest that EE and OT may hold promise as therapeutic interventions to enhance certain aspects of cognitive performance.

The dynamic strategy shifting task: Optimisation of an operant task for assessing cognitive flexibility in rats

Introduction

Although schizophrenia is associated with a broad range of symptoms including hallucinations, delusions, and reduced motivation, measures of cognitive dysfunction, including cognitive flexibility and executive function, are the strongest predictors of functional outcomes. Antipsychotic medications are useful for reducing psychotic symptoms, but they are ineffective at improving cognitive deficits. Despite extensive investment by industry, the transition from preclinical to clinical trials has not been successful for developing precognitive medications for individuals with schizophrenia. Here, we describe the optimisation of a novel dynamic strategy shifting task (DSST) using standard operant chambers to investigate the optimal stimuli required to limit the extensive training times required in previous tasks.

Methods

We determined that optimal learning by male and female Sprague Dawley rats for the flexibility task incorporated dynamic strategy shifts between spatial rules, such as following a visual cue or responding at one location, and non-spatial rules, such as responding to a central visual or auditory cue. A minimum of 6 correct consecutive responses were required to make a within-session change in the behavioural strategies. As a proof of concept, we trained and tested 84 Sprague Dawley rats on the DSST, and then assessed their cognitive flexibility using a within-subject design after an acute dose of ketamine (0, 3, 10 mg/kg). Rats made fewer premature and more perseverant responses to initiate a trial following ketamine. The effects of ketamine on trials to criterion was dependent on the rule.

Results

Ketamine induced a significant improvement on the reversal of a non-spatial visual discrimination rule. There was no significant effect of ketamine on the spatial visual or response discrimination rules.

Discussion

The DSST is a novel assay for studying distinct forms of cognitive flexibility and offers a rapid and adaptable means of assessing the ability to shift between increasingly challenging rule conditions. The DSST has potential utility in advancing our understanding of cognitive processes and the underlying neurobiological mechanisms related to flexibility in neuropsychiatric and neurological conditions where executive dysfunctions occur.>.

Characterization of striatal dopamine projections across striatal subregions in behavioral flexibility

Behavioural flexibility is key to survival in a dynamic environmentWhile flexible, goal-directed behaviours are initially dependent on dorsomedial striatum, they become dependent on lateral striatum as behaviours become inflexible. Similarly, lesions of dopamine terminals in lateral striatum disrupt the development of inflexible habits. This work suggests that dopamine release in lateral striatum may drive inflexible behaviours, though few studies have investigated a causative role of subpopulations of striatal dopamine terminals in reversal learning, a measure of flexibility. Here, we performed two optogenetic experiments to activate dopamine terminals in dorsomedial (DMS), dorsolateral (DLS) or ventral (nucleus accumbens [NAc]) striatum in DAT-Cre mice that expressed channelrhodopsin-2 via viral injection (Experiment I) or through transgenic breeding with an Ai32 reporter line (Experiment II) to determine how specific dopamine subpopulations impact reversal learning. Mice performed a reversal task in which they self-stimulated DMS, DLS, or NAc dopamine terminals by pressing one of two levers before action-outcome lever contingencies were reversed. Largely consistent with presumed ventromedial/lateral striatal function, we found that mice self-stimulating medial dopamine terminals reversed lever preference following contingency reversal, while mice self-stimulating NAc showed parial flexibility, and DLS self-stimulation resulted in impaired reversal. Impairments in DLS mice were characterized by more regressive errors and reliance on lose-stay strategies following reversal, as well as reduced within-session learning, suggesting reward insensitivity and overreliance on previously learned actions. This study supports a model of striatal function in which DMS and ventral dopamine facilitate goal-directed responding, and DLS dopamine supports more inflexible responding.

Kappa opioid receptor in nucleus accumbens regulates depressive-like behaviors following prolonged morphine withdrawal in mice

Prolonged withdrawal from opioids leads to negative emotions. Kappa opioid receptor (KOR) plays an important role in opioid addiction and affective disorders. However, the underlying mechanism of KOR in withdrawal-related depression is still lacking. We found that escitalopram treatment had a limited effect in improving depression symptoms in heroin-dependent patients. In mice, we demonstrated prolonged (4 weeks) but not acute (24 h) withdrawal from morphine induced depressive-like behaviors. The number of c-Fos positive cells and the expression of KOR in the nucleus accumbens (NAc), were significantly increased in the prolonged morphine withdrawal mice. Conditional KOR knockdown in NAc significantly improved depressive-like behaviors. Repeated but not acute treatment with the KOR antagonist norBNI improved depressive-like behaviors and reversed PSD95, synaptophysin, p-ERK, p-CREB, and BDNF in NAc. This study demonstrated the important role of striatal KOR in morphine withdrawal-related depressive-like behaviors and offered therapeutic potential for the treatment of withdrawal-related depression.

The Addiction-Susceptibility TaqIA/Ankk1 Controls Reward and Metabolism Through D2 Receptor-Expressing Neurons

BACKGROUND

A large body of evidence highlights the importance of genetic variants in the development of psychiatric and metabolic conditions. Among these, the TaqIA polymorphism is one of the most commonly studied in psychiatry. TaqIA is located in the gene that codes for the ankyrin repeat and kinase domain containing 1 kinase (Ankk1) near the dopamine D2 receptor (D2R) gene. Homozygous expression of the A1 allele correlates with a 30% to 40% reduction of striatal D2R, a typical feature of addiction, overeating, and other psychiatric pathologies. The mechanisms by which the variant influences dopamine signaling and behavior are unknown.

METHODS

Here, we used transgenic and viral-mediated strategies to reveal the role of Ankk1 in the regulation of activity and functions of the striatum.

RESULTS

We found that Ankk1 is preferentially enriched in striatal D2R-expressing neurons and that Ankk1 loss of function in the dorsal and ventral striatum leads to alteration in learning, impulsivity, and flexibility resembling endophenotypes described in A1 carriers. We also observed an unsuspected role of Ankk1 in striatal D2R-expressing neurons of the ventral striatum in the regulation of energy homeostasis and documented differential nutrient partitioning in humans with or without the A1 allele.

CONCLUSIONS

Overall, our data demonstrate that the Ankk1 gene is necessary for the integrity of striatal functions and reveal a new role for Ankk1 in the regulation of body metabolism.

Striatopallidal adenosine A2A receptor modulation of goal-directed behavior: Homeostatic control with cognitive flexibility

Dysfunction of goal-directed behaviors under stressful or pathological conditions results in impaired decision-making and loss of flexibility of thoughts and behaviors, which underlie behavioral deficits ranging from depression, obsessive-compulsive disorders and drug addiction. Tackling the neuromodulators fine-tuning this core behavioral element may facilitate the development of effective strategies to control these deficits present in multiple psychiatric disorders. The current investigation of goal-directed behaviors has concentrated on dopamine and glutamate signaling in the corticostriatal pathway. In accordance with the beneficial effects of caffeine intake on mood and cognitive dysfunction, we now propose that caffeine's main site of action - adenosine A2A receptors (A2AR) - represent a novel target to homeostatically control goal-directed behavior and cognitive flexibility. A2AR are abundantly expressed in striatopallidal neurons and colocalize and interact with dopamine D2, NMDA and metabotropic glutamate 5 receptors to integrate dopamine and glutamate signaling. Specifically, striatopallidal A2AR (i) exert an overall "break" control of a variety of cognitive processes, making A2AR antagonists a novel strategy for improving goal-directed behavior; (ii) confer homeostatic control of goal-directed behavior by acting at multiple sites with often opposite effects, to enhance cognitive flexibility; (iii) integrate dopamine and adenosine signaling through multimeric A2AR-D2R heterocomplexes allowing a temporally precise fine-tuning in response to local signaling changes. As the U.S. Food and Drug Administration recently approved the A2AR antagonist Nourianz® (istradefylline) to treat Parkinson's disease, striatal A2AR-mediated control of goal-directed behavior may offer a new and real opportunity for improving deficits of goal-directed behavior and enhance cognitive flexibility under various neuropsychiatric conditions. This article is part of the Special Issue on "Purinergic Signaling: 50 years".

Striatal dopamine supports reward expectation and learning: A simultaneous PET/fMRI study

Converging evidence from both human neuroimaging and animal studies has supported a model of mesolimbic processing underlying reward learning behaviors, based on the computation of reward prediction errors. However, competing evidence supports human dopamine signaling in the basal ganglia as also contributing to the generation of higher order learning heuristics. Here, we present data from a large (N = 81, 18-30yo), multi-modal neuroimaging study using simultaneously acquired task fMRI, affording temporal resolution of reward system function, and PET imaging with [¹¹C]Raclopride (RAC), assessing striatal dopamine (DA) D2/3 receptor binding, during performance of a probabilistic reward learning task. Both fMRI activation and PET DA measures showed ventral striatum involvement for signaling rewards. However, greater DA release was uniquely associated with learning strategies (i.e., learning rates) that were more task-optimal within the best fitting reinforcement learning model. This DA response was associated with BOLD activation of a network of regions including anterior cingulate cortex, medial prefrontal cortex, thalamus and posterior parietal cortex, primarily during expectation, rather than prediction error, task epochs. Together, these data provide novel, human in vivo evidence that striatal dopaminergic signaling interacts with a network of cortical regions to generate task-optimal learning strategies, rather than representing reward outcomes in isolation.

Medial septum activation improves strategy switching once strategies are well-learned via bidirectional regulation of dopamine neuron population activity

Strategy switching is a form of cognitive flexibility that requires inhibiting a previously successful strategy and switching to a new strategy of a different categorical modality. It is dependent on dopamine (DA) receptor activation and release in ventral striatum and prefrontal cortex, two primary targets of ventral tegmental area (VTA) DA projections. Although the circuitry that underlies strategy switching early in learning has been studied, few studies have examined it after extended discrimination training. This may be important as DA activity and release patterns change across learning, with several studies demonstrating a critical role for substantia nigra pars compacta (SNc) DA activity and release once behaviors are well-learned. We have demonstrated that medial septum (MS) activation simultaneously increased VTA and decreased SNc DA population activity, as well as improved reversal learning via these actions on DA activity. We hypothesized that MS activation would improve strategy switching both early in learning and after extended training through its ability to increase VTA DA population activity and decrease SNc DA population activity, respectively. We chemogenetically activated the MS of male and female rats and measured their performance on an operant-based strategy switching task following 1, 10, or 15 days of discrimination training. Contrary to our hypothesis, MS activation did not affect strategy switching after 1 day of discrimination training. MS activation improved strategy switching after 10 days of training, but only in females. MS activation improved strategy switching in both sexes after 15 days of training. Infusion of bicuculline into the ventral subiculum (vSub) inhibited the MS-mediated decrease in SNc DA population activity and attenuated the improvement in strategy switching. Intra-vSub infusion of scopolamine inhibited the MS-mediated increase in VTA DA population activity but did not affect the improvement in strategy switching. Intra-vSub infusion of both bicuculline and scopolamine inhibited the MS-mediated effects on DA population activity in both the SNc and VTA and completely prevented the improvement in strategy switching. These data indicate that MS activation improves strategy switching once the original strategy has been sufficiently well-learned, and that this may occur via the MS's regulation of DA neuron responsivity.

Ca2+-modulated photoactivatable imaging reveals neuron-astrocyte glutamatergic circuitries within the nucleus accumbens

Astrocytes are key elements of brain circuits that are involved in different aspects of the neuronal physiology relevant to brain functions. Although much effort is being made to understand how the biology of astrocytes affects brain circuits, astrocytic network heterogeneity and plasticity is still poorly defined. Here, we have combined structural and functional imaging of astrocyte activity recorded in mice using the Ca²⁺-modulated photoactivatable ratiometric integrator and specific optostimulation of glutamatergic pathways to map the functional neuron-astrocyte circuitries in the nucleus accumbens (NAc). We showed pathway-specific astrocytic responses induced by selective optostimulation of main inputs from the prefrontal cortex, basolateral amygdala, and ventral hippocampus. Furthermore, co-stimulation of glutamatergic pathways induced non-linear Ca²⁺-signaling integration, revealing integrative properties of NAc astrocytes. All these results demonstrate the existence of specific neuron-astrocyte circuits in the NAc, providing an insight to the understanding of how the NAc integrates information.

Neuron-astrocyte communication is fundamental for brain physiology, yet the heterogeneity in the functional interaction between these two elements remains poorly understood. Here we show how different neuron-astrocyte networks integrate information from distinct glutamatergic inputs.

Dopamine drives neuronal excitability via KCNQ channel phosphorylation for reward behavior

Dysfunctional dopamine signaling is implicated in various neuropsychological disorders. Previously, we reported that dopamine increases D1 receptor (D1R)-expressing medium spiny neuron (MSN) excitability and firing rates in the nucleus accumbens (NAc) via the PKA/Rap1/ERK pathway to promote reward behavior. Here, the results show that the D1R agonist, SKF81297, inhibits KCNQ-mediated currents and increases D1R-MSN firing rates in murine NAc slices, which is abolished by ERK inhibition. In vitro ERK phosphorylates KCNQ2 at Ser414 and Ser476; in vivo, KCNQ2 is phosphorylated downstream of dopamine signaling in NAc slices. Conditional deletion of Kcnq2 in D1R-MSNs reduces the inhibitory effect of SKF81297 on KCNQ channel activity, while enhancing neuronal excitability and cocaine-induced reward behavior. These effects are restored by wild-type, but not phospho-deficient KCNQ2. Hence, D1R-ERK signaling controls MSN excitability via KCNQ2 phosphorylation to regulate reward behavior, making KCNQ2 a potential therapeutical target for psychiatric diseases with a dysfunctional reward circuit.

Nucleus Accumbens Core Dopamine D2 Receptor-Expressing Neurons Control Reversal Learning but Not Set-Shifting in Behavioral Flexibility in Male Mice

The ability to use environmental cues to flexibly guide responses is crucial for adaptive behavior and is thought to be controlled within a series of cortico-basal ganglia-thalamo-cortical loops. Previous evidence has indicated that different prefrontal cortical regions control dissociable aspects of behavioral flexibility, with the medial prefrontal cortex (mPFC) necessary for the ability to shift attention to a novel strategy (set-shifting) and the orbitofrontal cortex (OFC) necessary for shifting attention between learned stimulus-outcome associations (reversal learning). The nucleus accumbens (NAc) is a major downstream target of both the mPFC and the OFC; however, its role in controlling reversal learning and set-shifting abilities is still unclear. Here we investigated the contribution of the two major NAc neuronal populations, medium spiny neurons expressing either dopamine D1 or D2 receptors (D1-/D2-MSNs), in guiding reversal learning and set-shifting in an attentional set-shifting task (ASST). Persistent inhibition of neurotransmitter release from NAc D2-MSNs, but not D1-MSNs, resulted in an impaired ability for reversal learning, but not set-shifting in male mice. These findings suggest that NAc D2-MSNs play a critical role in suppressing responding toward specific learned cues that are now associated with unfavorable outcomes (i.e., in reversal stages), but not in the suppression of more general learned strategies (i.e., in set-shifting). This study provides further evidence for the anatomical separation of reversal learning and set-shifting abilities within cortico-basal ganglia-thalamo-cortical loops.

Context coding in the mouse nucleus accumbens modulates motivationally relevant information

Neural activity in the nucleus accumbens (NAc) is thought to track fundamentally value-centric quantities linked to reward and effort. However, the NAc also contributes to flexible behavior in ways that are difficult to explain based on value signals alone, raising the question of if and how nonvalue signals are encoded in NAc. We recorded NAc neural ensembles while head-fixed mice performed an odor-based biconditional discrimination task where an initial discrete cue modulated the behavioral significance of a subsequently presented reward-predictive cue. We extracted single-unit and population-level correlates related to the cues and found value-independent coding for the initial, context-setting cue. This context signal occupied a population-level coding space orthogonal to outcome-related representations and was predictive of subsequent behaviorally relevant responses to the reward-predictive cues. Together, these findings support a gating model for how the NAc contributes to behavioral flexibility and provide a novel population-level perspective from which to view NAc computations.

Reduction of DNMT3a and RORA in the nucleus accumbens plays a causal role in post-traumatic stress disorder-like behavior: reversal by combinatorial epigenetic therapy

Post-traumatic stress disorder (PTSD) is an incapacitating trauma-related disorder, with no reliable therapy. Although PTSD has been associated with epigenetic alterations in peripheral white blood cells, it is unknown where such changes occur in the brain, and whether they play a causal role in PTSD. Using an animal PTSD model, we show distinct DNA methylation profiles of PTSD susceptibility in the nucleus accumbens (NAc). Data analysis revealed overall hypomethylation of different genomic CG sites in susceptible animals. This was correlated with the reduction in expression levels of the DNA methyltransferase, DNMT3a. Since epigenetic changes in diseases involve different gene pathways, rather than single candidate genes, we next searched for pathways that may be involved in PTSD. Analysis of differentially methylated sites identified enrichment in the RAR activation and LXR/RXR activation pathways that regulate Retinoic Acid Receptor (RAR) Related Orphan Receptor A (RORA) activation. Intra-NAc injection of a lentiviral vector expressing either RORA or DNMT3a reversed PTSD-like behaviors while knockdown of RORA and DNMT3a increased PTSD-like behaviors. To translate our results into a potential pharmacological therapeutic strategy, we tested the effect of systemic treatment with the global methyl donor S-adenosyl methionine (SAM), for supplementing DNA methylation, or retinoic acid, for activating RORA downstream pathways. We found that combined treatment with the methyl donor SAM and retinoic acid reversed PTSD-like behaviors. Thus, our data point to a novel approach to the treatment of PTSD, which is potentially translatable to humans.

Infralimbic cortex pyramidal neuron GIRK signaling contributes to regulation of cognitive flexibility but not affect-related behavior in male mice

Dysfunction of the infralimbic cortical (ILC) region of the medial prefrontal cortex (mPFC) is thought to be an underlying factor in both affect- and cognition-related behavioral deficits that co-occur across neuropsychiatric disorders. Increasing evidence highlights pathological imbalances in prefrontal pyramidal neuron excitability and associated aberrant firing as an underlying factor in this dysfunction. G protein-gated inwardly rectifying K⁺ (GIRK/Kir3) channels mediate excitability of mPFC pyramidal neurons, however the functional role of these channels in ILC-dependent regulation of behavior and pyramidal neuron excitation is unknown. The present study used a viral-cre approach in male mice harboring a 'floxed' version of the kcnj3 (Girk1) gene, to disrupt GIRK1-containing channel expression in pyramidal neurons within the ILC. Loss of GIRK1-dependent signaling increased excitability and spike firing of pyramidal neurons but did not alter affective behavior measured in an elevated plus maze, forced swim test, or progressive ratio test of motivation. Alternatively, ablation of GIRK1 impaired performance in an operant-based attentional set-shifting task designed to assess cognitive flexibility. These data highlight a unique role for GIRK1 signaling in ILC pyramidal neurons in the regulation of strategy shifting but not affect and suggest that these channels may represent a therapeutic target for treatment of cognitive deficits in neuropsychiatric disease.

Distributed cell assemblies spanning prefrontal cortex and striatum

Highly synchronous neuronal assembly activity is deemed essential for cognitive brain function. In theory, such synchrony could coordinate multiple brain areas performing complementary processes. However, cell assemblies have been observed only in single structures, typically cortical areas, and little is known about their synchrony with downstream subcortical structures, such as the striatum. Here, we demonstrate distributed cell assemblies activated at high synchrony (∼10 ms) spanning prefrontal cortex and striatum. In addition to including neurons at different brain hierarchical levels, surprisingly, they synchronized functionally distinct limbic and associative sub-regions. These assembly activations occurred when members shifted their firing phase relative to ongoing 4 Hz and theta rhythms, in association with high gamma oscillations. This suggests that these rhythms could mediate the emergence of cross-structural assemblies. To test for the role of assemblies in behavior, we trained the rats to perform a task requiring cognitive flexibility, alternating between two different rules in a T-maze. Overall, assembly activations were correlated with task-relevant parameters, including impending choice, reward, rule, or rule order. Moreover, these behavioral correlates were more robustly expressed by assemblies than by their individual member neurons. Finally, to verify whether assemblies can be endogenously generated, we found that they were indeed spontaneously reactivated during sleep and quiet immobility. Thus, cell assemblies are a more general coding mechanism than previously envisioned, linking distributed neocortical and subcortical areas at high synchrony.

Parallel and hierarchical neural mechanisms for adaptive and predictive behavioral control

Our brain can be recognized as a network of largely hierarchically organized neural circuits that operate to control specific functions, but when acting in parallel, enable the performance of complex and simultaneous behaviors. Indeed, many of our daily actions require concurrent information processing in sensorimotor, associative, and limbic circuits that are dynamically and hierarchically modulated by sensory information and previous learning. This organization of information processing in biological organisms has served as a major inspiration for artificial intelligence and has helped to create in silico systems capable of matching or even outperforming humans in several specific tasks, including visual recognition and strategy-based games. However, the development of human-like robots that are able to move as quickly as humans and respond flexibly in various situations remains a major challenge and indicates an area where further use of parallel and hierarchical architectures may hold promise. In this article we review several important neural and behavioral mechanisms organizing hierarchical and predictive processing for the acquisition and realization of flexible behavioral control. Then, inspired by the organizational features of brain circuits, we introduce a multi-timescale parallel and hierarchical learning framework for the realization of versatile and agile movement in humanoid robots.

Prefrontal deep projection neurons enable cognitive flexibility via persistent feedback monitoring

Cognitive flexibility, the ability to alter strategy according to changing stimulus-response-reward relationships, is critical for updating learned behavior. Attentional set-shifting, a test of cognitive flexibility, depends on the activity of prefrontal cortex (PFC). It remains unclear, however, what role PFC neurons play to support set-shifting. Using optogenetics and two-photon calcium imaging, we demonstrate that medial PFC activity does not bias sensorimotor responses during set-shifting, but rather enables set-shifting by encoding trial feedback information, a role it has been known to play in other contexts. Unexpectedly, the functional properties of PFC cells did not vary with their efferent projection targets. Instead, representations of trial feedback formed a topological gradient, with cells more strongly selective for feedback information located further from the pial surface, where afferent input from the anterior cingulate cortex was denser. These findings identify a critical role for deep PFC projection neurons in enabling set-shifting through behavioral feedback monitoring.

Differential effects of d- and l-enantiomers of govadine on distinct forms of cognitive flexibility and a comparison with dopaminergic drugs

RATIONALE

There is an urgent need for novel drugs for treating cognitive deficits that are defining features of schizophrenia. The individual d- and l-enantiomers of the tetrahydroprotoberberine (THPB) d,l-govadine have been proposed for the treatment of cognitive deficiencies and positive symptoms of schizophrenia, respectively.

OBJECTIVES

We examined the effects of d-, l-, or d,l-govadine on two distinct forms of cognitive flexibility perturbed in schizophrenia and compared them to those induced by a selective D1 receptor agonist and D2 receptor antagonist.

METHODS

Male rats received d-, l-, or d,l-govadine (0.3, 0.5, and 1.0 mg/kg), D1 agonist SKF81297(0.1, 0.3, and 1.0 mg/kg), or D2 antagonist haloperidol (0.1-0.2 mg/kg). Experiment 1 used a strategy set-shifting task (between-subjects). In experiment 2, well-trained rats were tested on a probabilistic reversal task (within-subjects).

RESULTS

d-Govadine improved set-shifting across all doses, whereas higher doses of l-govadine impaired set-shifting. SKF81297 reduced perseverative errors at the lowest dose. Low/high doses of haloperidol increased/decreased set-shifting errors, the latter "improvement" attributable to impaired retrieval of a previous acquired rule. Probabilistic reversal performance was less affected by these drugs, but d-govadine reduced errors during the first reversal, whereas l-govadine impaired initial discrimination learning. d,l-Govadine had no reliable cognitive effects but caused psychomotor slowing like l-govadine and haloperidol.

CONCLUSIONS

These findings further highlight differences between two enantiomers of d,l-govadine that may reflect differential modulation of D1 and D2 receptors. These preclinical findings give further impetus to formal clinical evaluation of d-govadine as a treatment for cognitive deficiencies related to schizophrenia.

Exploring the Role of the Nucleus Accumbens in Adaptive Behavior Using Concurrent Intracranial and Extracranial Electrophysiological Recordings in Humans

Recent human electrophysiological evidence implicated θ-band communication between the nucleus accumbens (NAc) and frontal and parietal cortex in cognitive flexibility. Since the NAc is connected with the motor system, we tested whether phase and amplitude-based NAc-cortical connectivity and power modulation likewise underlie flexibility in motor action control. We combined concurrently recorded intracranial and extracranial electroencephalograms from seven psychiatric patients implanted with deep brain stimulation (DBS) electrodes who performed a stop signal task (SST). Inhibition success, as opposed to failure, was associated with greater prestimulus information flow from right NAc to medial frontal cortex through phase coupling of θ oscillations. Inhibition failure evoked θ power increases in the left NAc and medial frontal cortex, whereas parieto-occipital cortex showed an α power decrease. We conclude that NAc-to-frontal θ connectivity, possibly facilitating processing of task-relevant information, and α and θ power modulations, possibly reflecting post-error engagement of cognitive control, contribute to adaptive behavior pertaining motor control.

Long-term effects of maternal cannabis vapor exposure on emotional reactivity, social behavior, and behavioral flexibility in offspring

The use of cannabis during pregnancy is a growing public health concern. As more countries implement legislation permitting recreational cannabis use, there is an urgent need to better understand its impact on fetal neurodevelopment and its long-term effects in exposed offspring. Studies examining effects of prenatal cannabis exposure typically employ injections of synthetic cannabinoids or isolated cannabis constituents that may not accurately model cannabis use in human populations. To address this limitation, we developed a novel e-cigarette technology-based system to deliver vaporized cannabis extracts to pregnant Long Evans rats. We used this model to determine effects of prenatal cannabis exposure on emotional, social, and cognitive endpoints of male and female offspring during early development and into adulthood. Dams were exposed to cannabis vapor (CAN_THC: 400 mg/ml), vehicle vapor (VEH), or no vapor (AIR) twice daily during mating and gestation. Offspring exposed to CAN_THC and VEH showed reduced weight gain relative to AIR offspring prior to weaning. CAN_THC offspring made more isolation-induced ultrasonic vocalizations (USVs) on postnatal day 6 (P6) relative to VEH-exposed offspring, which is indicative of increased emotional reactivity. Male CAN_THC offspring engaged in fewer social investigation behaviors than VEH-exposed male offspring during a social play test on P26. In adulthood, CAN_THC-exposed offspring spent less time exploring the open arms of the elevated plus maze and exhibited dose-dependent deficits in behavioral flexibility in an attentional set-shifting task relative to AIR controls. These data collectively indicate that prenatal cannabis exposure may cause enduring effects on the behavioral profile of offspring.

Deletion of the Mitochondrial Matrix Protein CyclophilinD Prevents Parvalbumin Interneuron Dysfunctionand Cognitive Deficits in a Mouse Model of NMDA Hypofunction

Redox dysregulation and oxidative stress are final common pathways in the pathophysiology of a variety of psychiatric disorders, including schizophrenia. Oxidative stress causes dysfunction of GABAergic parvalbumin (PV)-positive interneurons (PVI), which are crucial for the coordination of neuronal synchrony during sensory and cognitive processing. Mitochondria are the main source of reactive oxygen species (ROS) in neurons and they control synaptic activity through their roles in energy production and intracellular calcium homeostasis. We have previously shown that in male mice transient blockade of NMDA receptors (NMDARs) during development [subcutaneous injections of 30 mg/kg ketamine (KET) on postnatal days 7, 9, and 11] results in long-lasting alterations in synaptic transmission and reduced PV expression in the adult prefrontal cortex (PFC), contributing to a behavioral phenotype that mimics multiple symptoms associated with schizophrenia. These changes correlate with oxidative stress and impaired mitochondrial function in both PVI and pyramidal cells. Here, we show that genetic deletion (Ppif-/-) of the mitochondrial matrix protein cyclophilin D (CypD) prevents perinatal KET-induced increases in ROS and the resulting deficits in PVI function, and changes in excitatory and inhibitory synaptic transmission in the PFC. Deletion of CypD also prevented KET-induced behavioral deficits in cognitive flexibility, social interaction, and novel object recognition (NOR). Taken together, these data highlight how mitochondrial activity may play an integral role in modulating PVI-mediated cognitive processes.SIGNIFICANCE STATEMENT Mitochondria are important modulators of oxidative stress and cell function, yet how mitochondrial dysfunction affects cell activity and synaptic transmission in psychiatric illnesses is not well understood. NMDA receptor (NMDAR) blockade with ketamine (KET) during development causes oxidative stress, dysfunction of parvalbumin (PV)-positive interneurons (PVI), and long-lasting physiological and behavioral changes. Here we show that mice deficient for the mitochondrial matrix protein cyclophilin D (CypD) show robust protection from PVI dysfunction following perinatal NMDAR blockade. Mitochondria serve as an essential node for a number of stress-induced signaling pathways and our experiments suggest that failure of mitochondrial redox regulation can contribute to PVI dysfunction.

A high-resolution fMRI approach to characterize functionally distinct neural pathways within dopaminergic midbrain and nucleus accumbens during reward and salience processing

Processing of reward and salience without reward association are known to critically rely on the dopamine system. A growing body of evidence from animal studies suggests that both functions may be subserved by distinct subregions in midbrain and ventral striatum, specifically nucleus accumbens (NAcc). Yet in vivo investigation of these brain structures in humans has been rare. Here we examined blood oxygen level dependent signals in response to frequently presented rewarding events and infrequently presented neutral events in 20 healthy subjects using high-resolution functional magnetic resonance imaging (fMRI) for imaging the human midbrain and NAcc. The present findings revealed distinct activation patterns in brain regions of interest, namely increased activation in substantia nigra pars compacta (SNc) and dorsolateral NAcc in response to neutral events, while the VTA and both the ventromedial and dorsolateral NAcc were significantly activated due to rewarding events. Moreover, psychophysiological interaction analyses demonstrated regionally specialized processing pathways, such as a dorsolateral pathway when processing salience per se, i.e. increased functional interactions between SNc, dorsolateral NAcc and dorsolateral and medial prefrontal cortex (PFC); and a ventromedial pathway during reward processing, i.e. increased functional coupling between VTA and ventromedial NAcc. Thus, these findings may not only accelerate the integration of animal models of brain function with human neuroscience but may also improve diagnosis and treatment in patients with neuropsychiatric disorders such as schizophrenia and depression in which dopaminergic dysfunction and aberrant attribution of salience have been implicated.

Dorsal and ventral striatal dopamine D1 and D2 receptors differentially modulate distinct phases of serial visual reversal learning

Impaired cognitive flexibility in visual reversal-learning tasks has been observed in a wide range of neurological and neuropsychiatric disorders. Although both human and animal studies have implicated striatal D2-like and D1-like receptors (D2R; D1R) in this form of flexibility, less is known about the contribution they make within distinct sub-regions of the striatum and the different phases of visual reversal learning. The present study investigated the involvement of D2R and D1R during the early (perseverative) phase of reversal learning as well as in the intermediate and late stages (new learning) after microinfusions of D2R and D1R antagonists into the nucleus accumbens core and shell (NAcC; NAcS), the anterior and posterior dorsomedial striatum (DMS) and the dorsolateral striatum (DLS) on a touchscreen visual serial reversal-learning task. Reversal learning was improved after dopamine receptor blockade in the nucleus accumbens; the D1R antagonist, SCH23390, in the NAcS and the D2R antagonist, raclopride, in the NAcC selectively reduced early, perseverative errors. In contrast, reversal learning was impaired by D2R antagonism, but not D1R antagonism, in the dorsal striatum: raclopride increased errors in the intermediate phase after DMS infusions, and increased errors across phases after DLS infusions. These findings indicate that D1R and D2R modulate different stages of reversal learning through effects localised to different sub-regions of the striatum. Thus, deficits in behavioral flexibility observed in disorders linked to dopamine perturbations may be attributable to specific D1R and D2R dysfunction in distinct striatal sub-regions.

Cognitive rigidity and BDNF-mediated frontostriatal glutamate neuroadaptations during spontaneous nicotine withdrawal

Cognitive flexibility is the ability to switch strategic responses adaptively in changing environments. Cognitive rigidity imposed by neural circuit adaptations during nicotine abstinence may foster maladaptive nicotine taking in addicts. We systematically examined the effects of spontaneous withdrawal in mice exposed to either nicotine (6.3 or 18 mg/kg/day) or saline for 14 days on cognitive flexibility using an operant strategy set-shifting task. Because frontostriatal circuits are critical for cognitive flexibility and brain-derived neurotrophic factor (BDNF) modulates glutamate plasticity in these circuits, we also explored the effects of nicotine withdrawal on these neurochemical substrates. Mice undergoing nicotine withdrawal required more trials to attain strategy-switching criterion. Error analysis show that animals withdrawn from both nicotine doses committed higher perseverative errors, which correlated with measures of anxiety. However, animals treated with the higher nicotine dose also displayed more strategy maintenance errors that remained independent of negative affect. BDNF mRNA expression increased in the medial prefrontal cortex (mPFC) following nicotine withdrawal. Surprisingly, BDNF protein declined in mPFC but was elevated in dorsal striatum (DS). DS BDNF protein positively correlated with perseverative and maintenance errors, suggesting mPFC-DS overflow of BDNF during withdrawal. BDNF-evoked glutamate release and synapsin phosphorylation was attenuated within DS synapses, but enhanced in the nucleus accumbens, suggesting a dichotomous role of BDNF signaling in striatal regions. Taken together, these data suggest that spontaneous nicotine withdrawal impairs distinct components of cognitive set-shifting and these deficits may be linked to BDNF-mediated alterations in glutamate signaling dynamics in discrete frontostriatal circuits.

Targeting the cholinergic system in Parkinson's disease

Motor control in the striatum is an orchestra played by various neuronal populations. Loss of harmony due to dopamine deficiency is considered the primary pathological cause of the symptoms of Parkinson’s disease (PD). Recent progress in experimental approaches has enabled us to examine the striatal circuitry in a much more comprehensive manner, not only reshaping our understanding of striatal functions in movement regulation but also leading to new opportunities for the development of therapeutic strategies for treating PD. In addition to dopaminergic innervation, giant aspiny cholinergic interneurons (ChIs) within the striatum have long been recognized as a critical node for balancing dopamine signaling and regulating movement. With the roles of ChIs in motor control further uncovered and more specific manipulations available, striatal ChIs and their corresponding receptors are emerging as new promising therapeutic targets for PD. This review summarizes recent progress in functional studies of striatal circuitry and discusses the translational implications of these new findings for the treatment of PD.

Dose-dependent effects of estrogen on prediction error related neural activity in the nucleus accumbens of healthy young women

RATIONALE

Whereas the effect of the sex steroid 17-beta-estradiol (E2) on dopaminergic (DA) transmission in the nucleus accumbens (NAc) is well evidenced in female rats, studies in humans are inconsistent. Moreover, linear and inverted u-shaped dose response curves have been observed for E2's effects on hippocampal plasticity, but the shape of dose response curves for E2's effects on the NAc is much less characterized.

OBJECTIVES

Investigation of dose response curves for E2's effects on DA-related neural activity in the human NAc.

METHODS

Placebo or E2 valerate in doses of 2, 4, 6 or 12 mg was orally administered to 125 naturally cycling young women during the low-hormone menstruation phase on two consecutive days using a randomized, double-blinded design. The E2 treatment regimen induced a wide range of E2 levels, from physiological (2- and 4-mg groups; equivalent to cycle peak) to supraphysiological levels (6- and 12-mg groups; equivalent to early pregnancy). This made it possible to study different dose response functions for E2's effects on NAc activity. During E2 peak, participants performed a well-established reversal learning paradigm. We used trial-wise prediction errors (PE) estimated via a computational reinforcement learning model as a proxy for dopaminergic activity. Linear and quadratic regression analyses predicting PE-related NAc activity from salivary E2 levels were calculated.

RESULTS

There was a positive linear relationship between PE-associated NAc activity and salivary E2 increases.

CONCLUSIONS

The randomized, placebo-controlled elevation of E2 levels stimulates NAc activity in the human brain, likely mediated by dopaminergic processes.

The Nucleus Accumbens Core Is Necessary for Responding to Incentive But Not Instructive Stimuli

An abundant literature has highlighted the importance of the nucleus accumbens core (NAcC) in behavioral tasks dependent on external stimuli. Yet, some studies have also reported the absence of involvement of the NAcC in stimuli processing. We aimed at comparing, in male rats, the underlying neuronal determinants of incentive and instructive stimuli in the same task. We developed a variant of a GO/NOGO task that reveals important differences in these two types of stimuli. The incentive stimulus invites the rat to engage in the task sequence. Once the rat has decided to initiate a trial, it remains engaged in the task until the end of the trial. This task revealed the differential contribution of the NAcC to responding to different types of stimuli: responding to the incentive stimulus depended on NAcC AMPA/NMDA and dopamine D1 receptors, but the retrieval of the response associated with the instructive stimuli (lever pressing on GO, withholding on NOGO) did not. Our electrophysiological study showed that more NAcC neurons responded more strongly to the incentive than the instructive stimuli. Furthermore, when animals did not respond to the incentive stimulus, the induced excitation was suppressed for most projection neurons, whereas interneurons were strongly activated at a latency preceding that found in projection neurons. This work provides insight on the underlying neuronal processes explaining the preferential implication of the NAcC in deciding whether and when to engage in reward-seeking rather than to decide which action to perform.SIGNIFICANCE STATEMENT The nucleus accumbens core (NAcC) is essential to process information carried by reward-predicting stimuli. Yet, stimuli have distinct properties: incentive stimuli orient the attention toward reward-seeking, whereas instructive stimuli inform about the action to perform. Our study shows that, in male rats, NAcC perturbation with glutamate or dopamine antagonists impeded responses to the incentive but not to the instructive stimulus. NAcC neuronal recordings revealed a stronger representation of incentive than instructive stimuli. Furthermore, we found that interneurons are recruited when rats fail to respond to incentive stimuli. This work provides insight on the underlying neuronal processes explaining the preferential implication of the NAcC in deciding whether and when to engage in reward-seeking rather than to decide which action to perform.

The medial septum enhances reversal learning via opposing actions on ventral tegmental area and substantia nigra dopamine neurons

Cognitive flexibility deficits are one of the most pervasive symptoms across psychiatric disorders, making continued investigation of the circuitry underlying this function a top priority. Medial septum (MS) lesions lead to perseverative, inflexible-type behavior; however, a role for this region in cognitive flexibility circuitry has never been examined. We activated the MS (DREADDs) and measured performance in a T-maze spatial reversal learning task in male Sprague-Dawley rats. Systemic activation of the MS (CNO) significantly decreased both trials to perform a reversal and entries into the previously baited arm. Intra-ventral subiculum CNO enhanced reversal learning in the same manner as systemic CNO and also significantly increased ventral tegmental area and decreased substantia nigra dopamine neuron population activity. Finally, co-injection of the D1 antagonist SCH23390 with CNO prevented the enhanced reversal learning performance seen in the previous two experiments. Taken together, these data suggest a key role for the MS in cognitive flexibility, and suggest that MS-mediated changes in midbrain dopamine neuron population activity could be one mechanism by which this occurs.

Tonic exploration governs both flexibility and lapses

In many cognitive tasks, lapses (spontaneous errors) are tacitly dismissed as the result of nuisance processes like sensorimotor noise, fatigue, or disengagement. However, some lapses could also be caused by exploratory noise: randomness in behavior that facilitates learning in changing environments. If so, then strategic processes would need only up-regulate (rather than generate) exploration to adapt to a changing environment. This view predicts that more frequent lapses should be associated with greater flexibility because these behaviors share a common cause. Here, we report that when rhesus macaques performed a set-shifting task, lapse rates were negatively correlated with perseverative error frequency across sessions, consistent with a common basis in exploration. The results could not be explained by local failures to learn. Furthermore, chronic exposure to cocaine, which is known to impair cognitive flexibility, did increase perseverative errors, but, surprisingly, also improved overall set-shifting task performance by reducing lapse rates. We reconcile these results with a state-switching model in which cocaine decreases exploration by deepening attractor basins corresponding to rule states. These results support the idea that exploratory noise contributes to lapses, affecting rule-based decision-making even when it has no strategic value, and suggest that one key mechanism for regulating exploration may be the depth of rule states.

Persistent Alterations of Accumbal Cholinergic Interneurons and Cognitive Dysfunction after Adolescent Intermittent Ethanol Exposure

Adolescent binge drinking renders young drinkers vulnerable to alcohol use disorders in adulthood; therefore, understanding alcohol-induced brain damage and associated cognitive dysfunctions is of paramount importance. Here we investigated the effects of binge-like adolescent intermittent ethanol (AIE) exposure on nonspatial working memory, behavioral flexibility and cholinergic alterations in the nucleus accumbens (NAc) in male and female rats. On postnatal days P25-57 rats were intubated with water or ethanol (at a dose of 5 g/kg) on a 2-day-on/2-day-off cycle and were then tested in adulthood on social recognition and probabilistic reversal learning tasks. During the social recognition task AIE-treated rats spent similar amounts of time interacting with familiar and novel juveniles, indicating an impaired ability to sustain memory of the familiar juvenile. During probabilistic reversal learning, AIE-treated male and female rats showed behavioral inflexibility as indicated by a higher number of trials needed to complete three reversals within a session, longer response latencies for lever selection, and for males, a higher number of errors as compared to water-treated rats. AIE exposure also reduced the number of cholinergic interneurons in the NAc in males and females. These findings indicate AIE-related pathologies of accumbal cholinergic interneurons and long lasting cognitive-behavioral deficits, which may be associated with cortico-striatal hypofunction.

Accumbal Adenosine A2A Receptors Enhance Cognitive Flexibility by Facilitating Strategy Shifting

The deficits of cognitive flexibility (including attentional set-shifting and reversal learning) concomitant with dysfunction of the striatum are observed in several neuropsychiatric disorders. Rodent and human studies have identified the striatum [particularly the dorsomedial striatum (DMS) and nucleus accumbens (NAc)] as the critical locus for control of cognitive flexibility, but the effective neuromodulator and pharmacological control of cognitive flexibility remains to be determined. The adenosine A2A receptors (A2ARs) are highly enriched in the striatopallidal neurons where they integrate dopamine and glutamate signals to modulate several cognitive behaviors, but their contribution to cognitive flexibility control is unclear. In this study, by coupling an automated operant cognitive flexibility task with striatal subregional knockdown (KD) of the A2AR via the Cre-loxP strategy, we demonstrated that NAc A2AR KD improved cognitive flexibility with enhanced attentional set-shifting and reversal learning by decreasing regressive and perseverative errors, respectively. This facilitation was not attributed to mnemonic process or motor activity as NAc A2AR KD did not affect the visual discrimination, lever-pressing acquisition, and locomotor activity, but was associated with increased attention and motivation as evident by the progressive ratio test (PRT). In contrast to NAc A2ARs, DMS A2ARs KD neither affected visual discrimination nor improved set-shifting nor reversal learning, but promoted the effort-related motivation. Thus, NAc and DMS A2ARs exert dissociable controls of cognitive flexibility with NAc A2ARs KD selectively enhancing cognitive flexibility by facilitating strategy shifting with increased motivation/attention.

More rapid reversal learning following overtraining in the rat is evidence that behavioural and cognitive flexibility are dissociable

Cognitive flexibility is a term used to describe the brain processes underlying the phenomenon of adaptive change in behaviour in response to changed contingencies in the internal or external environment. Cognitive flexibility is often assessed in complex tasks measuring perceptual attentional shifting or response or task switching, but, arguably, reversal learning is a simple assay of cognitive flexibility. Reversal learning requires the detection of a changed outcome, the cessation of a previously-rewarded response and the selection of an alternative, previously-unrewarded, response. This study addressed the issue of the relationship between reversal learning and cognitive flexibility. In a single testing session, rats completed a series of 2-alternative forced-choice discriminations between digging bowls. The bowls differed according to both the medium within the bowl and the odor of the bowl. Having learned which cue (one of the odors or one of the digging media) indicated the food-baited bowl, half the rats were given additional trials of "over-training". To test reversal learning, the meaning of the cues predictive of reward/non-reward was then switched. There was a robust effect of over-training, with over-trained rats performing reversal learning in fewer trials than rats trained to criterion only. The pattern of errors supported the hypothesis that more rapid reversing results from the formation of an attentional set. This is the same attentional mechanism that results in less rapid shifting or switching. We conclude that the behavioural flexibility demonstrated in reversal learning does not provide a scale on which cognitive flexibility can be measured.

Neonatal handling impairs intradimensional shift and alters plasticity markers in the medial prefrontal cortex of adult rats

Stress response can be modulated by neonatal/childhood events. Neonatal handling (NH) is an animal model in which the animals are subjected to brief separations from the dam during the first days of life, and it leads to lower emotionality and behavioral changes in adulthood. The aim of this study was to observe if early events, such as (NH), may program associative learning and behavioral flexibility in adult male rats and if these changes could be related to altered neurochemistry in the medial prefrontal cortex (mPFC). We evaluated proteins related to synaptic plasticity (brain-derived neurotrophic factor [BDNF] and synaptophysin [SYP]) as well as Na⁺/K⁺-ATPase activity. Additionally, we evaluated proteins related to the dopaminergic system (tyrosine hydroxylase [TH] and phosphorylated TH [pTH]), since this system appears to be affected in some neonatal interventions. Neonatally handled animals exhibited impairment in simple discrimination and intradimensional shift but not in reversal or compound discrimination; in addition, no alteration in switching from an egocentric spatial to a cued strategy was observed. These effects were accompanied by a decrease in SYP levels and Na⁺/K⁺-ATPase activity, suggesting reduced synaptic function. These results indicate that NH increases attention to irrelevant stimuli and/or impairs associative learning, and this is accompanied by neurochemical alterations in the (mPFC).

Cell-Type-Specific Afferent Innervation of the Nucleus Accumbens Core and Shell

The nucleus accumbens (NAc) is clearly implicated in reward processing and drug addiction, as well as in numerous neurological and psychiatric disorders; nevertheless, the circuit mechanisms underlying the diverse functions of the NAc remain poorly understood. Here, we characterized the whole-brain and monosynaptic inputs to two main projection cell types – D1 dopamine receptor expressing medium spiny neurons (D1R-MSNs) and D2 dopamine receptor expressing medium spiny neurons (D2R-MSNs) – within the NAc core and NAc shell by rabies-mediated trans-synaptic tracing. We discovered that D1R-MSNs and D2R-MSNs in both NAc subregions receive similar inputs from diverse sources. Inputs to the NAc core are broadly scattered, whereas inputs to the NAc shell are relatively concentrated. Furthermore, we identified numerous brain areas providing important contrasting inputs to different NAc subregions. The anterior cortex preferentially innervates the NAc core for both D1R-MSNs and D2R-MSNs, whereas the lateral hypothalamic area (LH) preferentially targets D1R-MSNs in the NAc shell. Characterizing the cell-type-specific connectivity of different NAc subregions lays a foundation for studying how diverse functions of the NAc are mediated by specific pathways.

Persistent coding of outcome-predictive cue features in the rat nucleus accumbens

The nucleus accumbens (NAc) is important for learning from feedback, and for biasing and invigorating behaviour in response to cues that predict motivationally relevant outcomes. NAc encodes outcome-related cue features such as the magnitude and identity of reward. However, little is known about how features of cues themselves are encoded. We designed a decision making task where rats learned multiple sets of outcome-predictive cues, and recorded single-unit activity in the NAc during performance. We found that coding of cue identity and location occurred alongside coding of expected outcome. Furthermore, this coding persisted both during a delay period, after the rat made a decision and was waiting for an outcome, and after the outcome was revealed. Encoding of cue features in the NAc may enable contextual modulation of on-going behaviour, and provide an eligibility trace of outcome-predictive stimuli for updating stimulus-outcome associations to inform future behaviour.

Dopamine receptors mediate strategy abandoning via modulation of a specific prelimbic cortex-nucleus accumbens pathway in mice

The ability to abandon old strategies and adopt new ones is essential for survival in a constantly changing environment. While previous studies suggest the importance of the prefrontal cortex and some subcortical areas in the generation of strategy-switching flexibility, the fine neural circuitry and receptor mechanisms involved are not fully understood. In this study, we showed that optogenetic excitation and inhibition of the prelimbic cortex-nucleus accumbens (NAc) pathway in the mouse respectively enhances and suppresses strategy-switching ability in a cross-modal spatial-egocentric task. This ability is dependent on an intact dopaminergic tone in the NAc, as local dopamine denervation impaired the performance of the animal in the switching of tasks. In addition, based on a brain-slice preparation obtained from Drd2-EGFP BAC transgenic mice, we demonstrated direct innervation of D2 receptor-expressing medium spiny neurons (D2-MSNs) in the NAc by prelimbic cortical neurons, which is under the regulation by presynaptic dopamine receptors. While presynaptic D1-type receptor activation enhances the glutamatergic transmission from the prelimbic cortex to D2-MSNs, D2-type receptor activation suppresses this synaptic connection. Furthermore, manipulation of this pathway by optogenetic activation or administration of a D1-type agonist or a D2-type antagonist could restore impaired task-switching flexibility in mice with local NAc dopamine depletion; this restoration is consistent with the effects of knocking down the expression of specific dopamine receptors in the pathway. Our results point to a critical role of a specific prelimbic cortex-NAc subpathway in mediating strategy abandoning, allowing the switching from one strategy to another in problem solving.

Using a Simple Neural Network to Delineate Some Principles of Distributed Economic Choice

The brain uses a mixture of distributed and modular organization to perform computations and generate appropriate actions. While the principles under which the brain might perform computations using modular systems have been more amenable to modeling, the principles by which the brain might make choices using distributed principles have not been explored. Our goal in this perspective is to delineate some of those distributed principles using a neural network method and use its results as a lens through which to reconsider some previously published neurophysiological data. To allow for direct comparison with our own data, we trained the neural network to perform binary risky choices. We find that value correlates are ubiquitous and are always accompanied by non-value information, including spatial information (i.e., no pure value signals). Evaluation, comparison, and selection were not distinct processes; indeed, value signals even in the earliest stages contributed directly, albeit weakly, to action selection. There was no place, other than at the level of action selection, at which dimensions were fully integrated. No units were specialized for specific offers; rather, all units encoded the values of both offers in an anti-correlated format, thus contributing to comparison. Individual network layers corresponded to stages in a continuous rotation from input to output space rather than to functionally distinct modules. While our network is likely to not be a direct reflection of brain processes, we propose that these principles should serve as hypotheses to be tested and evaluated for future studies.

Differential Contributions of Nucleus Accumbens Subregions to Cue-Guided Risk/Reward Decision Making and Implementation of Conditional Rules

The nucleus accumbens (NAc) is a key node within corticolimbic circuitry for guiding action selection and cost/benefit decision making in situations involving reward uncertainty. Preclinical studies have typically assessed risk/reward decision making using assays where decisions are guided by internally generated representations of choice-outcome contingencies. Yet, real-life decisions are often influenced by external stimuli that inform about likelihoods of obtaining rewards. How different subregions of the NAc mediate decision making in such situations is unclear. Here, we used a novel assay colloquially termed the "Blackjack" task that models these types of situations. Male Long-Evans rats were trained to choose between one lever that always delivered a one-pellet reward and another that delivered four pellets with different probabilities [either 50% (good-odds) or 12.5% (poor-odds)], which were signaled by one of two auditory cues. Under control conditions, rats selected the large/risky option more often on good-odds versus poor-odds trials. Inactivation of the NAc core caused indiscriminate choice patterns. In contrast, NAc shell inactivation increased risky choice, more prominently on poor-odds trials. Additional experiments revealed that both subregions contribute to auditory conditional discrimination. NAc core or shell inactivation reduced Pavlovian approach elicited by an auditory CS+, yet shell inactivation also increased responding during presentation of a CS-. These data highlight distinct contributions for NAc subregions in decision making and reward seeking guided by discriminative stimuli. The core is crucial for implementation of conditional rules, whereas the shell refines reward seeking by mitigating the allure of larger, unlikely rewards and reducing expression of inappropriate or non-rewarded actions.SIGNIFICANCE STATEMENT Using external cues to guide decision making is crucial for adaptive behavior. Deficits in cue-guided behavior have been associated with neuropsychiatric disorders, such as attention deficit hyperactivity disorder and schizophrenia, which in turn has been linked to aberrant processing in the nucleus accumbens. However, many preclinical studies have often assessed risk/reward decision making in the absence of explicit cues. The current study fills that gap by using a novel task that allows for the assessment of cue-guided risk/reward decision making in rodents. Our findings identified distinct yet complementary roles for the medial versus lateral portions of this nucleus that provide a broader understanding of the differential contributions it makes to decision making and reward seeking guided by discriminative stimuli.

Assessment of intradimensional/extradimensional attentional set-shifting in rats

The rat intradimensional/extradimensional (ID/ED) task, first described by Birrell and Brown 18 years ago, has become the predominant means by which attentional set-shifting is investigated in rodents: the use of rats in the task has been described in over 135 publications by researchers from nearly 90 universities and pharmaceutical companies. There is variation in the protocols used by different groups, including differences in apparatus, stimuli (both stimulus dimensions and exemplars within), and also the methodology. Nevertheless, most of these variations seem to be of little consequence: there is remarkable similarity in the profile of published data, with consistency of learning rates and in the size and reliability of the set-shifting and reversal 'costs'. However, we suspect that there may be inconsistent data that is unpublished or perhaps 'failed experiments' that may have been caused by unintended deviations from effective protocols. The purpose of this review is to describe our approach and the rationale behind certain aspects of the protocol, including common pitfalls that are encountered when establishing an effective local protocol.

Ventral striatum lesions do not affect reinforcement learning with deterministic outcomes on slow time scales

A large body of work has implicated the ventral striatum (VS) in aspects of reinforcement learning (RL). However, less work has directly examined the effects of lesions in the VS, or other forms of inactivation, on 2-armed bandit RL tasks. We have recently found that lesions in the VS in macaque monkeys affect learning with stochastic schedules but have minimal effects with deterministic schedules. The reasons for this are not currently clear. Because our previous work used short intertrial intervals, one possibility is that the animals were using working memory to bridge stimulus-reward associations from 1 trial to the next. In the present study, we examined learning of 60 pairs of objects, in which the animals received only 1 trial per day with each pair. The large number of object pairs and the long interval (approximately 24 hr) between trials with a given pair minimized the chances that the animals could use working memory to bridge trials. We found that monkeys with VS lesions were unimpaired relative to controls, which suggests that animals with VS lesions can still learn to select rewarded objects even when they cannot make use of working memory. (PsycINFO Database Record

Rule Encoding in Orbitofrontal Cortex and Striatum Guides Selection

Active maintenance of rules, like other executive functions, is often thought to be the domain of a discrete executive system. An alternative view is that rule maintenance is a broadly distributed function relying on widespread cortical and subcortical circuits. Tentative evidence supporting this view comes from research showing some rule selectivity in the orbitofrontal cortex and dorsal striatum. We recorded in these regions and in the ventral striatum, which has not been associated previously with rule representation, as macaques performed a Wisconsin Card Sorting Task. We found robust encoding of rule category (color vs shape) and rule identity (six possible rules) in all three regions. Rule identity modulated responses to potential choice targets, suggesting that rule information guides behavior by highlighting choice targets. The effects that we observed were not explained by differences in behavioral performance across rules and thus cannot be attributed to reward expectation. Our results suggest that rule maintenance and rule-guided selection of options are distributed processes and provide new insight into orbital and striatal contributions to executive control.

SIGNIFICANCE STATEMENT

Rule maintenance, an important executive function, is generally thought to rely on dorsolateral brain regions. In this study, we examined activity of single neurons in orbitofrontal cortex and in ventral and dorsal striatum of macaques in a Wisconsin Card Sorting Task. Neurons in all three areas encoded rules and rule categories robustly. Rule identity also affected neural responses to potential choice options, suggesting that stored information is used to influence decisions. These results endorse the hypothesis that rule maintenance is a broadly distributed mental operation.

Effects of Ventral Striatum Lesions on Stimulus-Based versus Action-Based Reinforcement Learning

Learning the values of actions versus stimuli may depend on separable neural circuits. In the current study, we evaluated the performance of rhesus macaques with ventral striatum (VS) lesions on a two-arm bandit task that had randomly interleaved blocks of stimulus-based and action-based reinforcement learning (RL). Compared with controls, monkeys with VS lesions had deficits in learning to select rewarding images but not rewarding actions. We used a RL model to quantify learning and choice consistency and found that, in stimulus-based RL, the VS lesion monkeys were more influenced by negative feedback and had lower choice consistency than controls. Using a Bayesian model to parse the groups' learning strategies, we also found that VS lesion monkeys defaulted to an action-based choice strategy. Therefore, the VS is involved specifically in learning the value of stimuli, not actions.SIGNIFICANCE STATEMENT Reinforcement learning models of the ventral striatum (VS) often assume that it maintains an estimate of state value. This suggests that it plays a general role in learning whether rewards are assigned based on a chosen action or stimulus. In the present experiment, we examined the effects of VS lesions on monkeys' ability to learn that choosing a particular action or stimulus was more likely to lead to reward. We found that VS lesions caused a specific deficit in the monkeys' ability to discriminate between images with different values, whereas their ability to discriminate between actions with different values remained intact. Our results therefore suggest that the VS plays a specific role in learning to select rewarded stimuli.

New Variations for Strategy Set-shifting in the Rat

Behavioral flexibility is crucial for survival in changing environments. Broadly defined, behavioral flexibility requires a shift of behavioral strategy based on a change in governing rules. We describe a strategy set-shifting task that requires an attentional shift from one stimulus dimension to another. The paradigm is often used for testing cognitive flexibility in primates. However, the rodent version has not been as extensively developed. We have recently extended an established set-shifting task in the rat¹ by requiring attention to different stimuli according to context. All the experimental conditions required animals to choose either a left or right lever. Initially, all animals had to choose on the basis of the location of the lever. Subsequently, a change in the rule occurred, which required a shift in set from location-based rule to a rule in which the correct lever was indicated by a light cue. We compared performance on three different versions of the task, in which the light stimulus was either novel, previously relevant, or previously irrelevant. We found that specific neurochemical lesions selectively impaired the ability to make particular types of set shift as measured by the performance on the different versions of the task.