Nucleus accumbens core lesions retard instrumental learning and performance with delayed reinforcement in the rat

Selective chemogenetic inactivation of corticoaccumbal projections disrupts trait choice impulsivity

Impulsive choice has enduring trait-like characteristics and is defined by preference for small immediate rewards over larger delayed ones. Importantly, it is a determining factor in the development and persistence of substance use disorder (SUD). Emerging evidence from human and animal studies suggests frontal cortical regions exert influence over striatal reward processing areas during decision-making in impulsive choice or delay discounting (DD) tasks. The goal of this study was to examine how these circuits are involved in decision-making in animals with defined trait impulsivity. To this end, we trained adolescent male rats to stable behavior on a DD procedure and then re-trained them in adulthood to assess trait-like, conserved impulsive choice across development. We then used chemogenetic tools to selectively and reversibly target corticostriatal projections during performance of the DD task. The prelimbic region of the medial prefrontal cortex (mPFC) was injected with a viral vector expressing inhibitory designer receptors exclusively activated by designer drugs (Gi-DREADD), and then mPFC projections to the nucleus accumbens core (NAc) were selectively suppressed by intra-NAc administration of the Gi-DREADD actuator clozapine-n-oxide (CNO). Inactivation of the mPFC-NAc projection elicited a robust increase in impulsive choice in rats with lower vs. higher baseline impulsivity. This demonstrates a fundamental role for mPFC afferents to the NAc during choice impulsivity and suggests that maladaptive hypofrontality may underlie decreased executive control in animals with higher levels of choice impulsivity. Results such as these may have important implications for the pathophysiology and treatment of impulse control, SUDs, and related psychiatric disorders.

Error-related signaling in nucleus accumbens D2 receptor-expressing neurons guides inhibition-based choice behavior in mice

Learned associations between environmental cues and the outcomes they predict (cue-outcome associations) play a major role in behavioral control, guiding not only which responses we should perform, but also which we should inhibit, in order to achieve a specific goal. The encoding of such cue-outcome associations, as well as the performance of cue-guided choice behavior, is thought to involve dopamine D1 and D2 receptor-expressing medium spiny neurons (D1-/D2-MSNs) of the nucleus accumbens (NAc). Here, using a visual discrimination task in male mice, we assessed the role of NAc D1-/D2-MSNs in cue-guided inhibition of inappropriate responding. Cell-type specific neuronal silencing and in-vivo imaging revealed NAc D2-MSNs to contribute to inhibiting behavioral responses, with activation of NAc D2-MSNs following response errors playing an important role in optimizing future choice behavior. Our findings indicate that error-signaling by NAc D2-MSNs contributes to the ability to use environmental cues to inhibit inappropriate behavior.

Activating the dorsomedial and ventral midbrain projections to the striatum differentially impairs goal-directed action in male mice

The cognitive symptoms of schizophrenia are wide ranging and include impaired goal-directed action. This could be driven by an increase in dopamine transmission in the dorsomedial striatum, a pathophysiological hallmark of schizophrenia. Although commonly associated with psychotic symptoms, dopamine signalling in this region also modulates associative learning that aids in the execution of actions. To gain a better understanding of the role of subcortical dopamine in learning and decision-making, we assessed goal-directed action in male mice using the cross-species outcome-specific devaluation task (ODT). First, we administered systemic amphetamine during training to determine the impact of altered dopaminergic signaling on associative learning. Second, we used pathway-specific chemogenetic approaches to activate the dorsomedial and ventral striatal pathways (that originate in the midbrain) to separately assess learning and performance. Amphetamine treatment during learning led to a dose-dependent impairment in goal-directed action. Activation of both striatal pathways during learning also impaired performance. However, when these pathways were activated during choice, only activation of the ventral pathway impaired goal-directed action. This suggests that elevated transmission in the dorsomedial striatal pathway impairs associative learning processes that guide the goal-directed execution of actions. By contrast, elevated transmission of the ventral striatal pathway disrupts the encoding of outcome values that are important for both associative learning and choice performance. These findings highlight the differential roles of the dorsomedial and ventral inputs into the striatum in goal-directed action and provides insight into how striatal dopamine signaling may contribute to the cognitive problems in those with schizophrenia.

The ventral striatum contributes to the activity of the motor cortex and motor outputs in monkeys

The ventral striatum (VSt) is thought to be involved in the vigor of motivated behavior and is suggested to be a limbic-motor interface between limbic areas involved in motivational processes and neural circuits regulating behavioral outputs. However, there is little direct evidence demonstrating the involvement of the VSt in motor control for motivated behaviors. To clarify the functional role of the VSt in motor control, we investigated the effect of reversible pharmacological inactivation of the VSt on the oscillatory activity of the sensorimotor cortices and motor outputs in two macaque monkeys. VSt inactivation reduced movement-related activities of the primary motor cortex and premotor area at 15–120 Hz and increased those at 5–7 Hz. These changes were accompanied by reduced torque outputs but had no effect on the correct performance rate. The present study provides direct evidence that the VSt regulates activities of the motor cortices and motor output.

Choice-selective sequences dominate in cortical relative to thalamic inputs to NAc to support reinforcement learning

How are actions linked with subsequent outcomes to guide choices? The nucleus accumbens, which is implicated in this process, receives glutamatergic inputs from the prelimbic cortex and midline regions of the thalamus. However, little is known about whether and how representations differ across these input pathways. By comparing these inputs during a reinforcement learning task in mice, we discovered that prelimbic cortical inputs preferentially represent actions and choices, whereas midline thalamic inputs preferentially represent cues. Choice-selective activity in the prelimbic cortical inputs is organized in sequences that persist beyond the outcome. Through computational modeling, we demonstrate that these sequences can support the neural implementation of reinforcement-learning algorithms, in both a circuit model based on synaptic plasticity and one based on neural dynamics. Finally, we test and confirm a prediction of our circuit models by direct manipulation of nucleus accumbens input neurons.

Impulsivity and risk-seeking as Bayesian inference under dopaminergic control

Bayesian models successfully account for several of dopamine (DA)'s effects on contextual calibration in interval timing and reward estimation. In these models, tonic levels of DA control the precision of stimulus encoding, which is weighed against contextual information when making decisions. When DA levels are high, the animal relies more heavily on the (highly precise) stimulus encoding, whereas when DA levels are low, the context affects decisions more strongly. Here, we extend this idea to intertemporal choice and probability discounting tasks. In intertemporal choice tasks, agents must choose between a small reward delivered soon and a large reward delivered later, whereas in probability discounting tasks, agents must choose between a small reward that is always delivered and a large reward that may be omitted with some probability. Beginning with the principle that animals will seek to maximize their reward rates, we show that the Bayesian model predicts a number of curious empirical findings in both tasks. First, the model predicts that higher DA levels should normally promote selection of the larger/later option, which is often taken to imply that DA decreases 'impulsivity,' and promote selection of the large/risky option, often taken to imply that DA increases 'risk-seeking.' However, if the temporal precision is sufficiently decreased, higher DA levels should have the opposite effect-promoting selection of the smaller/sooner option (higher impulsivity) and the small/safe option (lower risk-seeking). Second, high enough levels of DA can result in preference reversals. Third, selectively decreasing the temporal precision, without manipulating DA, should promote selection of the larger/later and large/risky options. Fourth, when a different post-reward delay is associated with each option, animals will not learn the option-delay contingencies, but this learning can be salvaged when the post-reward delays are made more salient. Finally, the Bayesian model predicts correlations among behavioral phenotypes: Animals that are better timers will also appear less impulsive.

Neuroplasticity and Multilevel System of Connections Determine the Integrative Role of Nucleus Accumbens in the Brain Reward System

A growing body of evidence suggests that nucleus accumbens (NAc) plays a significant role not only in the physiological processes associated with reward and satisfaction but also in many diseases of the central nervous system. Summary of the current state of knowledge on the morphological and functional basis of such a diverse function of this structure may be a good starting point for further basic and clinical research. The NAc is a part of the brain reward system (BRS) characterized by multilevel organization, extensive connections, and several neurotransmitter systems. The unique role of NAc in the BRS is a result of: (1) hierarchical connections with the other brain areas, (2) a well-developed morphological and functional plasticity regulating short- and long-term synaptic potentiation and signalling pathways, (3) cooperation among several neurotransmitter systems, and (4) a supportive role of neuroglia involved in both physiological and pathological processes. Understanding the complex function of NAc is possible by combining the results of morphological studies with molecular, genetic, and behavioral data. In this review, we present the current views on the NAc function in physiological conditions, emphasizing the role of its connections, neuroplasticity processes, and neurotransmitter systems.

Opposing roles for striatonigral and striatopallidal neurons in dorsolateral striatum in consolidating new instrumental actions

Comparatively little is known about how new instrumental actions are encoded in the brain. Using whole-brain c-Fos mapping, we show that neural activity is increased in the anterior dorsolateral striatum (aDLS) of mice that successfully learn a new lever-press response to earn food rewards. Post-learning chemogenetic inhibition of aDLS disrupts consolidation of the new instrumental response. Similarly, post-learning infusion of the protein synthesis inhibitor anisomycin into the aDLS disrupts consolidation of the new response. Activity of D1 receptor-expressing medium spiny neurons (D1-MSNs) increases and D2-MSNs activity decreases in the aDLS during consolidation. Chemogenetic inhibition of D1-MSNs in aDLS disrupts the consolidation process whereas D2-MSN inhibition strengthens consolidation but blocks the expression of previously learned habit-like responses. These findings suggest that D1-MSNs in the aDLS encode new instrumental actions whereas D2-MSNs oppose this new learning and instead promote expression of habitual actions.

Opioid-induced structural and functional plasticity of medium-spiny neurons in the nucleus accumbens

Opioid Use Disorder (OUD) is a chronic relapsing clinical condition with tremendous morbidity and mortality that frequently persists, despite treatment, due to an individual's underlying psychological, neurobiological, and genetic vulnerabilities. Evidence suggests that these vulnerabilities may have neurochemical, cellular, and molecular bases. Key neuroplastic events within the mesocorticolimbic system that emerge through chronic exposure to opioids may have a determinative influence on behavioral symptoms associated with OUD. In particular, structural and functional alterations in the dendritic spines of medium spiny neurons (MSNs) within the nucleus accumbens (NAc) and its dopaminergic projections from the ventral tegmental area (VTA) are believed to facilitate these behavioral sequelae. Additionally, glutamatergic neurons from the prefrontal cortex, the basolateral amygdala, the hippocampus, and the thalamus project to these same MSNs, providing an enriched target for synaptic plasticity. Here, we review literature related to neuroadaptations in NAc MSNs from dopaminergic and glutamatergic pathways in OUD. We also describe new findings related to transcriptional, epigenetic, and molecular mechanisms in MSN plasticity in the different stages of OUD.

The transition to compulsion in addiction

Compulsion is a cardinal symptom of drug addiction (severe substance use disorder). However, compulsion is observed in only a small proportion of individuals who repeatedly seek and use addictive substances. Here, we integrate accounts of the neuropharmacological mechanisms that underlie the transition to compulsion with overarching learning theories, to outline how compulsion develops in addiction. Importantly, we emphasize the conceptual distinctions between compulsive drug-seeking behaviour and compulsive drug-taking behaviour (that is, use). In the latter, an individual cannot stop using a drug despite major negative consequences, possibly reflecting an imbalance in frontostriatal circuits that encode reward and aversion. By contrast, an individual may compulsively seek drugs (that is, persist in seeking drugs despite the negative consequences of doing so) when the neural systems that underlie habitual behaviour dominate goal-directed behavioural systems, and when executive control over this maladaptive behaviour is diminished. This distinction between different aspects of addiction may help to identify its neural substrates and new treatment strategies.

Molecular Plasticity of the Nucleus Accumbens Revisited-Astrocytic Waves Shall Rise

Part of the ventral striatal division, the nucleus accumbens (NAc) drives the circuit activity of an entire macrosystem about reward like a "flagship," signaling and leading diverse conducts. Accordingly, NAc neurons feature complex inhibitory phenotypes that assemble to process circuit inputs and generate outputs by exploiting specific arrays of opposite and/or parallel neurotransmitters, neuromodulatory peptides. The resulting complex combinations enable versatile yet specific forms of accumbal circuit plasticity, including maladaptive behaviors. Although reward signaling and behavior are elaborately linked to neuronal circuit activities, it is plausible to propose whether these neuronal ensembles and synaptic islands can be directly controlled by astrocytes, a powerful modulator of neuronal activity. Pioneering studies showed that astrocytes in the NAc sense citrate cycle metabolites and/or ATP and may induce recurrent activation. We argue that the astrocytic calcium, GABA, and Glu signaling and altered sodium and chloride dynamics fundamentally shape metaplasticity by providing active regulatory roles in the synapse- and network-level flexibility of the NAc.

Cognitive and behavioral training interventions to promote self-control

This review article discusses various cognitive and behavioral interventions that have been developed with the goal of promoting self-controlled responding. Self-control can exert a significant impact on human health and impulsive behaviors are associated with a wide range of diseases and disorders, leading to the suggestion that impulsivity is a trans-disease process. The self-control interventions include effort exposure, reward discrimination, reward bundling, interval schedules of reinforcement, impulse control training, and mindfulness training. Most of the interventions have been consistently shown to increase self-control, except for mindfulness training. Some of the successful interventions are long-lasting, whereas others may be transient. Most interventions are domain-specific, targeting specific cognitive and behavioral processes that relate to self-control rather than targeting overall self-control. For example, effort exposure appears to primarily increase effort tolerance, which in turn can improve self-control. Similarly, interval schedules primarily target interval timing, which promotes self-controlled responses. A diagram outlining a proposed set of intervention effects on self-control is introduced to motivate further research in this area. The diagram suggests that the individual target processes of the interventions may potentially summate to produce general self-control, or perhaps even produce synergistic effects. In addition, it is suggested that developing a self-control profile may be advantageous for aligning specific interventions to mitigate specific deficits. Overall, the results indicate that interventions are a promising avenue for promoting self-control and may help to contribute to changing health outcomes associated with a wide variety of diseases and disorders. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

Nucleus accumbens core lesions induce sub-optimal choice and reduce sensitivity to magnitude and delay in impulsive choice tasks

The nucleus accumbens core (NAc) has long been recognized as an important contributor to the computation of reward value that is critical for impulsive choice behavior. Impulsive choice refers to choosing a smaller-sooner (SS) over a larger-later (LL) reward when the LL is more optimal in terms of the rate of reward delivery. Two experiments examined the role of the NAc in impulsive choice and its component processes of delay and magnitude processing. Experiment 1 delivered an impulsive choice task with manipulations of LL reward magnitude, followed by a reward magnitude discrimination task. Experiment 2 tested impulsive choice under manipulations of LL delay, followed by temporal bisection and progressive interval tasks. NAc lesions, in comparison to sham control lesions, produced suboptimal preferences that resulted in lower reward earning rates, and led to reduced sensitivity to magnitude and delay within the impulsive choice task. The secondary tasks revealed intact reward magnitude and delay discrimination abilities, but the lesion rats persisted in responding more as the progressive interval increased during the session. The results suggest that the NAc is most critical for demonstrating good sensitivity to magnitude and delay, and adjusting behavior accordingly. Ultimately, the NAc lesions induced suboptimal choice behavior rather than simply promoting impulsive choice, suggesting that an intact NAc is necessary for optimal decision making.

Nucleus Accumbens Shell Dopamine Preferentially Tracks Information Related to Outcome Value of Reward

Effective decision-making requires organisms to predict reward values and bias behavior toward the best available option. The mesolimbic dopamine system, including the nucleus accumbens (NAc) shell and core, is involved in this process. Although studies support a role of the shell and core in specific aspects of decision-making (e.g., risk, effort, delay), no studies have directly compared dopamine release dynamics in these subregions to cues exclusively signaling the availability of different reward magnitudes. Here, fast-scan cyclic voltammetry was used to compare rapid dopamine release dynamics in the NAc subregions during a magnitude-based decision-making task. Rats learned that distinct cues signaled the availability of either a small or large reward (one or two sugar pellets), and then were given an opportunity to choose their preferred option. We found that peak dopamine release tracked the more preferred (higher-magnitude) option in both core and shell subregions. Critically, however, overall (i.e., global) dopamine release was significantly higher and longer lasting in the shell and tracked the preferred magnitude during the entire cue period. Further, in the shell (not core), dopamine signaling significantly declined immediately at the lever press for reward but increased during the period of reward consumption. Collectively, the results indicate that although dopamine release in both the core and shell are activated by cues signaling the opportunity to respond for rewards of different magnitudes, dopamine release in the shell plays a differential and unique role in tracking information related to the outcome value of reward.

Exploring the neural mechanisms of finasteride: a proteomic analysis in the nucleus accumbens

The enzyme 5α-reductase (5αR) catalyzes the conversion of progesterone and testosterone into neuroactive steroids implicated in a wide array of behavioral functions. The prototypical 5αR inhibitor, finasteride (FIN), is clinically approved for the treatment of androgenic alopecia and benign prostatic hyperplasia. Recent evidence has shown that FIN, albeit generally well tolerated, can induce untoward psychological effects in a subset of patients; furthermore, this drug may have therapeutic efficacy for a number of different neuropsychiatric conditions, ranging from Tourette syndrome to schizophrenia. In rat models of these conditions, FIN has been shown to block the effects of dopamine receptors in the nucleus accumbens (NAcc), a key terminal of the dopamine mesolimbic system. The biological underpinnings of these effects, however, remain mostly elusive. To elucidate the neurochemical networks that may be responsible for the behavioral effects of FIN, we evaluated the proteomic profile of the NAcc following acute (100mg/kg, IP) and subchronic (7 days; 100mg/kg/day, IP) treatment with this drug, in comparison with vehicle treatment (n=5/group). Two-dimensional electrophoresis (2-DE) analysis coupled to mass spectrometry revealed significant changes in the expression of nine proteins (CRMP2, PSMD1, STX18, KCNC3, CYP255, GABRP, GABT, PRPS1, CYP2B3), which were further analyzed by ontological classification (PANTHER). These results point to a number of novel potential chemical targets of FIN, and may help elucidate the underpinnings of FIN's behavioral effects and therapeutic potential for neuropsychiatric disorders.

Acute ethanol effects on neural encoding of reward size and delay in the nucleus accumbens

Acute ethanol administration can cause impulsivity, resulting in increased preference for immediately available rewards over delayed but more valuable alternatives. The manner in which reward size and delay are represented in neural firing is not fully understood, and very little is known about ethanol effects on this encoding. To address this issue, we used in vivo electrophysiology to characterize neural firing in the core of the nucleus accumbens (NAcc) in rats responding for rewards that varied in size or delay after vehicle or ethanol administration. The NAcc is a central element in the circuit that governs decision-making and importantly, promotes choice of delayed rewards. We found that NAcc firing in response to reward-predictive cues encoded anticipated reward value after vehicle administration, but ethanol administration disrupted this encoding, resulting in a loss of discrimination between immediate and delayed rewards in cue-evoked neural responses. In addition, NAcc firing occurring at the time of the operant response (lever pressing) was inversely correlated with behavioral response latency, such that increased firing rates were associated with decreased latencies to lever press. Ethanol administration selectively attenuated this lever press-evoked firing when delayed but not immediate rewards were expected. These effects on neural firing were accompanied by increased behavioral latencies to respond for delayed rewards. Our results suggest that ethanol effects on NAcc cue- and lever press-evoked encoding may contribute to ethanol-induced impulsivity.

Central Thalamic Deep-Brain Stimulation Alters Striatal-Thalamic Connectivity in Cognitive Neural Behavior

Central thalamic deep brain stimulation (CT-DBS) has been proposed as an experimental therapeutic approach to produce consistent sustained regulation of forebrain arousal for several neurological diseases. We investigated local field potentials (LFPs) induced by CT-DBS from the thalamic central lateral nuclei (CL) and the striatum as potential biomarkers for the enhancement of lever-pressing skill learning. LFPs were simultaneously recorded from multiple sites in the CL, ventral striatum (Vstr), and dorsal striatum (Dstr). LFP oscillation power and functional connectivity were assessed and compared between the CT-DBS and sham control groups. The theta and alpha LFP oscillations were significantly increased in the CL and striatum in the CT-DBS group. Furthermore, interhemispheric coherences between bilateral CL and striatum were increased in the theta band. Additionally, enhancement of c-Fos activity, dopamine D2 receptor (Drd2), and α4-nicotinic acetylcholine receptor (α4-nAChR) occurred after CT-DBS treatment in the striatum and hippocampus. CT-DBS strengthened thalamic-striatal functional connectivity, which demonstrates that the inter-regional connectivity enhancement might contribute to synaptic plasticity in the striatum. Altered dopaminergic and cholinergic receptors resulted in modulation of striatal synaptic plasticity's ability to regulate downstream signaling cascades for higher brain functions of lever-pressing skill learning.

A Framework for Understanding the Emerging Role of Corticolimbic-Ventral Striatal Networks in OCD-Associated Repetitive Behaviors

Significant interest in the mechanistic underpinnings of obsessive-compulsive disorder (OCD) has fueled research on the neural origins of compulsive behaviors. Converging clinical and preclinical evidence suggests that abnormal repetitive behaviors are driven by dysfunction in cortico-striatal-thalamic-cortical (CSTC) circuits. These findings suggest that compulsive behaviors arise, in part, from aberrant communication between lateral orbitofrontal cortex (OFC) and dorsal striatum. An important body of work focused on the role of this network in OCD has been instrumental to progress in the field. Disease models focused primarily on these regions, however, fail to capture an important aspect of the disorder: affective dysregulation. High levels of anxiety are extremely prevalent in OCD, as is comorbidity with major depressive disorder. Furthermore, deficits in processing rewards and abnormalities in processing emotional stimuli are suggestive of aberrant encoding of affective information. Accordingly, OCD can be partially characterized as a disease in which behavioral selection is corrupted by exaggerated or dysregulated emotional states. This suggests that the networks producing OCD symptoms likely expand beyond traditional lateral OFC and dorsal striatum circuit models, and highlights the need to cast a wider net in our investigation of the circuits involved in generating and sustaining OCD symptoms. Here, we address the emerging role of medial OFC, amygdala, and ventral tegmental area projections to the ventral striatum (VS) in OCD pathophysiology. The VS receives strong innervation from these affect and reward processing regions, and is therefore poised to integrate information crucial to the generation of compulsive behaviors. Though it complements functions of dorsal striatum and lateral OFC, this corticolimbic-VS network is less commonly explored as a potential source of the pathology underlying OCD. In this review, we discuss this network's potential role as a locus of OCD pathology and effective treatment.

Impulsivity in borderline personality disorder

BACKGROUND

Impulsivity is a core feature of borderline personality disorder (BPD) and is most frequently measured using self-rating scales. There is a need to find objective, valid and reliable measures of impulsivity. This study aimed to examine performance of participants with BPD compared with healthy controls on delay and probabilistic discounting tasks and the stop-signal task (SST), which are objective measures of choice and motor impulsivity, respectively.

METHOD

A total of 20 participants with BPD and 21 healthy control participants completed delay and probabilistic discounting tasks and the SST. They also completed the Barratt Impulsiveness Scale (BIS), a self-rating measure of impulsivity.

RESULTS

Participants with BPD showed significantly greater delay discounting than controls, manifest as a greater tendency to accept the immediately available lesser reward rather than waiting longer for a greater reward. Similarly they showed significantly greater discounting of rewards by the probability of payout, which correlated with past childhood trauma. Participants with BPD were found to choose the more certain and/or immediate rewards, irrespective of the value. On the SST the BPD and control groups did not differ significantly, demonstrating no difference in motor impulsivity. There was no significant difference between groups on self-reported impulsivity as measured by the BIS.

CONCLUSIONS

Measures of impulsivity show that while motor impulsivity was not significantly different in participants with BPD compared with controls, choice or reward-related impulsivity was significantly affected in those with BPD. This suggests that choice impulsivity but not motor impulsivity is a core feature of BPD.

Dopamine D1/D2 Receptor Activity in the Nucleus Accumbens Core But Not in the Nucleus Accumbens Shell and Orbitofrontal Cortex Modulates Risk-Based Decision Making

BACKGROUND

It is well known that brain dopamine (DA) signals support risk-based decision making; however, the specific terminal regions of midbrain DA neurons through which DA signals mediate risk-based decision making are unknown.

METHODS

Using microinfusions of the D1/D2 receptor antagonist flupenthixol, we sought to explore the role of D1/D2 receptor activity in the rat orbitofrontal cortex (OFC) and core and shell regions of the nucleus accumbens (AcbC and AcbS, respectively) in the regulation of risky choices. A risk-discounting task was used that involves choices between a certain small-reward lever that always delivered 1 pellet or a risky large-reward lever which delivered 4 pellets but had a decreasing probability of receiving the reward across 4 subsequent within-session trial blocks (100%, 50%, 25%, 12.5%). To validate task sensitivity to experimental manipulations of DA activity, we also examined the effects of systemic amphetamine and flupenthixol.

RESULTS

Systemic amphetamine increased while systemic flupenthixol reduced risky choices. Results further demonstrate that rats that received intra-AcbC flupenthixol were able to track increasing risk associated with the risky lever but displayed a generally reduced preference for the risky lever across all trial blocks, including in the initial trial block (large reward at 100%). Microinfusions of flupenthixol into the AcbS or OFC did not alter risk-based decision making.

CONCLUSIONS

Our data suggest that intra-AcbC D1/D2 receptor signaling does not support the ability to track shifts in reward probabilities but does bias risk-based decision making. That is, it increased the rats' preference for the response option known to be associated with higher risk-related costs.

Frontostriatal systems comprising connections between ventral medial prefrontal cortex and nucleus accumbens subregions differentially regulate motor impulse control in rats

RATIONALE

Deficits in impulse control are prevalent in several neuropsychiatric disorders that are based on impaired frontostriatal communication. The ventral medial prefrontal cortex (vmPFC) and the nucleus accumbens (NAc) are key substrates of impulse control in rats. The NAc core and shell are considered to be differentially involved suggesting a functional distinction between the connections of the vmPFC and particular NAc subregions concerning impulse control.

OBJECTIVES/METHODS

In the present study, simultaneous inactivation of the rats' vmPFC and NAc core or shell by contralateral microinfusion of the GABAA receptor agonist muscimol was used to investigate their relevance for impulse control in the five-choice serial reaction time task (5-CSRTT).

RESULTS

Disconnection of the vmPFC and NAc shell produced specific impairments in inhibitory control, indicated by significantly increased premature responding and an enhanced number of time-out responses, closely resembling the effects of bilateral inactivation of either the vmPFC or NAc shell previously reported using the same task. In contrast, disconnection of the vmPFC and NAc core only slightly increased the rate of omissions and latency of reward collection indicating attentional and motivational deficits.

CONCLUSIONS

Our results extend previous findings indicating the functional specialisation of frontostriatal networks and show a differential contribution of specific vmPFC-NAc connections to behavioural control depending on the NAc subregion. We conclude that the regulation of impulse control in rats requires an intact connection between the vmPFC and the NAc shell, while the vmPFC-NAc core projection seems to be of minor importance.

Nucleus accumbens core and shell inactivation differentially affects impulsive behaviours in rats

Impulsivity is a multifactorial phenomenon, determined by deficits in decision-making (impulsive choice) and impulse control (impulsive action). Recent findings indicate that impulsive behaviour is not only top-down controlled by cortical areas, but also modulated at subcortical level. The nucleus accumbens (NAc) might be a key substrate in cortico-limbic-striatal circuits involved in impulsive behaviour. Dissociable effects of the NAc subregions in various behavioural paradigms point to a potential functional distinction between NAc core and shell concerning different types of impulsivity. The present study used reversible inactivation of the rats' NAc core and shell via bilateral microinfusion of the GABAA receptor agonist muscimol (0.05μg/0.3μl) and fluorophore-conjugated muscimol (FCM, 0.27μg/0.3μl) in order to study their contribution to different aspects of impulse control in a 5-choice serial reaction time task (5-CSRTT) and impulsive choice in a delay-based decision-making T-maze task. Acute inactivation of NAc core as well as shell by muscimol increased impulsive choice, with higher impairments of the rats' waiting capacity in the T-maze following core injections compared to shell. Intra-NAc shell infusion of muscimol also induced specific impulse control deficits in the 5-CSRTT, while deactivation of the core caused severe general impairments in task performance. FCM did not affect animal behaviour. Our findings reveal clear involvement of NAc shell in both forms of impulsivity. Both subareas play a key role in the regulation of impulsive decision-making, but show functional dichotomy regarding impulse control with the core being more implicated in motivational and motor aspects.

A general role for medial prefrontal cortex in event prediction

A recent computational neural model of medial prefrontal cortex (mPFC), namely the predicted response-outcome (PRO) model (Alexander and Brown, 2011), suggests that mPFC learns to predict the outcomes of actions. The model accounted for a wide range of data on the mPFC. Nevertheless, numerous recent findings suggest that mPFC may signal predictions and prediction errors even when the predicted outcomes are not contingent on prior actions. Here we show that the existing PRO model can learn to predict outcomes in a general sense, and not only when the outcomes are contingent on actions. A series of simulations show how this generalized PRO model can account for an even broader range of findings in the mPFC, including human ERP, fMRI, and macaque single-unit data. The results suggest that the mPFC learns to predict salient events in general and provides a theoretical framework that links mPFC function to model-based reinforcement learning, Bayesian learning, and theories of cognitive control.

Neural and psychological mechanisms underlying compulsive drug seeking habits and drug memories--indications for novel treatments of addiction

This review discusses the evidence for the hypothesis that the development of drug addiction can be understood in terms of interactions between Pavlovian and instrumental learning and memory mechanisms in the brain that underlie the seeking and taking of drugs. It is argued that these behaviours initially are goal-directed, but increasingly become elicited as stimulus-response habits by drug-associated conditioned stimuli that are established by Pavlovian conditioning. It is further argued that compulsive drug use emerges as the result of a loss of prefrontal cortical inhibitory control over drug seeking habits. Data are reviewed that indicate these transitions from use to abuse to addiction depend upon shifts from ventral to dorsal striatal control over behaviour, mediated in part by serial connectivity between the striatum and midbrain dopamine systems. Only some individuals lose control over their drug use, and the importance of behavioural impulsivity as a vulnerability trait predicting stimulant abuse and addiction in animals and humans, together with consideration of an emerging neuroendophenotype for addiction are discussed. Finally, the potential for developing treatments for addiction is considered in light of the neuropsychological advances that are reviewed, including the possibility of targeting drug memory reconsolidation and extinction to reduce Pavlovian influences on drug seeking as a means of promoting abstinence and preventing relapse.

Lesions of the nucleus accumbens core modulate development of matching behavior

Background

The development of choice is a crucial determinant in the performance of appetitive responses. Given two options with different reinforcement rates, animals match their relative rate of responding to the relative rates of reinforcement (i.e., matching behavior). A previous study has shown that the nucleus accumbens core (AcbC) is involved in the performance of matching behavior in trained animals. However, the role of the AcbC in the acquisition of matching behavior has not been addressed.

Results

We conducted a series of experimental sessions to examine the role of the AcbC on the development of matching behavior. Instrumental responding was measured in rats with excitotoxic lesions of the AcbC. Rats were given two options that differed in the relative rate of reinforcement under concurrent variable-interval schedules. The locations of the more frequently reinforced option and the alternative option were randomly switched between sessions. Lesions of the AcbC accelerated the development of matching behavior compared to the sham-operated group. The AcbC-lesioned rats exhibited closer conformity to the matching law than shams when the options were in the same positions as in the previous session (the same condition), but not when the option locations had been switched (the different condition). The AcbC rats showed smaller probabilities of switching behavior between alternatives than shams. Post-reinforcement pausing was not affected by the AcbC lesion. Neither numbers of rewards obtained nor number of lever presses were different between the AcbC-lesioned rats and shams over session blocks.

Conclusions

Our results suggest that the AcbC plays a regulatory role in the development of matching behavior through switching probabilities rather than perception of reward magnitude. The differential effect of AcbC lesions on the matching behavior between the same and different conditions suggests influence of the spontaneous recovery, that is, reversion to a previously reinforced choice at the beginning of the next session, on the development of matching behavior in the AcbC-lesioned rats.

Behavioral flexibility is increased by optogenetic inhibition of neurons in the nucleus accumbens shell during specific time segments

Behavioral flexibility is vital for survival in an environment of changing contingencies. The nucleus accumbens may play an important role in behavioral flexibility, representing learned stimulus–reward associations in neural activity during response selection and learning from results. To investigate the role of nucleus accumbens neural activity in behavioral flexibility, we used light-activated halorhodopsin to inhibit nucleus accumbens shell neurons during specific time segments of a bar-pressing task requiring a win–stay/lose–shift strategy. We found that optogenetic inhibition during action selection in the time segment preceding a lever press had no effect on performance. However, inhibition occurring in the time segment during feedback of results—whether rewards or nonrewards—reduced the errors that occurred after a change in contingency. Our results demonstrate critical time segments during which nucleus accumbens shell neurons integrate feedback into subsequent responses. Inhibiting nucleus accumbens shell neurons in these time segments, during reinforced performance or after a change in contingencies, increases lose–shift behavior. We propose that the activity of nucleus shell accumbens shell neurons in these time segments plays a key role in integrating knowledge of results into subsequent behavior, as well as in modulating lose–shift behavior when contingencies change.

Cocaine self-administration abolishes associative neural encoding in the nucleus accumbens necessary for higher-order learning

BACKGROUND

Cocaine use is often associated with diminished cognitive function, persisting even after abstinence from the drug. Likely targets for these changes are the core and shell of the nucleus accumbens (NAc), which are critical for mediating the rewarding aspects of drugs of abuse as well as supporting associative learning. To understand this deficit, we recorded neural activity in the NAc of rats with a history of cocaine self-administration or control subjects while they learned Pavlovian first- and second-order associations.

METHODS

Rats were trained for 2 weeks to self-administer intravenous cocaine or water. Later, rats learned a first-order Pavlovian discrimination where a conditioned stimulus (CS)+ predicted food, and a control (CS-) did not. Rats then learned a second-order association where, absent any food reinforcement, a novel cued (SOC+) predicted the CS+ and another (SOC-) predicted the CS-. Electrophysiological recordings were taken during performance of these tasks in the NAc core and shell.

RESULTS

Both control subjects and cocaine-experienced rats learned the first-order association, but only control subjects learned the second-order association. Neural recordings indicated that core and shell neurons encoded task-relevant information that correlated with behavioral performance, whereas this type of encoding was abolished in cocaine-experienced rats.

CONCLUSIONS

The NAc core and shell perform complementary roles in supporting normal associative learning, functions that are impaired after cocaine experience. This impoverished encoding of motivational behavior, even after abstinence from the drug, might provide a key mechanism to understand why addiction remains a chronically relapsing disorder despite repeated attempts at sobriety.

Stress-induced anhedonia is associated with hypertrophy of medium spiny neurons of the nucleus accumbens

There is accumulating evidence that the nucleus accumbens (NAc) has an important role in the pathophysiology of depression. As the NAc is a key component in the neural circuitry of reward, it has been hypothesized that anhedonia, a core symptom of depression, might be related to dysfunction of this brain region. Neuronal morphology and expression of plasticity-related molecules were examined in the NAc of rats displaying anhedonic behavior (measured in the sucrose-consumption test) in response to chronic mild stress. To demonstrate the relevance of our measurements to depression, we tested whether the observed changes were sensitive to reversal with antidepressants (imipramine and fluoxetine). Data show that animals displaying anhedonic behavior display an hypertrophy of medium spiny neurons in the NAc and, in parallel, have increased expression of the genes encoding for brain-derived neurotrophic factor, neural cell adhesion molecule and synaptic protein synapsin 1. Importantly, the reversal of stress-induced anhedonia by antidepressants is linked to a restoration of gene-expression patterns and dendritic morphology in the NAc. Using an animal model of depression, we show that stress induces anhedonic behavior that is associated with specific changes in the neuronal morphology and in the gene-expression profile of the NAc that are effectively reversed after treatment with antidepressants.

Rapid dopamine dynamics in the accumbens core and shell: learning and action

The catecholamine dopamine (DA) has been implicated in a host of neural processes as diverse as schizophrenia, parkinsonism and reward encoding. Importantly, these distinct features of DA function are due in large part to separate neural circuits involving connections arising from different DA-releasing nuclei and projections to separate afferent targets. Emerging data has suggested that this same principle of separate neural circuits may be applicable within structural subregions, such as the core and shell of the nucleus accumbens (NAc). Further, DA may act selectively on smaller ensembles of cells (or, microcircuits) via differential DA receptor density and distinct inputs and outputs of the microcircuits, thus enabling new learning about Pavlovian cues, instrumental responses, subjective reward processing and decision-making. In this review, by taking advantage of studies using subsecond voltammetric techniques in behaving animals to study how rapid changes in DA levels affect behavior, we examine the spatial and temporal features of DA release and how it relates to both normal learning and similarities to pathological learning in the form of addiction.

Integrating behavioral economics and behavioral genetics: delayed reward discounting as an endophenotype for addictive disorders

Delayed reward discounting is a behavioral economic index of impulsivity, referring to how much an individual devalues a reward based on its delay in time. As a behavioral process that varies considerably across individuals, delay discounting has been studied extensively as a model for self-control, both in the general population and in clinical samples. There is growing interest in genetic influences on discounting and, in particular, the prospect of discounting as an endophenotype for addictive disorders (i.e., a heritable mechanism partially responsible for conferring genetic risk). This review assembles and critiques the evidence supporting this hypothesis. Via numerous cross-sectional studies and a small number of longitudinal studies, there is considerable evidence that impulsive discounting is associated with addictive behavior and appears to play an etiological role. Moreover, there is increasing evidence from diverse methodologies that impulsive delay discounting is temporally stable, heritable, and that elevated levels are present in nonaffected family members. These findings suggest that impulsive discounting meets the criteria for being considered an endophenotype. In addition, recent findings suggest that genetic variation related to dopamine neurotransmission is significantly associated with variability in discounting preferences. A significant caveat, however, is that the literature is modest in some domains and, in others, not all the findings have been supportive or consistent. In addition, important methodological considerations are necessary in future studies. Taken together, although not definitive, there is accumulating support for the hypothesis of impulsive discounting as an endophenotype for addictive behavior and a need for further systematic investigation.

Stimulant drugs trigger transient volumetric changes in the human ventral striatum

The ventral striatum (VStr) integrates mesolimbic dopaminergic and corticolimbic glutamatergic afferents and forms an essential component of the neural circuitry regulating impulsive behaviour. This structure represents a primary target of psychostimulant medication, the first-choice treatment for attention-deficit/hyperactivity disorder (ADHD), and is biochemically modified by these drugs in animals. However, the effects of stimulants on the human VStr remain to be determined. We acquired anatomical brain MRI scans from 23 never-medicated adult patients with ADHD, 31 adult patients with a history of stimulant treatment and 32 control subjects, and VStr volumes were determined using individual rater-blinded region of interest delineation on high-resolution neuroanatomical scans. Furthermore, we also extracted VStr volumes before and after methylphenidate treatment in a subsample of the medication-naïve adult patients as well as in 20 never-medicated children with ADHD. We observed smaller VStr volumes in adult patients with a history of stimulant treatment in comparison to never-medicated patients. Moreover, our longitudinal analyses uncovered a reduction of grey matter volume in the bilateral VStr in adult patients after exposure to methylphenidate, which was followed by volumetric recovery to control level. In children, the same pattern of VStr volume changes was observed after treatment with methylphenidate. These findings suggest that the altered VStr volumes previously observed in patients with ADHD may represent a transitory effect of stimulant exposure rather than an intrinsic feature of the disorder. More generally, these data show that stimulant drugs can render plastic volume changes in human VStr neuroanatomy.

Selective involvement by the medial orbitofrontal cortex in biasing risky, but not impulsive, choice

Separate regions of the orbitofrontal cortex (OFC) have been implicated in mediating different aspects of cost-benefit decision-making in humans and animals. Anatomical and functional imaging studies indicate that the medial (mOFC) and lateral OFC may subserve dissociable functions related to reward and decision-making processes, yet the majority of studies in rodents have focused on the lateral OFC. The present study investigated the contribution of the rat mOFC to risk and delay-based decision-making, assessed with probabilistic and delay-discounting tasks. In well-trained rats, reversible inactivation of the mOFC increase a risky choice on the probabilistic discounting task, irrespective of whether the odds of obtaining a larger/risky reward decreased (100-12.5%) or increased (12.5-100%) over the course of a session. The increase in risky choice was associated with enhanced win-stay behavior, wherein rats showed an increased tendency to choose the risky option after being rewarded for the risky choice on a preceding trial. In contrast, mOFC inactivation did not alter delay discounting. These findings suggest that the mOFC plays a selective role in decisions involving reward uncertainty, mitigating the impact that larger, probabilistic rewards exert on subsequent choice behavior. This function may promote the exploration of novel options when reward contingencies change.

Linking ADHD, impulsivity, and drug abuse: a neuropsychological perspective

In this chapter, we consider the relevance of impulsivity as both a psychological construct and endophenotype underlying attention-deficit/hyperactivity disorder (ADHD) and drug addiction. The case for executive dysfunction in ADHD and drug addiction is critically reviewed in the context of dissociable cognitive control processes mediated by the dorsolateral prefrontal cortex (DLPFC), the orbital and ventral medial prefrontal cortex (VMPFC). We argue that such neuroanatomical divisions within the prefrontal cortex are likely to account for the multidimensional basis of impulsivity conceptually categorized in terms of "motoric" and "choice" impulsivity. The relevance of this distinction for the etiology of ADHD and drug addiction is integrated within a novel theoretical framework. This scheme embraces animal learning theory to help explain the heterogeneity of impulse control disorders, which are exemplified by ADHD as a vulnerability disorder for drug addiction.

Nucleus accumbens and delay discounting in rats: evidence from a new quantitative protocol for analysing inter-temporal choice

RATIONALE

There is evidence that the core of the nucleus accumbens (AcbC) is involved in inter-temporal choice behaviour.

OBJECTIVE

A new behavioural protocol was used to examine the effect of destruction of the AcbC on delay discounting in inter-temporal choice schedules in rats.

METHOD

Rats with excitotoxic lesions of the AcbC or sham lesions made repeated choices on an adjusting-delay schedule between a smaller reinforcer (A) that was delivered immediately and a larger reinforcer (B) that was delivered after a delay which increased or decreased depending on the subject's choices. In two phases of the experiment, reinforcer sizes were selected which enabled theoretical parameters expressing delay discounting and sensitivity to reinforcer size to be estimated from the ratio of the indifference delays (i.e. the quasi-stable values of the adjusting delay seen after extended training) obtained in the two phases.

RESULTS

In both groups, indifference delays were shorter when the sizes of A and B were 14 and 25 μl than when they were 25 and 100 μl of a 0.6 M sucrose solution. Indifference delays were shorter in AcbC-lesioned than in sham-lesioned rats. Estimates of delay discounting rate based on the ratio of the indifference delays were lower in the AcbC-lesioned than in the sham-lesioned rats. The size sensitivity parameter did not differ between the groups. Adjusting delays in successive blocks of trials were analysed using Fourier transform. The period corresponding to the dominant frequency of the power spectrum and power within the dominant frequency band did not differ between the groups.

CONCLUSIONS

Destruction of the AcbC increased the rate of delay discounting.

Dorsal and ventral striatal protein synthesis inhibition affect reinforcer valuation but not the consolidation of instrumental learning

The evidence for a role of the striatum in the acquisition of uncued instrumental responding is ambiguous. It has been shown that post-session infusions of anisomycin into the core of the nucleus accumbens (NAcc) impaired instrumental acquisition, but pre-training lesions of the NAcc suggest that it is not necessary. Recently, we demonstrated that the infusion of anisomycin into the anterior cingulate cortex impaired instrumental acquisition indirectly through a taste aversion. Thus, we hypothesized that post-session anisomycin infusions into the NAcc affected instrumental acquisition through an effect on reinforcer valuation. For the dorsal striatum, both post-session infusions of anisomycin and pre-training lesion studies suggest that neither the dorsolateral nor the dorsomedial striatum is necessary for the acquisition of instrumental responding. However, it has not been attempted to block plasticity in both regions concurrently, and we hypothesized that both regions independently contribute to acquisition through goal-directed and habitual learning. In the current experiments, we first replicated the effect of unprotected post-session anisomycin infusions into the NAcc on instrumental acquisition. Subsequently, we investigated the effect of protein synthesis inhibition in the NAcc and dorsomedial and dorsolateral striatum concurrently on instrumental acquisition, critically controlling for effects on reinforcer valuation. The anisomycin infusions induced an aversive state, but did not affect instrumental acquisition.

Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision making

The nucleus accumbens (NAc) has been implicated in mediating different forms of decision making in humans and animals. In the present study, we observed that inactivation of the rat NAc, via infusion of GABA agonists, reduced preference for a large/risky option and increased response latencies on a probabilistic discounting task. Discrete inactivations of the NAc shell and core revealed further differences between these regions in mediating choice and response latencies, respectively. The effect on choice was attributable to reduced win-stay performance (i.e., choosing risky after a being rewarded for a risky choice on a preceding trial). Moreover, NAc inactivation altered choice only when the large/risky option had greater long-term value, in terms of the amount of food that could be obtained over multiple trials relative to the small/certain option. Inactivation of the NAc or the shell subregion also slightly reduced preference for larger rewards on a reward magnitude discrimination. Thus, the NAc seems to play a small role in biasing choice toward larger rewards, but its contribution to behavior is amplified when delivery of these rewards is uncertain, helping to direct response selection toward more favorable outcomes.

Multiple representations and algorithms for reinforcement learning in the cortico-basal ganglia circuit

Accumulating evidence shows that the neural network of the cerebral cortex and the basal ganglia is critically involved in reinforcement learning. Recent studies found functional heterogeneity within the cortico-basal ganglia circuit, especially in its ventromedial to dorsolateral axis. Here we review computational issues in reinforcement learning and propose a working hypothesis on how multiple reinforcement learning algorithms are implemented in the cortico-basal ganglia circuit using different representations of states, values, and actions.

Bilateral striatal lesions disrupt performance in an operant delayed reinforcement task in rats

In order to provide an animal model of the impulsivity observed in Huntington's disease, the effects of bilateral neostriatal lesions in rats were evaluated in an operant delayed reinforcement task. When given a choice between responding to one lever for a small but immediate reward and a second lever for a larger delayed reward, normal rats exhibit a marked preference for responding to the high reward lever when the imposed delay is short, but progressively choose the lever associated with immediate small reward as the delays increase. Following striatal lesions, the animals continue to express similar preferences, but the lesions initially impose a distinct flattening of the delay-preference function, suggesting a relative insensitivity to the increasing delay parameter in making their response choices. However, this deficit declines with extend retraining on the task, such that 1-2 months after lesion the delay-dependent shift of preference from the delayed to the immediate lever as the delays lengthened was comparable in lesion and sham animals. Amphetamine further disinhibited all animals, apparent as a further increase in the number and reduction of the latencies of responses made to the lever associated with immediate reward. Striatal lesions had little influence on the effects of amphetamine on task performance, other than the increase in the numbers of omissions of lever and panel responses induced by the drug was more marked in the lesion than sham animals, and the lesioned animals exhibited less delay-dependency than the controls in their preference for responding to the lever associated with the larger delayed reinforcement at the highest (1.5 mg/kg) dose tested. The present results indicate small but clear effects of dorsal striatal lesions in an operant delayed reinforcement task, suggestive of an initial impairment in response selection and a reduction in their sensitivity to the delay interval itself. This deficit recovered with further training, which may be dependent upon relearning choice response procedures disrupted by the lesion, but might be reinstated by treatment with stimulant drugs. This article is part of a special issue entitled 'Behavioural, Anatomical, and Genetic Characterisation of Mouse and Rat Models of Huntington's Disease.'

The structural basis of inter-individual differences in human behaviour and cognition

Inter-individual variability in perception, thought and action is frequently treated as a source of 'noise' in scientific investigations of the neural mechanisms that underlie these processes, and discarded by averaging data from a group of participants. However, recent MRI studies in the human brain show that inter-individual variability in a wide range of basic and higher cognitive functions - including perception, motor control, memory, aspects of consciousness and the ability to introspect - can be predicted from the local structure of grey and white matter as assessed by voxel-based morphometry or diffusion tensor imaging. We propose that inter-individual differences can be used as a source of information to link human behaviour and cognition to brain anatomy.

Social withdrawal and gambling-like profile after lentiviral manipulation of DAT expression in the rat accumbens

Dysfunction of brain dopamine transporter (DAT) has been associated with sensation seeking and impulse-control disorders. We recently generated a new animal model by stereotaxical inoculation of lentiviral vectors, which allowed localized intra-accumbal delivery of modulators for DAT gene: GFP (green fluorescent protein) control, silencers (Sil), a regulatable enhancer (DAT+), or both (DAT+Sil). Wistar male rats were followed both for socio-emotional profiles and for propensity to seek risky, uncertain rewards. Elevated anxiety and affiliation towards an unfamiliar partner emerged in Sil rats. Interestingly, in DAT+Sil rats (and Sil rats to a lesser extent) levels of playful social interaction were markedly reduced compared to controls. These DAT+Sil rats displayed a marked 'gambling-like' profile (i.e. preference for a large/uncertain over a small/sure reward), which disappeared upon doxycycline-induced switch-off onto DAT enhancer, but consistently reappeared with doxycycline removal. MRI-guided 1H-MRS (at 4.7 T) examinations in vivo (under anaesthesia) revealed changes in the bioenergetic metabolites (phosphocreatine and total creatine) for DAT+Sil rats, indicating a functional up-regulation of dorsal striatum (Str) and conversely a down-regulation of ventral striatum (i.e. nucleus accumbens, NAc). A combined profile of (1) enhanced proneness to gambling and (2) strong social withdrawal is thus associated with altered DAT-induced balance within forebrain dopamine systems. In fact, risk of developing a gambling-prone, social-avoidant psychopathology might be associated with (1) dominant semi-automatic strategies and/or habits, developed within Str circuits, and (2) reduced NAc function, with poorer feedback adjustment on decisions by aversive experiences.

Differential effects of dopaminergic manipulations on risky choice

RATIONALE

Evaluation of risks and rewards associated with different options is facilitated by components of the mesocorticolimbic dopamine (DA) system. Augmenting or reducing DA activity increases or decreases preference for larger, uncertain rewards when reward probabilities decrease within a session. However, manipulations of DA activity may differentially alter risky choice when shifts in the relative value of probabilistic rewards are greater or lesser than those experienced previously.

OBJECTIVES

We investigated the effects of amphetamine and the DA antagonist flupenthixol on risk discounting, whereby we altered the manner in which reward probabilities changed.

METHODS

Rats chose between a "Small/Certain" (one pellet) and a "Large/Risky" lever that delivered four pellets in a probabilistic manner that changed during a session. Separate groups of rats were trained with a descending (100%, 50%, 25%, 12.5%), ascending (12.5-100%) or mixed (100%, 12.5%, 25%, 50%) order of probabilities associated with the large/risky option.

RESULTS

Flupenthixol consistently decreased preference for the large/risky option. In contrast, amphetamine increased preference for the large/risky lever when the probabilities decreased over a session, but reduced preference in the ascending condition.

CONCLUSIONS

Reductions in normal DA tone consistently biases choice away larger, probabilistic rewards. In contrast, increases in DA release may disrupt adjustments in behavior in response to changes in the relative value of certain versus uncertain rewards. These findings further clarify the role of DA in mediating risk/reward judgments and how perturbations in DA signaling may interfere with the ability to adjust decision making in response to changes in reward contingencies.

Neurochemistry of the nucleus accumbens and its relevance to depression and antidepressant action in rodents

There is accumulating evidence that the nucleus accumbens (NAc) plays an important role in the pathophysiology of depression. Given that clinical depression is marked by anhedonia (diminished interest or pleasure), dysfunction of the brain reward pathway has been suggested as contributing to the pathophysiology of depression.Since the NAc is the center of reward and learning, it is hypothesized that anhedonia might be produced by hampering the function of the NAc. Indeed, it has been reported that stress, drug exposure and drug withdrawal, all of which produce a depressive-phenotype, alter various functions within the NAc, leading to inhibited dopaminergic activity in the NAc.In this review, we describe various factors as possible candidates within the NAc for the initiation of depressive symptoms. First, we discuss the roles of several neurotransmitters and neuropeptides in the functioning of the NAc, including dopamine, glutamate, gamma-aminobutyric acid (GABA), acetylcholine, serotonin, dynorphin, enkephaline, brain-derived neurotrophic factor (BDNF), cAMP response element-binding protein (CREB), melanin-concentrating hormone (MCH) and cocaine- and amphetamine-regulated transcript (CART). Second, based on previous studies, we propose hypothetical relationships among these substances and the shell and core subregions of the NAc.

Prefrontal cortical contribution to risk-based decision making

Damage to various regions of the prefrontal cortex (PFC) impairs decision making involving evaluations about risks and rewards. However, the specific contributions that different PFC subregions make to risk-based decision making are unclear. We investigated the effects of reversible inactivation of 4 subregions of the rat PFC (prelimbic medial PFC, orbitofrontal cortex [OFC], anterior cingulate, and insular cortex) on probabilistic (or risk) discounting. Rats were well trained to choose between either a "Small/Certain" lever that always delivered 1 food pellet, or another, "Large/Risky" lever, which delivered 4 pellets, but the probability of receiving reward decreased across 4 trial blocks (100%, 50%, 25%, and 12.5%). Infusions of gama-aminobutyric acid agonists muscimol/baclofen into the medial PFC increased risky choice. However, similar medial PFC inactivations decreased risky choice when the Large/Risky reward probability increased over a session. OFC inactivation increased response latencies in the latter trial blocks without affecting choice. Anterior cingulate or insular inactivations were without effect. The effects of prelimbic inactivations were not attributable to disruptions in response flexibility or judgments about the relative value of probabilistic rewards. Thus, the prelimbic, but not other PFC regions, plays a critical role in risk discounting, integrating information about changing reward probabilities to update value representations that facilitate efficient decision making.

Validation of decision-making models and analysis of decision variables in the rat basal ganglia

Reinforcement learning theory plays a key role in understanding the behavioral and neural mechanisms of choice behavior in animals and humans. Especially, intermediate variables of learning models estimated from behavioral data, such as the expectation of reward for each candidate choice (action value), have been used in searches for the neural correlates of computational elements in learning and decision making. The aims of the present study are as follows: (1) to test which computational model best captures the choice learning process in animals and (2) to elucidate how action values are represented in different parts of the corticobasal ganglia circuit. We compared different behavioral learning algorithms to predict the choice sequences generated by rats during a free-choice task and analyzed associated neural activity in the nucleus accumbens (NAc) and ventral pallidum (VP). The major findings of this study were as follows: (1) modified versions of an action-value learning model captured a variety of choice strategies of rats, including win-stay-lose-switch and persevering behavior, and predicted rats' choice sequences better than the best multistep Markov model; and (2) information about action values and future actions was coded in both the NAc and VP, but was less dominant than information about trial types, selected actions, and reward outcome. The results of our model-based analysis suggest that the primary role of the NAc and VP is to monitor information important for updating choice behaviors. Information represented in the NAc and VP might contribute to a choice mechanism that is situated elsewhere.

Short-term temporal discounting of reward value in human ventral striatum

Delayed rewards lose their value for economic decisions and constitute weaker reinforcers for learning. Temporal discounting of reward value already occurs within a few seconds in animals, which allows investigations of the underlying neurophysiological mechanisms. However, it is difficult to relate these mechanisms to human discounting behavior, which is usually studied over days and months and may engage different brain processes. Our study aimed to bridge the gap by using very short delays and measuring human functional magnetic resonance responses in one of the key reward centers of the brain, the ventral striatum. We used psychometric methods to assess subjective timing and valuation of monetary rewards with delays of 4.0-13.5 s. We demonstrated hyperbolic and exponential decreases of striatal responses to reward predicting stimuli within this time range, irrespective of changes in reward rate. Lower reward magnitudes induced steeper behavioral and striatal discounting. By contrast, striatal responses following the delivery of reward reflected the uncertainty in subjective timing associated with delayed rewards rather than value discounting. These data suggest that delays of a few seconds affect the neural processing of predicted reward value in the ventral striatum and engage the temporal sensitivity of reward responses. Comparisons with electrophysiological animal data suggest that ventral striatal reward discounting may involve dopaminergic and orbitofrontal inputs.