Impact of size and delay on neural activity in the rat limbic corticostriatal system

Cortico-striatal beta oscillations as a reward-related signal

The value associated with reward is sensitive to external factors, such as the time between the choice and reward delivery as classically manipulated in temporal discounting tasks. Subjective preference for two reward options is dependent on objective variables of reward magnitude and reward delay. Single neuron correlates of reward value have been observed in regions, including ventral striatum, orbital, and medial prefrontal cortex. Brain imaging studies show cortico-striatal-limbic network activity related to subjective preferences. To explore how oscillatory dynamics represent reward processing across brain regions, we measured local field potentials of rats performing a temporal discounting task. Our goal was to use a data-driven approach to identify an electrophysiological marker that correlates with reward preference. We found that reward-locked oscillations at beta frequencies signaled the magnitude of reward and decayed with longer temporal delays. Electrodes in orbitofrontal/medial prefrontal cortex, anterior insula, ventral striatum, and amygdala individually increased power and were functionally connected at beta frequencies during reward outcome. Beta power during reward outcome correlated with subjective value as defined by a computational model fit to the discounting behavior. These data suggest that cortico-striatal beta oscillations are a reward signal correlated, which may represent subjective value and hold potential to serve as a biomarker and potential therapeutic target.

Anterior cingulate cortex lesions impair multiple facets of task engagement not mediated by dorsomedial striatum neuron firing

The anterior cingulate cortex (ACC) has been implicated across multiple highly specialized cognitive functions-including task engagement, motivation, error detection, attention allocation, value processing, and action selection. Here, we ask if ACC lesions disrupt task performance and firing in dorsomedial striatum (DMS) during the performance of a reward-guided decision-making task that engages many of these cognitive functions. We found that ACC lesions impacted several facets of task performance-including decreasing the initiation and completion of trials, slowing reaction times, and resulting in suboptimal and inaccurate action selection. Reductions in movement times towards the end of behavioral sessions further suggested attenuations in motivation, which paralleled reductions in directional action selection signals in the DMS that were observed later in recording sessions. Surprisingly, however, beyond altered action signals late in sessions-neural correlates in the DMS were largely unaffected, even though behavior was disrupted at multiple levels. We conclude that ACC lesions result in overall deficits in task engagement that impact multiple facets of task performance during our reward-guided decision-making task, which-beyond impacting motivated action signals-arise from dysregulated attentional signals in the ACC and are mediated via downstream targets other than DMS.

Methylphenidate, but not citalopram, decreases impulsive choice in rats performing a temporal discounting task

Introduction

Drugs targeting monoamine systems remain the most common treatment for disorders with impulse control impairments. There is a body of literature suggesting that drugs affecting serotonin reuptake and dopamine reuptake can modulate distinct aspects of impulsivity - though such tests are often performed using distinct behavioral tasks prohibiting easy comparisons.

Methods

Here, we directly compare pharmacologic agents that affect dopamine (methylphenidate) vs serotonin (citalopram) manipulations on choice impulsivity in a temporal discounting task where rats could choose between a small, immediate reward or a large reward delayed at either 2 or 10s. In control conditions, rats preferred the large reward at a small (2s) delay and discounted the large reward at a long (10s) delay.

Results

Methylphenidate, a dopamine transport inhibitor that blocks reuptake of dopamine, dose-dependently increased large reward preference in the long delay (10s) block. Citalopram, a selective serotonin reuptake inhibitor, had no effect on temporal discounting behavior. Impulsive behavior on the temporal discounting task was at least partially mediated by the nucleus accumbens shell. Bilateral lesions to the nucleus accumbens shell reduced choice impulsivity during the long delay (10s) block. Following lesions, methylphenidate did not impact impulsivity.

Discussion

Our results suggest that striatal dopaminergic systems modulate choice impulsivity via actions within the nucleus accumbens shell, whereas serotonin systems may regulate different aspects of behavioral inhibition/impulsivity.

Dopamine D2 receptors in nucleus accumbens cholinergic interneurons increase impulsive choice

Impulsive choice, often characterized by excessive preference for small, short-term rewards over larger, long-term rewards, is a prominent feature of substance use and other neuropsychiatric disorders. The neural mechanisms underlying impulsive choice are not well understood, but growing evidence implicates nucleus accumbens (NAc) dopamine and its actions on dopamine D2 receptors (D2Rs). Because several NAc cell types and afferents express D2Rs, it has been difficult to determine the specific neural mechanisms linking NAc D2Rs to impulsive choice. Of these cell types, cholinergic interneurons (CINs) of the NAc, which express D2Rs, have emerged as key regulators of striatal output and local dopamine release. Despite these relevant functions, whether D2Rs expressed specifically in these neurons contribute to impulsive choice behavior is unknown. Here, we show that D2R upregulation in CINs of the mouse NAc increases impulsive choice as measured in a delay discounting task without affecting reward magnitude sensitivity or interval timing. Conversely, mice lacking D2Rs in CINs showed decreased delay discounting. Furthermore, CIN D2R manipulations did not affect probabilistic discounting, which measures a different form of impulsive choice. Together, these findings suggest that CIN D2Rs regulate impulsive decision-making involving delay costs, providing new insight into the mechanisms by which NAc dopamine influences impulsive behavior.

Neural dynamics underlying self-control in the primate subthalamic nucleus

The subthalamic nucleus (STN) is hypothesized to play a central role in neural processes that regulate self-control. Still uncertain, however, is how that brain structure participates in the dynamically evolving estimation of value that underlies the ability to delay gratification and wait patiently for a gain. To address that gap in knowledge, we studied the spiking activity of neurons in the STN of monkeys during a task in which animals were required to remain motionless for varying periods of time in order to obtain food reward. At the single-neuron and population levels, we found a cost-benefit integration between the desirability of the expected reward and the imposed delay to reward delivery, with STN signals that dynamically combined both attributes of the reward to form a single integrated estimate of value. This neural encoding of subjective value evolved dynamically across the waiting period that intervened after instruction cue. Moreover, this encoding was distributed inhomogeneously along the antero-posterior axis of the STN such that the most dorso-posterior-placed neurons represented the temporal discounted value most strongly. These findings highlight the selective involvement of the dorso-posterior STN in the representation of temporally discounted rewards. The combination of rewards and time delays into an integrated representation is essential for self-control, the promotion of goal pursuit, and the willingness to bear the costs of time delays.

Dopamine D2 receptors in nucleus accumbens cholinergic interneurons increase impulsive choice

Impulsive choice, often characterized by excessive preference for small, short-term rewards over larger, long-term rewards, is a prominent feature of substance use and other neuropsychiatric disorders. The neural mechanisms underlying impulsive choice are not well understood, but growing evidence implicates nucleus accumbens (NAc) dopamine and its actions on dopamine D2 receptors (D2Rs). Because several NAc cell types and afferents express D2Rs, it has been difficult to determine the specific neural mechanisms linking NAc D2Rs to impulsive choice. Of these cell types, cholinergic interneurons (CINs) of the NAc, which express D2Rs, have emerged as key regulators of striatal output and local dopamine release. Despite these relevant functions, whether D2Rs expressed specifically in these neurons contribute to impulsive choice behavior is unknown. Here, we show that D2R upregulation in CINs of the mouse NAc increases impulsive choice as measured in a delay discounting task without affecting reward magnitude sensitivity or interval timing. Conversely, mice lacking D2Rs in CINs showed decreased delay discounting. Furthermore, CIN D2R manipulations did not affect probabilistic discounting, which measures a different form of impulsive choice. Together, these findings suggest that CIN D2Rs regulate impulsive decision-making involving delay costs, providing new insight into the mechanisms by which NAc dopamine influences impulsive behavior.

Prior Cocaine Exposure Increases Firing to Immediate Reward While Attenuating Cue and Context Signals Related to Reward Value in the Insula

The insula contributes to behavioral control and is disrupted by substance abuse, yet we know little about the neural signals underlying these functions or how they are disrupted after chronic drug self-administration. Here, male and female rats self-administered either cocaine (experimental group) or sucrose (control) for 12 consecutive days. After a 1 month withdrawal period, we recorded from insula while rats performed a previously learned reward-guided decision-making task. Cocaine-exposed rats were more sensitive to value manipulations and were faster to respond. These behavioral changes were accompanied by elevated counts of neurons in the insula that increased firing to reward. These neurons also fired more strongly at the start of long-delay trials, when a more immediate reward would be expected, and fired less strongly in anticipation of the actual delivery of delayed rewards. Although reward-related firing to immediate reward was enhanced after cocaine self-administration, reward-predicting cue and context signals were attenuated. In addition to revealing novel firing patterns unique to insula, our data suggest changes in such neural activity likely contribute to impaired decision making observed after drug use.SIGNIFICANCE STATEMENT The insula plays a clear role in drug addiction and drug-induced impairments of decision making, yet there is little understanding of its underlying neural signals. We found that chronic cocaine self-administration reduces cue and context encoding in insula while enhancing signals related to immediate reward. These changes in neural activity likely contribute to impaired decision making and impulsivity observed after drug use.

Sex Differences and Effects of Predictive Cues on Delayed Punishment Discounting

The majority of the research studying punishment has focused on an aversive stimulus delivered immediately after an action. However, in real-world decision-making, negative consequences often occur long after a decision has been made. This can engender myopic decisions that fail to appropriately respond to consequences. Whereas discounting of delayed rewards has been well studied in both human and animal models, systematic discounting of delayed consequences remains largely unexplored. To address this gap in the literature, we developed the delayed punishment decision-making task. Rats chose between a small, single-pellet reinforcer and a large, three-pellet reinforcer accompanied by a mild foot shock. The shock was preceded by a delay, which systematically increased throughout the session (0, 4, 8, 12, 16 s). On average, rats discounted the negative value of delayed punishment, as indicated by increased choice of the large, punished reward as the delay preceding the shock lengthened. Female rats discounted delayed punishment less than males, and this behavior was not influenced by estrous cycling. The addition of a cue light significantly decreased the undervaluation of delayed consequences for both sexes. Finally, there was no correlation between the discounting of delayed punishments and a traditional reward delay discounting task for either sex. These data indicate that the ability of punishment to regulate decision-making is attenuated when punishment occurs later in time. This task provides an avenue for exploration of the neural circuitry underlying the devaluation of delayed punishment and may assist in developing treatments for substance use disorders.

Prelimbic Cortical Neurons Track Preferred Reward Value and Reflect Impulsive Choice during Delay Discounting Behavior

In delay discounting, individuals discount the value of a reward based on the delay to its receipt. The prelimbic cortex (PrL) is heavily interconnected with several brain regions implicated in delay discounting, but the specific contributions of the PrL to delay discounting are unknown. Here, we used multineuron electrophysiological recording methods in Long-Evans male (n = 10) and female (n = 9) rats to characterize the firing dynamics of PrL neurons during discrete cue and lever press events in a delay discounting task. Rats' initial preference for the large reward decreased as delays for that outcome increased across blocks, reflecting classic discounting behavior. Electrophysiological recordings revealed that subgroups of neurons exhibited phasic responses to cue presentations and lever presses. These phasic neurons were found to respond to either large/delay, small/immediate, or both trial types and the percentage of these neurons shifted across blocks as the expected value of the reward changed. Critically, this shift was only seen during trials in which animals could choose their preferred option (free choice trials) and not during trials where animals could choose only one option (forced choice trials). Further, this shift was dependent on rats' inherent impulsivity because high impulsive rats demonstrated a greater percentage of small/immediate-responsive neurons as the task progressed. Collectively, these findings suggest a unique role for the PrL in encoding reward value during delay discounting that is influenced by individual differences in impulsivity.SIGNIFICANCE STATEMENT In delay discounting, individuals discount the value of a reward based on the delay to its receipt. Here, we used electrophysiology to investigate the role of the prelimbic cortex (PrL) in this process. We found that subsets of neurons shifted activity as a function of the changing expected delay and reward magnitude, but this shift was only evident during trials in which animals could choose their preferred option. Further, this dynamic neural activity depended on rats' inherent impulsivity, with impulsive rats exhibiting a stronger neural shift toward the immediate reward as the task progressed. These findings suggest a role for the PrL in encoding reward value during delay discounting that is influenced by goal-directed context and individual differences in impulsivity.

Previous cocaine self-administration disrupts reward expectancy encoding in ventral striatum

The nucleus accumbens core (NAc) is important for integrating and providing information to downstream areas about the timing and value of anticipated reward. Although NAc is one of the first brain regions to be affected by drugs of abuse, we still do not know how neural correlates related to reward expectancy are affected by previous cocaine self-administration. To address this issue, we recorded from single neurons in the NAc of rats that had previously self-administered cocaine or sucrose (control). Neural recordings were then taken while rats performed an odor-guided decision-making task in which we independently manipulated value of expected reward by changing the delay to or size of reward across a series of trial blocks. We found that previous cocaine self-administration made rats more impulsive, biasing choice behavior toward more immediate reward. Further, compared to controls, cocaine-exposed rats showed significantly fewer neurons in the NAc that were responsive during odor cues and reward delivery, and in the reward-responsive neurons that remained, diminished directional and value encoding was observed. Lastly, we found that after cocaine exposure, reward-related firing during longer delays was reduced compared to controls. These results demonstrate that prior cocaine self-administration alters reward-expectancy encoding in NAc, which could contribute to poor decision making observed after chronic cocaine use.

The impact of drugs of abuse on executive function: characterizing long-term changes in neural correlates following chronic drug exposure and withdrawal in rats

Addiction has long been characterized by diminished executive function, control, and impulsivity management. In particular, these deficits often manifest themselves as impairments in reversal learning, delay discounting, and response inhibition. Understanding the neurobiological substrates of these behavioral deficits is of paramount importance to our understanding of addiction. Within the cycle of addiction, periods during and after withdrawal represent a particularly difficult point of intervention in that the negative physical symptoms associated with drug removal and drug craving increase the likelihood that the patient will relapse and return to drug use in order to abate these symptoms. Moreover, it is often during this time that drug induced deficits in executive function hinder the ability of the patient to refrain from drug use. Thus, it is necessary to understand the physiological and behavioral changes associated with withdrawal and drug craving-largely manifesting as deficits in executive control-to develop more effective treatment strategies. In this review, we address the long-term impact that drugs of abuse have on the behavioral and neural correlates that give rise to executive control as measured by reversal learning, delay discounting, and stop-signal tasks, focusing particularly on our work using rats as a model system.

To Act or Not to Act: Endocannabinoid/Dopamine Interactions in Decision-Making

Decision-making is an ethologically adaptive construct that is impaired in multiple psychiatric disorders. Activity within the mesocorticolimbic dopamine system has been traditionally associated with decision-making. The endocannabinoid system through its actions on inhibitory and excitatory synapses modulates dopamine activity and decision-making. The aim of this brief review is to present a synopsis of available data obtained when the endocannabinoid system is manipulated and dopamine activity recorded. To this end, we review research using different behavioral paradigms to provide further insight into how this ubiquitous signaling system biases dopamine-related behaviors to regulate decision-making.

Neural correlates of rules and conflict in medial prefrontal cortex during decision and feedback epochs

The ability to properly adjust behavioral responses to cues in a changing environment is crucial for survival. Activity in the medial Prefrontal Cortex (mPFC) is thought to both represent rules to guide behavior as well as detect and resolve conflicts between rules in changing contingencies. However, while lesion and pharmacological studies have supported a crucial role for mPFC in this type of set-shifting, an understanding of how mPFC represents current rules or detects and resolves conflict between different rules is unclear. Here, we directly address the role of rat mPFC in shifting rule based behavioral strategies using a novel behavioral task designed to tease apart neural signatures of rules, conflict and direction. We demonstrate that activity of single neurons in rat mPFC represent distinct rules. Further, we show increased firing on high conflict trials in a separate population of mPFC neurons. Reduced firing in both populations of neurons was associated with poor performance. Moreover, activity in both populations increased and decreased firing during the outcome epoch when reward was and was not delivered on correct and incorrect trials, respectively. In addition, outcome firing was modulated by the current rule and the degree of conflict associated with the previous decision. These results promote a greater understanding of the role that mPFC plays in switching between rules, signaling both rule and conflict to promote improved behavioral performance.

From ventral-medial to dorsal-lateral striatum: neural correlates of reward-guided decision-making

The striatum is critical for reward-guided and habitual behavior. Anatomical and interference studies suggest a functional heterogeneity within striatum. Medial regions, such as nucleus accumbens core and dorsal medial striatum play roles in goal-directed behavior, while dorsal lateral striatum is critical for control of habitual action. Subdivisions of striatum are topographically connected with different cortical and subcortical structures forming channels that carry information related to limbic, associative, and sensorimotor functions. Here, we describe data showing that as one progresses from ventral-medial to dorsal-lateral striatum, there is a shift from more prominent value encoding to activity more closely related to associative and motor aspects of decision-making. In addition, we will describe data suggesting that striatal circuits work in parallel to control behavior and that regions within striatum can compensate for each other when functions are disrupted.

Neurophysiology of Reward-Guided Behavior: Correlates Related to Predictions, Value, Motivation, Errors, Attention, and Action

Many brain areas are activated by the possibility and receipt of reward. Are all of these brain areas reporting the same information about reward? Or are these signals related to other functions that accompany reward-guided learning and decision-making? Through carefully controlled behavioral studies, it has been shown that reward-related activity can represent reward expectations related to future outcomes, errors in those expectations, motivation, and signals related to goal- and habit-driven behaviors. These dissociations have been accomplished by manipulating the predictability of positively and negatively valued events. Here, we review single neuron recordings in behaving animals that have addressed this issue. We describe data showing that several brain areas, including orbitofrontal cortex, anterior cingulate, and basolateral amygdala signal reward prediction. In addition, anterior cingulate, basolateral amygdala, and dopamine neurons also signal errors in reward prediction, but in different ways. For these areas, we will describe how unexpected manipulations of positive and negative value can dissociate signed from unsigned reward prediction errors. All of these signals feed into striatum to modify signals that motivate behavior in ventral striatum and guide responding via associative encoding in dorsolateral striatum.

Partial inactivation of nucleus accumbens core decreases delay discounting in rats without affecting sensitivity to delay or magnitude

Increased preference for smaller, sooner rewards (delay discounting) is associated with several behavioral disorders, including ADHD and substance use disorders. However, delay discounting is a complex cognitive process and the relationship is unclear between the pathophysiology of the disorders and the component processes underlying delay discounting, including sensitivity to reinforcer delay and sensitivity to reinforcer magnitude. To investigate these processes, male Long Evans rats were trained in one of three tasks measuring sensitivity to delay, sensitivity to magnitude, or both (typical delay discounting task). After learning the task, animals were implanted with bilateral cannulae into either the nucleus accumbens core (AcbC) or the lateral orbitofrontal cortex (lOFC), both of which have been implicated in delay discounting. Upon recovering from the surgery, a baclofen/muscimol cocktail was infused to temporarily inactivate each of these two regions and task performance was assessed. Unlike previous studies showing that lesions of the AcbC increased delay discounting, partial inactivation of the AcbC decreased delay discounting, although it had no effects on the tasks independently assessing either sensitivity to delay or magnitude. The effects of AcbC inactivation were larger in animals that had low levels of delay discounting at baseline. Inactivation of the lOFC had no effects on behavior in any task. These findings suggest that the AcbC may act to promote impulsive choice in individuals with low impulsivity. Furthermore, the data suggest that the AcbC is able to modulate delay and magnitude sensitivity together, but not either of the two in isolation.

Sex-dependent impacts of low-level lead exposure and prenatal stress on impulsive choice behavior and associated biochemical and neurochemical manifestations

A prior study demonstrated increased overall response rates on a fixed interval (FI) schedule of reward in female offspring that had been subjected to maternal lead (Pb) exposure, prenatal stress (PS) and offspring stress challenge relative to control, prenatal stress alone, lead alone and lead+prenatal stress alone (Virgolini et al., 2008). Response rates on FI schedules have been shown to directly relate to measures of self-control (impulsivity) in children and in infants (Darcheville et al., 1992, 1993). The current study sought to determine whether enhanced effects of Pb±PS would therefore be seen in a more direct measure of impulsive choice behavior, i.e., a delay discounting paradigm. Offspring of dams exposed to 0 or 50ppm Pb acetate from 2 to 3 months prior to breeding through lactation, with or without immobilization restraint stress (PS) on gestational days 16 and 17, were trained on a delay discounting paradigm that offered a choice between a large reward (three 45mg food pellets) after a long delay or a small reward (one 45mg food pellet) after a short delay, with the long delay value increased from 0s to 30s across sessions. Alterations in extinction of this performance, and its subsequent re-acquisition after reinforcement delivery was reinstated were also examined. Brains of littermates of behaviorally-trained offspring were utilized to examine corresponding changes in monoamines and in levels of brain derived neurotrophic factor (BDNF), the serotonin transporter (SERT) and the N-methyl-d-aspartate receptor (NMDAR) 2A in brain regions associated with impulsive choice behavior. Results showed that Pb±PS-induced changes in delay discounting occurred almost exclusively in males. In addition to increasing percent long delay responding at the indifference point (i.e., reduced impulsive choice behavior), Pb±PS slowed acquisition of delayed discounting performance, and increased numbers of both failures to and latencies to initiate trials. Overall, the profile of these alterations were more consistent with impaired learning/behavioral flexibility and/or with enhanced sensitivity to the downshift in reward opportunities imposed by the transition from delay discounting training conditions to delay discounting choice response contingencies. Consistent with these behavioral changes, Pb±PS treated males also showed reductions in brain serotonin function in all mesocorticolimbic regions, broad monoamine changes in nucleus accumbens, and reductions in both BDNF and NMDAR 2A levels and increases in SERT in frontal cortex, i.e., in regions and neurotransmitter systems known to mediate learning/behavioral flexibility, and which were of greater impact in males. The current findings do not fully support a generality of the enhancement of Pb effects by PS, as previously seen with FI performance in females (Virgolini et al., 2008), and suggest a dissociation of the behaviors controlled by FI and delay discounting paradigms, at least in response to Pb±PS in rats. Collectively, however, the findings remain consistent with sex-dependent differences in the impacts of both Pb and PS and with the need to understand both the role of contingencies of reinforcement and underlying neurobiological effects in these sex differences.

Dopaminergic modulation of impulsive decision making in the rat insular cortex

Neuroimaging studies have implicated the insular cortex in cognitive processes including decision making. Nonetheless, little is known about the mechanisms by which the insula contributes to impulsive decision making. In this regard, the dopamine system is known to be importantly involved in decision making processes, including impulsive decision making. The aim of the current set of experiments was to further elucidate the importance of dopamine signaling in the agranular insular cortex in impulsive decision making. This compartment of the insular cortex is highly interconnected with brain areas such as the medial prefrontal cortex, amygdala and ventral striatum which are implicated in decision making processes. Male rats were trained in a delay-discounting task and upon stable baseline performance implanted with bilateral cannulae in the agranular insular cortex. Intracranial infusions of the dopamine D1 receptor antagonist SCH23390 and dopamine D2 receptor antagonist eticlopride revealed that particularly blocking dopamine D1 receptors centered on the insular cortex promoted impulsive decision making. Together, the present results demonstrate an important role of the agranular insular cortex in impulsive decision making and, more specifically, highlight the contribution of dopamine D1-like receptors.

Endocannabinoids promote cocaine-induced impulsivity and its rapid dopaminergic correlates

BACKGROUND

Impaired decision making, a hallmark of addiction, is hypothesized to arise from maladaptive plasticity in the mesolimbic dopamine pathway. The endocannabinoid system modulates dopamine activity through activation of cannabinoid type 1 receptors (CB1Rs). Here, we investigated whether impulsive behavior observed following cocaine exposure requires CB1R activation.

METHODS

We trained rats in a delay-discounting task. Following acquisition of stable performance, rats were exposed to cocaine (10 mg/kg, intraperitoneal) every other day for 14 days and locomotor activity was measured. Two days later, delay-discounting performance was re-evaluated. To assess reversal of impulsivity, injections of a CB1R antagonist (1.5 mg/kg, intraperitoneal) or vehicle were given 30 minutes before the task. During the second experiment, aimed at preventing impulsivity rather than reversing it, CB1Rs were antagonized before each cocaine injection. In this experiment, subsecond dopamine release was measured in the nucleus accumbens during delay-discounting sessions before and after cocaine treatment.

RESULTS

Blockade of CB1Rs reversed and prevented cocaine-induced impulsivity. Electrochemical results showed that during baseline and following disruption of endocannabinoid signaling, there was a robust increase in dopamine for immediate large rewards compared with immediate small rewards, but this effect reversed when the delay for the large reward was 10 seconds. In contrast, dopamine release always increased for one-pellet options at minimal or moderate delays in vehicle-treated rats.

CONCLUSIONS

Endocannabinoids play a critical role in changes associated with cocaine exposure. Cannabinoid type 1 receptor blockade may thus counteract maladaptive alterations in afferents to dopamine neurons, thereby preventing changes in dopaminergic activity underlying a loss of self-control.

Ventral striatum lesions enhance stimulus and response encoding in dorsal striatum

BACKGROUND

The development of addiction is thought to reflect a transition from goal-directed to stimulus-response driven behavior, functions attributed to ventral (VS) and dorsal striatum (DS), respectively. In line with this theory, neuroadaptations that occur during prolonged drug use progress from VS to DS. Here we ask if VS dysfunction alone, independent of drug use, can affect neural selectivity in DS.

METHODS

To address this issue, we recorded from single neurons in DS while rats performed an odor-guided choice task for differently valued rewards in rats with and without unilateral VS lesions. In a separate group of animals, we used bilateral VS lesions to determine if VS was critical for performance on this task.

RESULTS

We describe data showing that unilateral lesions of VS enhance neural representations in DS during performance of a task that is dependent on VS. Furthermore, we show that VS is critical for reward-guided decision-making initially, but that rats regain function after several days.

CONCLUSIONS

These results suggest that loss of VS function, independent of chronic drug use, can trigger stronger encoding in DS in a reward-guided decision-making task and that the transition from VS to DS governed behavior observed in addiction might be due, in part, to initial loss of VS function.

Nucleus accumbens core lesions enhance two-way active avoidance

The majority of work examining the nucleus accumbens core (NAc) has focused on functions pertaining to behaviors guided by appetitive outcomes. These studies have pointed to the NAc as being critical for motivating behavior toward desirable outcomes. For example, we have recently shown that lesions of the NAc impaired performance on a reward-guided decision-making task that required rats to choose between differently valued rewards. Unfortunately, much less is known about the role that the NAc plays in motivating behavior when aversive outcomes are predicted. To address this issue we asked if NAc lesions impact performance on a two-way active avoidance task in which rats must learn to shuttle back and forth in a behavioral training box in order to avoid a footshock predicted by an auditory tone. Although bilateral NAc lesions initially impaired reward-guided decision-making, we found that the same lesions improved acquisition and retention of two-way active avoidance.

Increased firing to cues that predict low-value reward in the medial orbitofrontal cortex

Anatomical, imaging, and lesion work have suggested that medial and lateral aspects of orbitofrontal cortex (OFC) play different roles in reward-guided decision-making, yet few single-neuron recording studies have examined activity in more medial parts of the OFC (mOFC) making it difficult to fully assess its involvement in motivated behavior. Previously, we have shown that neurons in lateral parts of the OFC (lOFC) selectively fire for rewards of different values. In that study, we trained rats to respond to different fluid wells for rewards of different sizes or delivered at different delays. Rats preferred large over small reward, and rewards delivered after short compared with long delays. Here, we recorded from single neurons in rat rostral mOFC as they performed the same task. Similar to the lOFC, activity was attenuated for rewards that were delivered after long delays and was enhanced for delivery of larger rewards. However, unlike lOFC, odor-responsive neurons in the mOFC were more active when cues predicted low-value outcomes. These data suggest that odor-responsive mOFC neurons signal the association between environmental cues and unfavorable outcomes during decision making.

Signaling pathways involved in striatal synaptic plasticity are sensitive to temporal pattern and exhibit spatial specificity

The basal ganglia is a brain region critically involved in reinforcement learning and motor control. Synaptic plasticity in the striatum of the basal ganglia is a cellular mechanism implicated in learning and neuronal information processing. Therefore, understanding how different spatio-temporal patterns of synaptic input select for different types of plasticity is key to understanding learning mechanisms. In striatal medium spiny projection neurons (MSPN), both long term potentiation (LTP) and long term depression (LTD) require an elevation in intracellular calcium concentration; however, it is unknown how the post-synaptic neuron discriminates between different patterns of calcium influx. Using computer modeling, we investigate the hypothesis that temporal pattern of stimulation can select for either endocannabinoid production (for LTD) or protein kinase C (PKC) activation (for LTP) in striatal MSPNs. We implement a stochastic model of the post-synaptic signaling pathways in a dendrite with one or more diffusionally coupled spines. The model is validated by comparison to experiments measuring endocannabinoid-dependent depolarization induced suppression of inhibition. Using the validated model, simulations demonstrate that theta burst stimulation, which produces LTP, increases the activation of PKC as compared to 20 Hz stimulation, which produces LTD. The model prediction that PKC activation is required for theta burst LTP is confirmed experimentally. Using the ratio of PKC to endocannabinoid production as an index of plasticity direction, model simulations demonstrate that LTP exhibits spine level spatial specificity, whereas LTD is more diffuse. These results suggest that spatio-temporal control of striatal information processing employs these Gq coupled pathways.

How costs influence decision values for mixed outcomes

The things that we hold dearest often require a sacrifice, as epitomized in the maxim “no pain, no gain.” But how is the subjective value of outcomes established when they consist of mixtures of costs and benefits? We describe theoretical models for the integration of costs and benefits into a single value, drawing on both the economic and the empirical literatures, with the goal of rendering them accessible to the neuroscience community. We propose two key assays that go beyond goodness of fit for deciding between the dominant additive model and four varieties of interactive models. First, how they model decisions between costs when reward is not on offer; and second, whether they predict changes in reward sensitivity when costs are added to outcomes, and in what direction. We provide a selective review of relevant neurobiological work from a computational perspective, focusing on those studies that illuminate the underlying valuation mechanisms. Cognitive neuroscience has great potential to decide which of the theoretical models is actually employed by our brains, but empirical work has yet to fully embrace this challenge. We hope that future research improves our understanding of how our brain decides whether mixed outcomes are worthwhile.