Dynamics of neural coding in the accumbens during extinction and reinstatement of rewarded behavior

Distinct Action Signals by Subregions in the Nucleus Accumbens during STOP-Change Performance

The nucleus accumbens (NAc) is thought to contribute to motivated behavior by signaling the value of reward-predicting cues and the delivery of anticipated reward. The NAc is subdivided into core and shell, with each region containing different populations of neurons that increase or decrease firing to rewarding events. While there are numerous theories of functions pertaining to these subregions and cell types, most are in the context of reward processing, with fewer considering that the NAc might serve functions related to action selection more generally. We recorded from single neurons in the NAc as rats of both sexes performed a STOP-change task that is commonly used to study motor control and impulsivity. In this task, rats respond quickly to a spatial cue on 80% of trials (GO) and must stop and redirect planned movement on 20% of trials (STOP). We found that the activity of reward-excited neurons signaled accurate response direction on GO, but not STOP, trials and that these neurons exhibited higher precue firing after correct trials. In contrast, reward-inhibited neurons significantly represented response direction on STOP trials at the time of the instrumental response. Finally, the proportion of reward-excited to reward-inhibited neurons and the strength of precue firing decreased as the electrode traversed the NAc. We conclude that reward-excited cells (more common in core) promote proactive action selection, while reward-inhibited cells (more common in shell) contribute to accurate responding on STOP trials that require reactive suppression and redirection of behavior.

Nucleus accumbens neurons dynamically respond to appetitive and aversive associative learning

To survive, individuals must learn to associate cues in the environment with emotionally relevant outcomes. This association is partially mediated by the nucleus accumbens (NAc), a key brain region of the reward circuit that is mainly composed by GABAergic medium spiny neurons (MSNs), that express either dopamine receptor D1 or D2. Recent studies showed that both populations can drive reward and aversion, however, the activity of these neurons during appetitive and aversive Pavlovian conditioning remains to be determined. Here, we investigated the relevance of D1- and D2-neurons in associative learning, by measuring calcium transients with fiber photometry during appetitive and aversive Pavlovian tasks in mice. Sucrose was used as a positive valence unconditioned stimulus (US) and foot shock was used as a negative valence US. We show that during appetitive Pavlovian conditioning, D1- and D2-neurons exhibit a general increase in activity in response to the conditioned stimuli (CS). Interestingly, D1- and D2-neurons present distinct changes in activity after sucrose consumption that dynamically evolve throughout learning. During the aversive Pavlovian conditioning, D1- and D2-neurons present an increase in the activity in response to the CS and to the US (shock). Our data support a model in which D1- and D2-neurons are concurrently activated during appetitive and aversive conditioning.

Reinstatement of Pavlovian responses to alcohol cues by stress

RATIONALE

Stress may contribute to relapse to alcohol use in part by enhancing reactivity to cues previously paired with alcohol. Yet, standard models of stress-induced reinstatement generally use contingent presentations of alcohol-paired cues to reinforce instrumental behaviors, making it difficult to isolate the ability of cues to invigorate alcohol-seeking.

OBJECTIVE

Here we sought to test the impact of stress on behavioral responses to alcohol-paired cues, using a model of stress-induced reinstatement of Pavlovian conditioned approach, inspired by Nadia Chaudhri's work on context-induced reinstatement.

METHODS

Long Evans rats were trained to associate one auditory cue with delivery of alcohol or sucrose and an alternative auditory cue with no reward. Following extinction training, rats were exposed to a stressor prior to being re-exposed to the cues under extinction conditions. We assessed the effects of yohimbine, intermittent footshock and olfactory cues paired with social defeat on responses to alcohol-paired cues and the effects of yohimbine on responses to sucrose-paired cues.

RESULTS

The pharmacological stressor, yohimbine, enhanced alcohol seeking in a Pavlovian setting, but not in a cue-selective manner. Intermittent footshock and social defeat cues did not enhance alcohol seeking in this paradigm.

CONCLUSIONS

While yohimbine elicited reinstatement of reward-seeking in a Pavlovian setting, these effects may be unrelated to activation of stress systems or to interactions with specific cues.

The tubular striatum and nucleus accumbens distinctly represent reward-taking and reward-seeking

The ventral striatum regulates motivated behaviors that are essential for survival. The ventral striatum contains both the nucleus accumbens (NAc), which is well established to contribute to motivated behavior, and the adjacent tubular striatum (TuS), which is poorly understood in this context. We reasoned that these ventral striatal subregions may be uniquely specialized in their neural representation of goal-directed behavior. To test this, we simultaneously examined TuS and NAc single-unit activity as male mice engaged in a sucrose self-administration task, which included extinction and cue-induced reinstatement sessions. Although background levels of activity were comparable between regions, more TuS neurons were recruited upon reward-taking, and among recruited neurons, TuS neurons displayed greater changes in their firing during reward-taking and extinction than those in the NAc. Conversely, NAc neurons displayed greater changes in their firing during cue-reinstated reward-seeking. Interestingly, at least in the context of this behavioral paradigm, TuS neural activity predicted reward-seeking, whereas NAc activity did not. Together, by directly comparing their dynamics in several behavioral contexts, this work reveals that the NAc and TuS ventral striatum subregions distinctly represent reward-taking and reward-seeking.NEW & NOTEWORTHY The ventral striatum, considered the reward circuitry "hub," is composed of two regions: the NAc, which is well established for its role in reward processing, and the TuS, which has been largely excluded from such studies. This study provides a first step in directly contextualizing the TuS's activity in relation to that in the NAc and, by doing so, establishes a critical framework for future research seeking to better understand the brain basis for drug addiction.

Amygdala and nucleus accumbens involvement in appetitive extinction

Extinction of appetitive conditioning is regarded as an important model for the treatment of psychiatric disorders like addiction. However, very few studies have investigated its neural correlates. Therefore, we investigated neural correlates of appetitive extinction in a large human sample including all genders (N = 76, 40 females) to replicate and extend results from a previous study. During differential appetitive conditioning, one stimulus (CS+) was paired with the chance to win a monetary reward, whereas another stimulus (CS−) was not. During appetitive extinction on the next day, neither the CS+ nor the CS− were reinforced. After successful acquisition of appetitive conditioning, the extinction phase elicited significant reductions of valence and arousal ratings toward the CS+ and a significant reduction in skin conductance responses to the CS+ from early to late extinction. On a neural level, early extinction showed significant differential (CS+ − CS−) activation in dACC and hippocampus, whereas involvement of the vACC and caudate nucleus did not replicate. The differential activation of amygdala and nucleus accumbens during late extinction was replicated, with the amygdala displaying significantly higher differential activation during the late phase of extinction as compared to the early phase of extinction. We show discernible signals for reward learning and extinction in subregions of amygdala and nucleus accumbens after extinction learning. This successful replication underlines the role of nucleus accumbens and amygdala in neural models of appetitive extinction in humans that was previously only based on animal findings.

Previous cocaine self-administration disrupts reward expectancy encoding in ventral striatum

The nucleus accumbens core (NAc) is important for integrating and providing information to downstream areas about the timing and value of anticipated reward. Although NAc is one of the first brain regions to be affected by drugs of abuse, we still do not know how neural correlates related to reward expectancy are affected by previous cocaine self-administration. To address this issue, we recorded from single neurons in the NAc of rats that had previously self-administered cocaine or sucrose (control). Neural recordings were then taken while rats performed an odor-guided decision-making task in which we independently manipulated value of expected reward by changing the delay to or size of reward across a series of trial blocks. We found that previous cocaine self-administration made rats more impulsive, biasing choice behavior toward more immediate reward. Further, compared to controls, cocaine-exposed rats showed significantly fewer neurons in the NAc that were responsive during odor cues and reward delivery, and in the reward-responsive neurons that remained, diminished directional and value encoding was observed. Lastly, we found that after cocaine exposure, reward-related firing during longer delays was reduced compared to controls. These results demonstrate that prior cocaine self-administration alters reward-expectancy encoding in NAc, which could contribute to poor decision making observed after chronic cocaine use.

Ventral pallidum encodes relative reward value earlier and more robustly than nucleus accumbens

The ventral striatopallidal system, a basal ganglia network thought to convert limbic information into behavioral action, includes the nucleus accumbens (NAc) and the ventral pallidum (VP), typically described as a major output of NAc. Here, to investigate how reward-related information is transformed across this circuit, we measure the activity of neurons in NAc and VP when rats receive two highly palatable but differentially preferred rewards, allowing us to track the reward-specific information contained within the neural activity of each region. In VP, we find a prominent preference-related signal that flexibly reports the relative value of reward outcomes across multiple conditions. This reward-specific firing in VP is present in a greater proportion of the population and arises sooner following reward delivery than in NAc. Our findings establish VP as a preeminent value signaler and challenge the existing model of information flow in the ventral basal ganglia.

Coordinated Ramping of Dorsal Striatal Pathways preceding Food Approach and Consumption

The striatum controls food-related actions and consumption and is linked to feeding disorders, including obesity and anorexia nervosa. Two populations of neurons project from the striatum: direct pathway medium spiny neurons and indirect pathway medium spiny neurons. The selective contribution of direct pathway medium spiny neurons and indirect pathway medium spiny neurons to food-related actions and consumption remains unknown. Here, we used in vivo electrophysiology and fiber photometry in mice (of both sexes) to record both spiking activity and pathway-specific calcium activity of dorsal striatal neurons during approach to and consumption of food pellets. While electrophysiology revealed complex task-related dynamics across neurons, population calcium was enhanced during approach and inhibited during consumption in both pathways. We also observed ramping changes in activity that preceded both pellet-directed actions and spontaneous movements. These signals were heterogeneous in the spiking units, with neurons exhibiting either increasing or decreasing ramps. In contrast, the population calcium signals were homogeneous, with both pathways having increasing ramps of activity for several seconds before actions were initiated. An analysis comparing population firing rates to population calcium signals also revealed stronger ramping dynamics in the calcium signals than in the spiking data. In a second experiment, we trained the mice to perform an action sequence to evaluate when the ramping signals terminated. We found that the ramping signals terminated at the beginning of the action sequence, suggesting they may reflect upcoming actions and not preconsumption activity. Plasticity of such mechanisms may underlie disorders that alter action selection, such as drug addiction or obesity.SIGNIFICANCE STATEMENT Alterations in striatal function have been linked to pathological consumption in disorders, such as obesity and drug addiction. We recorded spiking and population calcium activity from the dorsal striatum during ad libitum feeding and an operant task that resulted in mice obtaining food pellets. Dorsal striatal neurons exhibited long ramps in activity that preceded actions by several seconds, and may reflect upcoming actions. Understanding how the striatum controls the preparation and generation of actions may lead to improved therapies for disorders, such as drug addiction or obesity.

Neural correlates of appetitive extinction in humans

Appetitive extinction receives attention as an important model for the treatment of psychiatric disorders. However, in humans, its underlying neural correlates remain unknown. To close this gap, we investigated appetitive acquisition and extinction with fMRI in a 2-day monetary incentive delay paradigm. During appetitive conditioning, one stimulus (CS+) was paired with monetary reward, while another stimulus (CS−) was never rewarded. Twenty-four hours later, subjects underwent extinction, in which neither CS was reinforced. Appetitive conditioning elicited stronger skin conductance responses to the CS+ as compared with the CS−. Regarding subjective ratings, the CS+ was rated more pleasant and arousing than the CS− after conditioning. Furthermore, fMRI-results (CS+ − CS−) showed activation of the reward circuitry including amygdala, midbrain and striatal areas. During extinction, conditioned responses were successfully extinguished. In the early phase of extinction, we found a significant activation of the caudate, the hippocampus, the dorsal and ventral anterior cingulate cortex (dACC and vACC). In the late phase, we found significant activation of the nucleus accumbens (NAcc) and the amygdala. Correlational analyses with subjective ratings linked extinction success to the vACC and the NAcc, while associating the dACC with reduced extinction. The results reveal neural correlates of appetitive extinction in humans and extend assumptions from models for human extinction learning.

Optogenetic activation of amygdala projections to nucleus accumbens can arrest conditioned and unconditioned alcohol consummatory behavior

Following a Pavlovian pairing procedure, alcohol-paired cues come to elicit behavioral responses that lead to alcohol consumption. Here we used an optogenetic approach to activate basolateral amygdala (BLA) axonal terminals targeting the shell of nucleus accumbens (AcbSh) and investigated a possible influence over cue-conditioned alcohol seeking and alcohol drinking, based on the demonstrated roles of these areas in behavioral responding to Pavlovian cues and in feeding behavior. Rats were trained to anticipate alcohol or sucrose following the onset of a discrete conditioned stimulus (CS). Channelrhodopsin-mediated activation of the BLA-to-AcbSh pathway concurrent with each CS disrupted cued alcohol seeking. Activation of the same pathway caused rapid cessation of alcohol drinking from a sipper tube. Neither effect was accompanied by an overall change in locomotion. Finally, the suppressive effect of photoactivation on cued-triggered seeking was also evidenced in animals trained with sucrose. Together these findings suggest that photoactivation of BLA terminals in the AcbSh can override the conditioned motivational properties of reward-predictive cues as well as unconditioned consummatory responses necessary for alcohol drinking. The findings provide evidence for a limbic-striatal influence over motivated behavior for orally consumed rewards, including alcohol.

Striatal and Tegmental Neurons Code Critical Signals for Temporal-Difference Learning of State Value in Domestic Chicks

To ensure survival, animals must update the internal representations of their environment in a trial-and-error fashion. Psychological studies of associative learning and neurophysiological analyses of dopaminergic neurons have suggested that this updating process involves the temporal-difference (TD) method in the basal ganglia network. However, the way in which the component variables of the TD method are implemented at the neuronal level is unclear. To investigate the underlying neural mechanisms, we trained domestic chicks to associate color cues with food rewards. We recorded neuronal activities from the medial striatum or tegmentum in a freely behaving condition and examined how reward omission changed neuronal firing. To compare neuronal activities with the signals assumed in the TD method, we simulated the behavioral task in the form of a finite sequence composed of discrete steps of time. The three signals assumed in the simulated task were the prediction signal, the target signal for updating, and the TD-error signal. In both the medial striatum and tegmentum, the majority of recorded neurons were categorized into three types according to their fitness for three models, though these neurons tended to form a continuum spectrum without distinct differences in the firing rate. Specifically, two types of striatal neurons successfully mimicked the target signal and the prediction signal. A linear summation of these two types of striatum neurons was a good fit for the activity of one type of tegmental neurons mimicking the TD-error signal. The present study thus demonstrates that the striatum and tegmentum can convey the signals critically required for the TD method. Based on the theoretical and neurophysiological studies, together with tract-tracing data, we propose a novel model to explain how the convergence of signals represented in the striatum could lead to the computation of TD error in tegmental dopaminergic neurons.

Orbitofrontal lesions eliminate signalling of biological significance in cue-responsive ventral striatal neurons

The ventral striatum has long been proposed as an integrator of biologically significant associative information to drive actions. While inputs from the amygdala and hippocampus have been much studied, the role of prominent inputs from orbitofrontal cortex (OFC) are less well understood. Here we recorded single unit activity from ventral striatum core in rats with sham or ipsilateral neurotoxic lesions of lateral OFC, as they performed an odor-guided spatial choice task. Consistent with prior reports, we found that spiking activity recorded in sham rats during cue sampling was related to both reward magnitude and reward identity, with higher firing rates observed for cues that predicted more reward. Lesioned rats also showed differential activity to the cues, but this activity was unbiased towards larger rewards. These data support a role for OFC in shaping activity in the ventral striatum to represent the biological significance of associative information in the environment.

Neurophysiology of Reward-Guided Behavior: Correlates Related to Predictions, Value, Motivation, Errors, Attention, and Action

Many brain areas are activated by the possibility and receipt of reward. Are all of these brain areas reporting the same information about reward? Or are these signals related to other functions that accompany reward-guided learning and decision-making? Through carefully controlled behavioral studies, it has been shown that reward-related activity can represent reward expectations related to future outcomes, errors in those expectations, motivation, and signals related to goal- and habit-driven behaviors. These dissociations have been accomplished by manipulating the predictability of positively and negatively valued events. Here, we review single neuron recordings in behaving animals that have addressed this issue. We describe data showing that several brain areas, including orbitofrontal cortex, anterior cingulate, and basolateral amygdala signal reward prediction. In addition, anterior cingulate, basolateral amygdala, and dopamine neurons also signal errors in reward prediction, but in different ways. For these areas, we will describe how unexpected manipulations of positive and negative value can dissociate signed from unsigned reward prediction errors. All of these signals feed into striatum to modify signals that motivate behavior in ventral striatum and guide responding via associative encoding in dorsolateral striatum.

Competitor suppresses neuronal representation of food reward in the nucleus accumbens/medial striatum of domestic chicks

To investigate the role of social contexts in controlling the neuronal representation of food reward, we recorded single neuron activity in the medial striatum/nucleus accumbens of domestic chicks and examined whether activities differed between two blocks with different contexts. Chicks were trained in an operant task to associate light-emitting diode color cues with three trial types that differed in the type of food reward: no reward (S-), a small reward/short-delay option (SS), and a large reward/long-delay alternative (LL). Amount and duration of reward were set such that both of SS and LL were chosen roughly equally. Neurons showing distinct cue-period activity in rewarding trials (SS and LL) were identified during an isolation block, and activity patterns were compared with those recorded from the same neuron during a subsequent pseudo-competition block in which another chick was allowed to forage in the same area, but was separated by a transparent window. In some neurons, cue-period activity was lower in the pseudo-competition block, and the difference was not ascribed to the number of repeated trials. Comparison at neuronal population level revealed statistically significant suppression in the pseudo-competition block in both SS and LL trials, suggesting that perceived competition generally suppressed the representation of cue-associated food reward. The delay- and reward-period activities, however, did not significantly different between blocks. These results demonstrate that visual perception of a competitive forager per se weakens the neuronal representation of predicted food reward. Possible functional links to impulse control are discussed.

Impact of appetitive and aversive outcomes on brain responses: linking the animal and human literatures

Decision-making is motivated by the possibility of obtaining reward and/or avoiding punishment. Though many have investigated behavior associated with appetitive or aversive outcomes, few have examined behaviors that rely on both. Fewer still have addressed questions related to how anticipated appetitive and aversive outcomes interact to alter neural signals related to expected value, motivation, and salience. Here we review recent rodent, monkey, and human research that address these issues. Further development of this area will be fundamental to understanding the etiology behind human psychiatric diseases and cultivating more effective treatments.

Separate populations of neurons in ventral striatum encode value and motivation

Neurons in the ventral striatum (VS) fire to cues that predict differently valued rewards. It is unclear whether this activity represents the value associated with the expected reward or the level of motivation induced by reward anticipation. To distinguish between the two, we trained rats on a task in which we varied value independently from motivation by manipulating the size of the reward expected on correct trials and the threat of punishment expected upon errors. We found that separate populations of neurons in VS encode expected value and motivation.

Ventral striatum encodes past and predicted value independent of motor contingencies

The ventral striatum (VS) is thought to signal the predicted value of expected outcomes. However, it is still unclear whether VS can encode value independently from variables often yoked to value such as response direction and latency. Expectations of high value reward are often associated with a particular action and faster latencies. To address this issue we trained rats to perform a task in which the size of the predicted reward was signaled before the instrumental response was instructed. Instrumental directional cues were presented briefly at a variable onset to reduce accuracy and increase reaction time. Rats were more accurate and slower when a large versus small reward was at stake. We found that activity in VS was high during odors that predicted large reward even though reaction times were slower under these conditions. In addition to these effects, we found that activity before the reward predicting cue reflected past and predicted reward. These results demonstrate that VS can encode value independent of motor contingencies and that the role of VS in goal-directed behavior is not just to increase vigor of specific actions when more is at stake.

Impact of size and delay on neural activity in the rat limbic corticostriatal system

A number of factors influence an animal’s economic decisions. Two most commonly studied are the magnitude of and delay to reward. To investigate how these factors are represented in the firing rates of single neurons, we devised a behavioral task that independently manipulated the expected delay to and size of reward. Rats perceived the differently delayed and sized rewards as having different values and were more motivated under short delay and big-reward conditions than under long delay and small reward conditions as measured by percent choice, accuracy, and reaction time. Since the creation of this task, we have recorded from several different brain areas including, orbitofrontal cortex, striatum, amygdala, substantia nigra pars reticulata, and midbrain dopamine neurons. Here, we review and compare those data with a substantial focus on those areas that have been shown to be critical for performance on classic time discounting procedures and provide a potential mechanism by which they might interact when animals are deciding between differently delayed rewards. We found that most brain areas in the cortico-limbic circuit encode both the magnitude and delay to reward delivery in one form or another, but only a few encode them together at the single neuron level.

Extinction of drug seeking

Drug seeking behavior can be reduced or inhibited via extinction. The brain mechanisms for extinction of drug seeking are poorly understood but are of significant interest because of their potential to identify novel approaches that promote abstinence from drug taking. Here we review recent literature on the neural mechanisms for extinction in drug self-administration paradigms. First, we consider the brain regions important for extinction of drug seeking. Functional inactivation studies have identified infralimbic prefrontal cortex, nucleus accumbens shell, as well as medial dorsal hypothalamus in the expression of extinction of drug seeking. These structures have been implicated in extinction expression across several reinforcers including cocaine, heroin, and alcohol. Second, we consider molecular studies which show that extinction training is associated with plasticity in glutamatergic signaling in both nucleus accumbens shell and core, and that this training may reverse or ameliorate the neuroadaptations produced by chronic drug exposure and spontaneous withdrawal. Finally, we consider the neural circuitry for extinction of drug seeking. Functional disconnection and neuroanatomical tracing studies show that extinction expression depends, at least in part, on cortico-striatal-hypothalamic and cortico-hypothalalmic-thalamic pathways. Moreover, they indicate that the expression of extinction and reinstatement of drug seeking may depend on parallel pathways that converge within lateral hypothalamus and paraventricular thalamus.

Individual Differences in Nucleus Accumbens Dopamine Receptors Predict Development of Addiction-Like Behavior: A Computational Approach

Clinical and experimental observations show individual differences in the development of addiction. Increasing evidence supports the hypothesis that dopamine receptor availability in the nucleus accumbens (NAc) predisposes drug reinforcement. Here, modeling striatal-midbrain dopaminergic circuit, we propose a reinforcement learning model for addiction based on the actor-critic model of striatum. Modeling dopamine receptors in the NAc as modulators of learning rate for appetitive—but not aversive—stimuli in the critic—but not the actor—we define vulnerability to addiction as a relatively lower learning rate for the appetitive stimuli, compared to aversive stimuli, in the critic. We hypothesize that an imbalance in this learning parameter used by appetitive and aversive learning systems can result in addiction. We elucidate that the interaction between the degree of individual vulnerability and the duration of exposure to drug has two progressive consequences: deterioration of the imbalance and establishment of an abnormal habitual response in the actor. Using computational language, the proposed model describes how development of compulsive behavior can be a function of both degree of drug exposure and individual vulnerability. Moreover, the model describes how involvement of the dorsal striatum in addiction can be augmented progressively. The model also interprets other forms of addiction, such as obesity and pathological gambling, in a common mechanism with drug addiction. Finally, the model provides an answer for the question of why behavioral addictions are triggered in Parkinson's disease patients by D2 dopamine agonist treatments.

The role of the nucleus accumbens core in impulsive choice, timing, and reward processing

The present series of experiments aimed to pinpoint the source of nucleus accumbens core (AcbC) effects on delay discounting. Rats were trained with an impulsive choice procedure between an adjusting smaller sooner reward and a fixed larger later reward. The AcbC-lesioned rats produced appropriate choice behavior when the reward magnitude was equal. An increase in reward magnitude resulted in a failure to increase preference for the larger later reward in the AcbC-lesioned rats, whereas a decrease in the larger later reward duration resulted in normal alterations in choice behavior in AcbC-lesioned rats. Subsequent experiments with a peak timing (Experiments 2 and 3) and a behavioral contrast (Experiment 4) indicated that the AcbC-lesioned rats suffered from decreased incentive motivation during changes in reward magnitude (Experiments 2 and 4) and when expected rewards were omitted (Experiments 2 and 3), but displayed intact anticipatory timing of reward delays (Experiments 2 and 3). The results indicate that the nucleus accumbens core is critical for determining the incentive value of rewards, but does not participate in the timing of reward delays.

Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards

The ventral striatum (VS) is thought to serve as a gateway whereby associative information from the amygdala and prefrontal regions can influence motor output to guide behavior. If VS mediates this "limbic-motor" interface, then one might expect neural correlates in VS to reflect this information. Specifically, neural activity should reflect the integration of motivational value with subsequent behavior. To test this prediction, we recorded from single units in VS while rats performed a choice task in which different odor cues indicated that reward was available on the left or on the right. The value of reward associated with a left or rightward movement was manipulated in separate blocks of trials by either varying the delay preceding reward delivery or by changing reward size. Rats' behavior was influenced by the value of the expected reward and the response required to obtain it, and activity in the majority of cue-responsive VS neurons reflected the integration of these two variables. Unlike similar cue-evoked activity reported previously in dopamine neurons, these correlates were only observed if the directional response was subsequently executed. Furthermore, activity was correlated with the speed at which the rats' executed the response. These results are consistent with the notion that VS serves to integrate information about the value of an expected reward with motor output during decision making.

Dynamic encoding of action selection by the medial striatum

Successful foragers respond flexibly to environmental stimuli. Behavioral flexibility depends on a number of brain areas that send convergent projections to the medial striatum, such as the medial prefrontal cortex, orbital frontal cortex, and amygdala. Here, we tested the hypothesis that neurons in the medial striatum are involved in flexible action selection, by representing changes in stimulus-reward contingencies. Using a novel Go/No-go reaction-time task, we changed the reward value of individual stimuli within single experimental sessions. We simultaneously recorded neuronal activity in the medial and ventral parts of the striatum of rats. The rats modified their actions in the task after the changes in stimulus-reward contingencies. This was preceded by dynamic modulations of spike activity in the medial, but not the ventral, striatum. Our results suggest that the medial striatum biases animals to collect rewards to potentially valuable stimuli and can rapidly influence flexible behavior.

Anticipatory reward signals in ventral striatal neurons of behaving rats

It has been proposed that the striatum plays a crucial role in learning to select appropriate actions, optimizing rewards according to the principles of 'Actor-Critic' models of trial-and-error learning. The ventral striatum (VS), as Critic, would employ a temporal difference (TD) learning algorithm to predict rewards and drive dopaminergic neurons. This study examined this model's adequacy for VS responses to multiple rewards in rats. The respective arms of a plus-maze provided rewards of varying magnitudes; multiple rewards were provided at 1-s intervals while the rat stood still. Neurons discharged phasically prior to each reward, during both initial approach and immobile waiting, demonstrating that this signal is predictive and not simply motor-related. In different neurons, responses could be greater for early, middle or late droplets in the sequence. Strikingly, this activity often reappeared after the final reward, as if in anticipation of yet another. In contrast, previous TD learning models show decremental reward-prediction profiles during reward consumption due to a temporal-order signal introduced to reproduce accurate timing in dopaminergic reward-prediction error signals. To resolve this inconsistency in a biologically plausible manner, we adapted the TD learning model such that input information is nonhomogeneously distributed among different neurons. By suppressing reward temporal-order signals and varying richness of spatial and visual input information, the model reproduced the experimental data. This validates the feasibility of a TD-learning architecture where different groups of neurons participate in solving the task based on varied input information.

Rat Nucleus Accumbens Neurons Persistently Encode Locations Associated With Morphine Reward

When rats and mice are free to explore a familiar environment they spend more time in a previously rewarded location. This conditioned place preference (CPP) results from an increased probability of initiating transitions from an unrewarded location to one previously paired with reward. We recorded nucleus accumbens (NAc) neurons while rats explored a three-room in-line apparatus. Before place conditioning, approximately equal proportions of NAc neurons show excitations or inhibitions when the rat is in each of the rooms (morphine paired, center or saline paired). Conditioning increased the proportion of neurons inhibited while the rat was in the morphine room and neurons excited in the saline room. Many of the neurons in these two groups responded during room transitions. Furthermore, the postconditioning increase in the population of neurons with room-selective responding persisted for several weeks after the last morphine treatment. This long-lasting change in population responses of NAc neurons to initially neutral locations is a neural correlate of the change in location preference manifest as CPP.

Encoding of palatability and appetitive behaviors by distinct neuronal populations in the nucleus accumbens

Obesity is a major public health problem. Palatability (i.e., the reinforcing value of food, derived from orosensory cues) is a significant factor in determining food intake and contributes to increased consumption leading to obesity. The nucleus accumbens is a ventral striatal region that is important for both appetitive and consummatory behaviors and has been implicated in modulating palatability. In this study, we investigated palatability encoding in the firing of nucleus accumbens neurons in rats. Nucleus accumbens neurons with significant changes in firing rate during consummatory behavior displayed one of two principal firing patterns. Firing in one class of nucleus accumbens neurons was correlated with the palatability of sucrose reinforcers; changes in neural activity in this class consisted primarily of excitations. Within this group of neurons, a subset was sensitive to the relative value of sucrose reinforcers, as assessed by a behavioral contrast paradigm. A second and distinct population of nucleus accumbens neurons, with changes in firing that were pre-dominantly inhibitions, was not sensitive to reinforcer palatability; rather, these inhibitions were present even during unreinforced bouts of licking. In addition, the onset of these inhibitions typically occurred before the initiation of the licking behavior itself. We propose that two primary classes of nucleus accumbens neurons contribute to neural processing immediately before and during reinforcer consumption: inhibitions related to initiation and maintenance of consummatory behaviors and excitations that encode reinforcer palatability.