Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards

Higher magnitude accumbal phasic firing changes among core neurons exhibiting tonic firing increases during cocaine self-administration

Studies using i.v. cocaine self-administration in rats have documented rapid-phasic changes in the firing rate of nucleus accumbens neurons within seconds of cocaine-reinforced lever presses, as well as changes that occur over the course of the cocaine self-administration experiment, i.e. tonic changes in firing rate. During the self-administration period of the experiment, individual neurons exhibit either a tonic increase, a tonic decrease, or no tonic change in firing rate, relative to the neuron's firing rate during the pre-drug period. We evaluated whether rapid-phasic changes in firing were differentially associated with tonically reduced or tonically elevated firing of nucleus accumbens core and shell neurons in cocaine self-administering rats. Rapid-phasic firing patterns within seconds of the cocaine-reinforced lever press were exhibited predominantly by core neurons that also exhibited tonic increases in firing. Conversely, core neurons that did not exhibit such rapid-phasic firing patterns were more likely to show tonically reduced firing. Moreover, core neurons were more likely than shell neurons to exhibit: 1) tonic increases in firing and 2) rapid-phasic increases in firing preceding the cocaine-reinforced lever press. These differences between accumbens subterritories may be related to their distinct involvement in operant responding; the present findings are consistent with an emerging literature which implicates shell in contextual stimulus-induced responding, and core in processing the instrumental response via its discrete output to classic basal ganglia structures. The distinct tendency of the core to exhibit increased firing, coupled with its dichotomous firing outputs (i.e. tonic decreases without rapid phasic responses or tonic increases with rapid phasic responses), may reflect particular sensitivity of these neurons to excitatory limbic afferent signaling involved in instrumental responding. Enhanced phasic responsivity in the core may be an integral component of the mechanism inherent in normal reward processing which is subverted by chronic drug exposure.

Neuron activity in the anterolateral motor cortex in operant food-acquiring and alcohol-acquiring behavior

The interactions of the neuronal mechanisms of food-acquiring behavior and newly formed operant alcohol-acquiring behavior were studied by recording the activity of individual neurons in the anterolateral area of the motor cortex in chronically alcoholized rabbits. Adult animals learned food-acquiring behavior in a cage with two feeders and two pedals, in the comers (the food in the feeders was presented after pressing the corresponding pedal). After nine months of chronic alcoholization, the same rabbits learned an alcohol-acquiring behavior in the same experimental cage (gelatin capsules filled with 15% ethanol solution were placed in the feeders instead of food). Analysis of neuron activity showed that the set of neurons involved in supporting food-acquiring and alcohol-acquiring behaviors overlapped, though not completely. These experiments not only help us understand the neuronal mechanisms of the newly formed and the previously formed behaviors, but also facilitate the development of concepts of the similarity of the neuronal mechanisms of long-term memory and long-term modifications of the nervous system, occurring in conditions of repeated intake of addictive substances.

The subthalamic nucleus exerts opposite control on cocaine and 'natural' rewards

A challenge in treating drug addicts is preventing their pathological motivation for the drug without impairing their general affective state toward natural reinforcers. Here we have shown that discrete lesions of the subthalamic nucleus greatly decreased the motivation of rats for cocaine while increasing it for food reward. The subthalamic nucleus, a key structure controlling basal ganglia outputs, is therefore able to oppositely modulate the effect of 'natural' rewards and drugs of abuse on behavior. Modulating the activity of the subthalamic nucleus might prove to be a new target for the treatment of cocaine addiction.

Neuronal firing in anterior cingulate neurons changes modes across trials in single states of multitrial reward schedules

The recorded responses of single neurons often vary considerably in the numbers of spikes emitted across repeats of a single experimental condition. Because of this irregularity and for theoretical convenience the responses are often approximated using a Poisson process. However, it has been frequently pointed out that many details of the responses, including the distribution of spike counts across similar trials, are not consistent with a Poisson process, even an inhomogeneous one. Wiener and Richmond (2003, J Neurosci 23:2394-2406) showed that the spike count distributions could usually be fitted nicely by mixtures of a few (1-3) Poisson distributions, a step they regarded as a computational convenience. Now, we find that a substantial proportion (47%) of the neuronal responses from anterior cingulate cortex, which we conceptualize as part of a system related to the balance between work and reward, have responses with multimodal firing rate distributions. When these distributions are modeled as mixtures of Poisson distributions, the proportions of the different Poisson distributions are related to behavioral state, and might be related to cognitive factors. This suggests that the neurons undergo behaviorally-related mode changes.

Relative reward processing in primate striatum

Rewards are often not only valued according to their physical characteristics but also relative to other available rewards. The striatum (caudate nucleus, putamen, ventral striatum including nucleus accumbens) is involved in the organization of movement and the processing of reward information. We studied the activity of single striatal neurons in macaques that were presented with different combinations of two rewards. We found in nearly half of the investigated neurons that the processing for one reward shifted, relative to the other rewards that were available in a given trial block. The relative reward processing concerned all forms of striatal activity related to reward-predicting visual stimuli, arm movements and reception of rewards. The observed changes may provide a neural basis for the known shifts in valuation of rewarding outcomes relative to known references.

Rat nucleus accumbens neurons predominantly respond to the outcome-related properties of conditioned stimuli rather than their behavioral-switching properties

It has been proposed that nucleus accumbens neurons respond to outcome (reward and punishment) and outcome-predictive information. Alternatively, it has been suggested that these neurons respond to salient stimuli, regardless of their outcome-predictive properties, to facilitate a switch in ongoing behavior. We recorded the activity of 82 single-nucleus accumbens neurons in thirsty rats responding within a modified go/no-go task. The task design allowed us to analyze whether neurons responded to conditioned stimuli that predicted rewarding (saccharin) or aversive (quinine) outcomes, and whether the neural responses correlated with behavioral switching. Approximately one third (28/82) of nucleus accumbens neurons exhibited 35 responses to conditioned stimuli. Over 2/3 of these responses encoded the nature of the upcoming rewarding (19/35) or aversive (5/35) outcome. No response was selective solely for the switching of the rat's behavior, although the activity of approximately one third of responses (11/35) predicted the upcoming outcome and was correlated with the presence or absence of a subsequent behavioral switch. Our data suggest a primary functional role for the nucleus accumbens in encoding outcome-predicting information and a more limited role in behavioral switching.

Nucleus accumbens neurons in the rat exhibit differential activity to conditioned reinforcers and primary reinforcers within a second-order schedule of saccharin reinforcement

The nucleus accumbens has been associated with processing information related to primary reinforcement and reward. Most neurophysiological studies report that nucleus accumbens neurons are phasically excited in response to the onsets of salient events during the seeking of reinforcement and to the delivery of primary reinforcers. However, a minority of studies report inhibition during primary reinforcement. We recorded from 65 neurons in the nucleus accumbens whilst thirsty rats performed under a second-order schedule of saccharin reinforcement. This allowed us to analyse neural activity and behaviour during reinforcer-seeking in the presence of conditioned reinforcers (second-order stimuli, also called 'conditioned stimuli'), and during primary reinforcer consumption. Specifically, we sought to examine the valence of potential neural responses to primary reinforcement, to compare these responses to second-order stimulus-evoked responses, and to determine whether responses were differential to second-order stimuli presented at different time points within the schedule. Fifty out of 65 neurons we sampled responded to the second-order stimulus and/or consumption of the primary reinforcer. Most neurons in our sample exhibited excitation following the second-order stimulus and inhibition to the primary reinforcer, a pattern also present over the average response of the neural population. However, there was no systematic variation in neural responses evoked by second-order stimuli presented at different temporal proximities to primary reinforcement. Our results provide evidence that partially overlapping mechanisms within the nucleus accumbens differentially process conditioned reinforcers and primary reinforcers.

Nucleus accumbens cell firing and rapid dopamine signaling during goal-directed behaviors in rats

The nucleus accumbens (Acb) is a key neural substrate underlying goal-directed behaviors for both drugs of abuse as well as 'natural' rewards. Here, I review electrophysiological and electrochemical studies completed in our laboratory that examined Acb cell firing and rapid dopamine signaling during behaviors directed toward reward procurement. Electrophysiological studies are reviewed showing that Acb neurons exhibit patterned discharges relative to operant responding for intravenous self-administration of cocaine versus 'natural' reinforcement in rodents. Importantly, subsequent studies showed that discrete subsets of Acb neurons are selectively activated during multiple schedules for a natural reward (water or food) versus cocaine self-administration. These later findings indicate that separate neural circuits selectively process information about goal-directed behaviors for cocaine versus natural reward. In addition, recent findings are reviewed showing that reinforcer selective firing of Acb neurons is not a direct consequence of chronic drug exposure. Next, electrochemical studies are summarized that used fast scan cyclic voltammetry to measure rapid (subsecond) changes in dopamine in the Acb during cocaine self-administration as well as 'natural' reinforcement in rodents. These findings are considered with respect to the role of dopamine in modulating the activity of Acb neurons that encode goal-directed behaviors, the functional organization of the Acb on a microcircuit level, and proposed directions for future studies.

Contrasting effects of dopamine and glutamate receptor antagonist injection in the nucleus accumbens suggest a neural mechanism underlying cue-evoked goal-directed behavior

Discriminative stimuli (DSs) inform animals that reward can be obtained contingent on the performance of a specific behavior. Such stimuli reinstate drug-seeking behavior, evoke dopamine release in the nucleus accumbens (NAc) and excite and inhibit specific subpopulations of NAc neurons. Here we show in rats that DSs can reinstate food-seeking behavior. In addition, we compare the effects of injecting dopamine receptor antagonists into the NAc with those of general NAc inactivation on the performance of a DS task. Selective antagonism of D1 receptors reduced responding to the DS and increased the latency to respond, whereas general inactivation of NAc neuronal activity increased the latency to respond to the DS and increased behaviors extraneous to the task, such as responding in the absence of cues and responding on the inactive lever. Based on these results and our previous findings that NAc neuronal responses to DSs are dependent on the ventral tegmental area, we propose a model for the functional role of NAc neurons in controlling behavioral responses to reward-predictive stimuli.

Differential encoding of information about progress through multi-trial reward schedules by three groups of ventral striatal neurons

In the course of daily activity we continually judge whether the goal sought is worth the work that must be done to obtain it. The ventral striatum is thought to play a central role in making such judgments. When reward schedules are used to investigate these judgments ventral striatum neurons show responses near the time of the cue, the bar-release, and/or the reward delivery. We evaluated the type of coding that occurs at these three time points by using codes or factorizations with: (1) two states for reward versus non-reward, (2) four states for the progress in the reward schedule, and (3) six states for all of the states of the schedule, quantified using information theory and ANOVA. For the bar-release- and reward-related responses the percent variance explained was as high for the two states code as with the six states code. The information for the four state code rose slightly but significantly for the bar-release-related neurons. For the cue-related neurons the code with six states carried more information than the simpler codes. Thus, responses at different times appear to play different roles. Responses occurring early in trials differentiate all states, i.e., the path to a reward, whereas those late in trials code knowledge of impending reward.

Behavioral context and coherent oscillations in the supplementary motor area

Movements with similar physical characteristics can occur in various behavioral contexts, as when they are embedded in different sequences or when the expected outcomes of movements vary. Similarly, neurons in various sensory and motor structures in the brain commonly display modulations in their activity according to contextual factors, such as expected reward. Although these contextual signals must be combined with incoming sensory inputs to generate appropriate behaviors according to the animal's motivational state, the mechanisms by which these two signals are integrated remain poorly understood. The present study examined the effects of contextual factors on the magnitude of coherent oscillations in the activity of individual neurons recorded in the supplementary motor area (SMA) of monkeys during a serial reaction time task. In this task, the animal produced a predictable sequence of hand movements repeatedly according to visual instructions. The performance of the animal was influenced by the location of the rewarded target as well as the ordinal position of the movement. In contrast, the level of coherent oscillations in the activity of SMA neurons was affected only by the rewarded target location but not by the ordinal position of the movement sequence. In addition, changes in coherent oscillations were not accounted for by systematic changes in the mean firing rates. These results are consistent with the proposal that synchronous spikes might be used to control the flow of information and suggest that coherent oscillations in the SMA might encode contextual variables, such as expected reward.

DNA targeting of rhinal cortex D2 receptor protein reversibly blocks learning of cues that predict reward

When schedules of several operant trials must be successfully completed to obtain a reward, monkeys quickly learn to adjust their behavioral performance by using visual cues that signal how many trials have been completed and how many remain in the current schedule. Bilateral rhinal (perirhinal and entorhinal) cortex ablations irreversibly prevent this learning. Here, we apply a recombinant DNA technique to investigate the role of dopamine D2 receptor in rhinal cortex for this type of learning. Rhinal cortex was injected with a DNA construct that significantly decreased D2 receptor ligand binding and temporarily produced the same profound learning deficit seen after ablation. However, unlike after ablation, the D2 receptor-targeted, DNA-treated monkeys recovered cue-related learning after 11-19 weeks. Injecting a DNA construct that decreased N-methyl-d-aspartate but not D2 receptor ligand binding did not interfere with learning associations between the cues and the schedules. A second D2 receptor-targeted DNA treatment administered after either recovery from a first D2 receptor-targeted DNA treatment (one monkey), after N-methyl-d-aspartate receptor-targeted DNA treatment (two monkeys), or after a vector control treatment (one monkey) also induced a learning deficit of similar duration. These results suggest that the D2 receptor in primate rhinal cortex is essential for learning to relate the visual cues to the schedules. The specificity of the receptor manipulation reported here suggests that this approach could be generalized in this or other brain pathways to relate molecular mechanisms to cognitive functions.

Transient inactivation of the rat nucleus accumbens does not impair guidance of instrumental behaviour by stimuli predicting reward magnitude

The involvement of the nucleus accumbens (NAc) in the determination of reaction times (RTs) of instrumental responses by the expectancy of future reward was investigated. A simple RT task demanding conditioned lever release was used, in which the upcoming reward magnitude (5 versus 1 pellet) was signalled in advance by discriminative cues. In rats which acquired the task, RTs of instrumental responses were significantly shorter to the discriminative cue predictive of high reward magnitude. Inactivation of the NAc by lidocaine had no effect on RTs and their determination by cue-associated reward magnitudes, and did not affect the rate of correct responses. In keeping with an earlier study, intra-NAc infusion of amphetamine decreased RTs, impaired RT determination by cue-associated reward magnitudes and reduced the rate of correct responses. The unexpected finding that lidocaine inactivation of the NAc had no effect parallels previous data showing that lesions of NAc did not impair RT performance, while manipulation of intra-NAc glutamate or dopamine transmission impaired various aspects of RT performance in comparable tasks. It is suggested that experimental manipulations such as transient and permanent inactivation, which almost completely inhibit NAc neuronal output, allow alternative routes to be used to effectively control behaviour in the task employed here.

Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology

Neurons in a small number of brain structures detect rewards and reward-predicting stimuli and are active during the expectation of predictable food and liquid rewards. These neurons code the reward information according to basic terms of various behavioural theories that seek to explain reward-directed learning, approach behaviour and decision-making. The involved brain structures include groups of dopamine neurons, the striatum including the nucleus accumbens, the orbitofrontal cortex and the amygdala. The reward information is fed to brain structures involved in decision-making and organisation of behaviour, such as the dorsolateral prefrontal cortex and possibly the parietal cortex. The neural coding of basic reward terms derived from formal theories puts the neurophysiological investigation of reward mechanisms on firm conceptual grounds and provides neural correlates for the function of rewards in learning, approach behaviour and decision-making.

The ventral tegmental area is required for the behavioral and nucleus accumbens neuronal firing responses to incentive cues

Reward-predictive cues exert powerful control over behavioral choice and may be a critical factor in drug addiction. Reward-seeking elicited by predictive cues is facilitated by the release of dopamine in the nucleus accumbens (NAc), yet the contribution of dopamine to the specific NAc firing patterns that underlie goal-directed behavior has remained elusive. We present evidence that subpopulations of NAc neurons that respond to predictive cues require the dopaminergic projection from the ventral tegmental area (VTA) to promote reward-seeking behavior. Rats trained to perform an operant response to a cue to obtain a sucrose reward were implanted with both multiunit recording electrodes in the NAc and microinjection cannulas in the VTA. Both the behavioral response to cues and the cue-evoked firing of NAc neurons were blocked by injection of the GABA(B) agonist baclofen into the VTA. An additional group of rats was trained on the same task and then implanted with microinjection cannulas in the NAc. Like VTA baclofen injection, injection of dopamine receptor antagonists into the NAc profoundly reduced cue-elicited reward seeking. Together, these results support the conclusion that both the behavioral response to the cue and the specific NAc neuronal firing that promotes the response depend on dopamine release within the NAc. Our findings suggest a neural mechanism by which the dopamine-dependent firing of NAc neurons mediates goal-directed behavior.

Neurons in monkey prefrontal cortex whose activity tracks the progress of a three-step self-ordered task

The self-ordered task is a powerful tool for the analysis of dorsal prefrontal deficits. Each trial consists of a number of steps, and subjects must remember their choices in previous steps. The task becomes more difficult as the number of objects to be remembered increases. We recorded the activity of 156 neurons in the mid-dorsal prefrontal cortex of two rhesus monkeys performing an oculomotor version of the task. Although the task requires working memory, there was no convincing evidence for activity selective for the working memory of the objects that the monkey had to remember. Instead, nearly one-half of neurons (47%, 74/156) showed activity that was modulated according to the step of the task in any one or more task periods. Although the monkey's reward also increased with step, the neurons exhibited little or no step modulation in a reward control task in which reward increased without a concurrent increase in task difficulty. The activity of some neurons was also selective for the location of saccade target that the monkey voluntarily chose. Neurons showed less step modulation in error trials, and there was no increase between the second and third step responses on trials in which the error was on the third step. These results suggest that the mid-dorsal prefrontal cortex contributes to the self-ordered task, not by providing an object working memory signal, but by regulating some general aspect of the performance in the difficult task.

Prefrontal cortical-ventral striatal interactions involved in affective modulation of attentional performance: implications for corticostriatal circuit function

Anatomically segregated systems linking the frontal cortex and the striatum are involved in various aspects of cognitive, affective, and motor processing. In this study, we examined the effects of combined unilateral lesions of the medial prefrontal cortex (mPFC) and the core subregion of the nucleus accumbens (AcbC) in opposite hemispheres (disconnection) on a continuous performance, visual attention test [five-choice serial reaction-time task (5CSRTT)]. The disconnection lesion produced a set of specific changes in performance of the 5CSRTT, resembling changes that followed bilateral AcbC lesions while, in addition, comprising a subset of the behavioral changes after bilateral mPFC lesions previously reported using the same task. Specifically, both mPFC/AcbC disconnection and bilateral AcbC lesions markedly affected aspects of response control related to affective feedback, as indexed by perseverative responding in the 5CSRTT. These effects were comparable, although not identical, to those in animals with either bilateral AcbC or mPFC/AcbC disconnection lesions. The mPFC/AcbC disconnection resulted in a behavioral profile largely distinct from that produced by disconnection of a similar circuit described previously, between the mPFC and the dorsomedial striatum, which were shown to form a functional network underlying aspects of visual attention and attention to action. This distinction provides an insight into the functional specialization of corticostriatal circuits in similar behavioral contexts.

The primate basal ganglia: parallel and integrative networks

The basal ganglia and frontal cortex operate together to execute goal directed behaviors. This requires not only the execution of motor plans, but also the behaviors that lead to this execution, including emotions and motivation that drive behaviors, cognition that organizes and plans the general strategy, motor planning, and finally, the execution of that plan. The components of the frontal cortex that mediate these behaviors, are reflected in the organization, physiology, and connections between areas of frontal cortex and in their projections through basal ganglia circuits. This comprises a series of parallel pathways. However, this model does not address how information flows between circuits thereby developing new learned behaviors (or actions) from a combination of inputs from emotional, cognitive, and motor cortical areas. Recent anatomical evidence from primates demonstrates that the neuro-networks within basal ganglia pathways are in a position to move information across functional circuits. Two networks are: the striato-nigral-striatal network and the thalamo-cortical-thalamic network. Within each of these sets of connected structures, there are both reciprocal connections linking up regions associated with similar functions and non-reciprocal connections linking up regions that are associated with different cortical basal ganglia circuits. Each component of information (from limbic to motor outcome) sends both feedback connection, and also a feedforward connection, allowing the transfer of information. Information is channeled from limbic, to cognitive, to motor circuits. Action decision-making processes are thus influenced by motivation and cognitive inputs, allowing the animal to respond appropriate to environmental cues.

Neurons in hippocampal afferent zones of rat striatum parse routes into multi-pace segments during maze navigation

Hippocampal 'place' neurons discharge when rats occupy specific regions within an environment. This finding is a cornerstone of the theory of the hippocampus as a cognitive map of space. But for navigation, representations of current position must be implemented by signals concerning where to go next, and how to get there. In recordings in hippocampal output structures associated with the motor system (nucleus accumbens and ventromedial caudate nucleus) in rats solving a plus-maze, neurons fired continuously from the moment the rat left one location until it arrived at the next goal site, or at an intermediate place, such as the maze centre. While other studies have shown discharges during reward approach behaviours, this is the first demonstration of activity corresponding to the parsing of complex routes into sequences of movements between landmarks, similar to the lists of instructions we often employ to communicate directions to follow between points on a map. As these cells fired during a series of several paces or re-orientation movements, perhaps this is homologous to 'chunking'. The temporal overlaps in the activity profiles of the individual neurons provide a possible substrate to successively trigger movements required to arrive at the goal. These hippocampally informed, and in some cases, spatially selective responses support the view of the ventral striatum as an interface between limbic and motor systems, permitting contextual representations to have an impact on fundamental action sequences for goal-directed behaviour.

The role of the medial prefrontal cortex in achieving goals

Achieving goals in changing environments requires the course of action to be selected on the basis of goal expectation and memory of action-outcome contingency. It is often also essential to evaluate action on the basis of immediate outcomes and the discrimination of early action steps from the final step towards the goal. Recently, in single-cell recordings in monkeys, the neuronal activity that appears to underlie these processes has been noted in the medial part of the prefrontal cortex. Medial prefrontal cells were also active when the subjects extracted the rules of a task in a novel environment. The processes described above might play important roles in rule learning.

Cue-Evoked Firing of Nucleus Accumbens Neurons Encodes Motivational Significance During a Discriminative Stimulus Task

The nucleus accumbens (NAc) has long been thought of as a limbic-motor interface. Despite behavioral and anatomical evidence in favor of this idea, little is known about how NAc neurons encode information about motivationally relevant environmental cues and use this information to affect motor action. We therefore investigated the firing of these neurons during the performance of a discriminative stimulus (DS) task using simultaneous multiple single-unit recordings in rats. In this task, two stimuli are randomly presented to the animal: a DS, which signals the availability of a sucrose reward contingent on an operant response, and a similar but nonrewarded stimulus (NS). Subpopulations of NAc neurons increased or decreased their firing in association with several distinct components of the task. In this paper, we investigate cue- and operant-responsive neurons. Neurons excited and inhibited by cues showed larger firing changes in response to the DS than the NS and larger changes when the animal made an operant response to the cue than when the animal failed to respond. Excitations during operant responding were not modulated by the information contained by the cue, whereas inhibitions during operant responding were somewhat larger if the operant response occurred during the DS and somewhat smaller if they occurred in the absence of a cue. These results are consistent with the hypothesis that the firing of subpopulations of NAc neurons encode both the predictive value of environmental stimuli and the specific motor behaviors required to respond to them.

Accumbal Neural Responses During the Initiation and Maintenance of Intravenous Cocaine Self-Administration

During a chronic extracellular recording session, animals with a history of cocaine self-administration were allowed to initiate drug seeking under drug-free conditions. Later, in the same recording session, animals engaged in intravenous cocaine self-administration. During the drug-free period, 31% of 70 accumbal neurons showed a significant increase in average firing rate in association with either or both the exposure to cues that signaled the onset of cocaine availability and the subsequent onset of drug-seeking behavior. The neurons that showed an average excitatory response during the drug-free period were the only group of neurons that showed an average excitatory phasic response to cocaine-reinforced lever presses during the drug self-administration session. A majority of the neurons that were activated during the drug-free period, like the majority of other neurons, showed decreases in average firing in response to self-administered cocaine. However, the neurons that were activated during the drug-free period maintained a higher rate of firing throughout the self-administration session than did other accumbal neurons. The data of the present study are consistent with the conclusion that accumbal neurons contribute to, or otherwise process, initiation of drug seeking under drug-free conditions and that they do so via primarily excitatory responses. Furthermore, there is continuity between the drug-free and -exposed conditions in neural responses associated with drug seeking. Finally, the data have potential implications for understanding mechanisms that transduce accumbal-mediated drug effects that contribute to drug addiction.

Reward, memory and substance abuse: functional neuronal circuits in the nucleus accumbens

The firing patterns of neurons in the nucleus accumbens (NA) are examined and discussed with respect to different types of rewards and reward conditions. Comparisons and contrasts between individually identified NA neuron responses to cocaine self-administration and water reinforcement are presented with an emphasis on the fact that the same neurons do not respond in a phasic manner to both types of rewards. However, the phasic firing patterns, even though segregated for each reinforcer, are quite similar, suggesting that the method of differentiation between rewarding stimuli in the NA is by sorting cell populations into distinct ensembles or networks for each type of reinforcer. These neural networks appear to be 'tuned' to respond to particular associative behavioral contexts that couple response execution to reward delivery, and in the process acquire a reciprocity to firing within reward contexts. This maintains the specificity of each reinforcer for the response and associated stimuli that produce it and, makes it possible to attach different NA networks to different reinforcing circumstances. Comparisons of cocaine and water reinforced NA cell firing patterns during rapid switching between these two reinforcers suggests that the networks are negatively coupled and mutually inhibit each other to maintain accurate encoding of immediately experienced, as well as expected (i.e. future) reward contingencies.

Selective encoding of cocaine versus natural rewards by nucleus accumbens neurons is not related to chronic drug exposure

We reported previously that subsets of nucleus accumbens (Acb) neurons differentially encode information about goal-directed behaviors for "natural" (food and water) versus cocaine reward in animals well trained to self-administer the drug (Carelli et al., 2000). Here, we examined whether repeated exposure to cocaine is the crucial determinate of the selective encoding of cocaine versus water reinforcement by Acb neurons. Acb cells were recorded during a water-cocaine multiple schedule from the first day of cocaine exposure as well as during repeated sessions. Specifically, animals were initially trained to press a lever for water and were then surgically prepared for extracellular recording in the Acb. After 1 week, Acb cells were recorded during acquisition of the water-cocaine multiple schedule. Because behavioral responding for water was already established, training on the multiple schedule was divided into three components corresponding to acquisition of self-administration: (1) "initial" (day 1 of self-administration), (2) "reliable" (self-administration behavior was present but erratic), and (3) "stable" (cocaine responding was stable). During the initial component, the percentage of water-selective neurons was high compared with cocaine neurons. However, this became approximately equal with repeated self-administration experience (i.e., during the stable component). Remarkably, the percentage of neurons showing overlapping (similar) neuronal firing patterns during initial exposure to cocaine was low (<8%) and remained low during reliable and stable components. These findings support the view that separate neural circuits in the Acb differentially encode information about cocaine versus natural reward, and that this functional organization is not a direct consequence of chronic drug exposure.

Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus

The prospect of immediate reward elicits goal-oriented behavior. However, animals often have to perform actions that do not immediately lead to reward in the pursuit of a long-term goal. Here we identify neural activity in monkey caudate nucleus that specifically correlates with rewarded and unrewarded eye movements. The monkey performed a visually guided saccade task in which only one position was associated with positive reinforcement. To advance in the experimental session, however, the monkey had no choice but to complete a saccade to an unrewarded position as well as to a rewarded position. Some caudate saccadic neurons showed enhanced activity around the time of the saccade in rewarded trials (rewarded-saccade neurons). Another subset of neurons discharged selectively around the execution of the saccade in unrewarded trials (unrewarded-saccade neurons). In both rewarded and unrewarded trials, stronger activity of these neurons was associated with reduced saccade latency. These results suggest that both rewarded and unrewarded saccades are facilitated by caudate saccadic activity. The neuronal activity of unrewarded-saccade neurons might reflect the required execution of unrewarded eye movements on the way to future reward.

Firing of nucleus accumbens neurons during the consummatory phase of a discriminative stimulus task depends on previous reward predictive cues

The nucleus accumbens (NAc) plays an important role in both appetitive and consummatory behavior. To examine how NAc neurons encode information during reward consumption, we recorded the firing activity of rat NAc neurons during the performance of a discriminative stimulus task. In this task, the animal must make an operant response to an intermittently presented cue to obtain a sucrose reward delivered in a reward receptacle. Uncued entries to the receptacle were not rewarded. Both excitations and inhibitions during reward consumption were observed, but substantially more neurons were inhibited than excited. These excitations and inhibitions began when the animal entered the reward receptacle and ended when the animal exited the receptacle. Both excitations and inhibitions were much smaller or nonexistent when the animal made uncued entries into the reward receptacle. In one set of experiments, we randomly withheld the reward in some cued trials that would otherwise have been rewarded. Excitations and inhibitions were of similar magnitude whether or not the reward was delivered. This indicates that the sensory stimulus of reward does not drive these phasic responses; instead, the reward-associated responses may be driven by the conditioned stimuli associated with reward, or they may encode information about consummatory motor activity. Another population of NAc neurons was excited on exit from the reward receptacle. Many of these excitations persisted for tens of seconds after the receptacle exit and showed a significant inverse correlation with the rate of uncued operant responding. These findings are consistent with a contribution of NAc neurons to both reward consummatory and reward seeking behavior.

Responses of tonically active neurons in the monkey striatum discriminate between motivationally opposing stimuli

The striatum is involved in the control of appetitively motivated behavior. We found previously that tonically active neurons (TANs) in the monkey striatum show discriminative responses to different stimuli that are appetitive or aversive. However, these differential responses may reflect the sensory qualities of the stimulus rather than its motivational value. In the present study, we sought to define more precisely the relationship between the particular aspect of the response of TANs and the motivational value of stimuli. For this purpose, three monkeys were presented with two types of aversive stimuli (loud sound and air puff) and one appetitive stimulus (fruit juice). In most instances, the TAN responses to the loud sound and the air puff were similar, in terms of response pattern and duration, whereas responses to the liquid reward showed distinct features. Using classical appetitive conditioning, we reversed the motivational value of a stimulus so that a previously aversive stimulus was now associatively paired with a reward and found that this manipulation selectively modifies the expression of TAN responses to the stimulus. These data indicate that the characteristics of neuronal responses undergo modifications when the valence of the stimulus is changed from aversive to appetitive during associative learning, suggesting that TANs may contribute to a form of stimulus encoding that is dependent on motivational attributes. The adaptation of TAN responses such as observed in the present study likewise reflects a neuronal system that adjusts to the motivational information about environmental events.

Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex

In several regions of the macaque brain, neurons fire during delayed response tasks at a rate determined by the value of the reward expected at the end of the trial. The activity of these neurons might be related either to the internal representation of the appetitive value of the expected reward or to motivation-dependent variations in the monkey's level of motor preparation or motor output. According to the first interpretation, reward-related activity should be most prominent in areas affiliated with the limbic system. According to the second interpretation, it should be most prominent in areas affiliated with the motor system. To distinguish between these alternatives, we carried out single-neuron recording while monkeys performed a memory-guided saccade task in which a visual cue presented early in each trial indicated whether the reward would be large or small. Neuronal activity accompanying task performance was monitored in the dorsolateral prefrontal cortex (PFC), the frontal eye field (FEF), a transitional zone caudal to the frontal eye field (FEF/PM), premotor cortex (PM), the supplementary eye field (SEF), and the rostral part of the supplementary motor area (SMAr). The tendency for neuronal activity to increase after cues that predicted a large reward became progressively stronger in progressively more posterior areas both in the lateral sector of the frontal lobe (PFC < FEF < FEF/PM < PM) and in the medial sector (SEF < SMAr). The very strong reward-related activity of premotor neurons was presumably attributable to the monkey's motivation-dependent level of motor preparation or motor output. This finding points to the need to determine whether reward-related activity in other nonlimbic brain areas, including dorsolateral prefrontal cortex and the dorsal striatum, genuinely represents the value of the expected reward or, alternatively, is related to motivational modulation of motor signals.

Cooperative activation of dopamine D1 and D2 receptors increases spike firing of nucleus accumbens neurons via G-protein betagamma subunits

Dopamine in the nucleus accumbens modulates both motivational and addictive behaviors. Dopamine D1 and D2 receptors are generally considered to exert opposite effects at the cellular level, but many behavioral studies find an apparent cooperative effect of D1 and D2 receptors in the nucleus accumbens. Here, we show that a dopamine-induced enhancement of spike firing in nucleus accumbens neurons in brain slices required both D1 and D2 receptors. One intracellular mechanism that might underlie cooperativity of D1 and D2 receptors is activation of specific subtypes of adenylyl cyclases by G-protein betagamma subunits (Gbetagamma) released from the Gi/o-linked D2 receptor in combination with Galpha(s)-like subunits from the D1 receptor. In this regard, dopaminergic enhancement of spike firing was prevented by inhibitors of protein kinase A or Gbetagamma. Furthermore, intracellular perfusion with Gbetagamma enabled D1 receptor activation but not D2 receptor activation to enhance spike firing. Finally, our data suggest that these pathways may increase spike firing by inhibition of a slow A-type potassium current. These results provide evidence for a novel cellular mechanism through which cooperative action of D1 and D2 receptors in the nucleus accumbens could mediate dopamine-dependent behaviors.

Differential changes in signal and background firing of accumbal neurons during cocaine self-administration

Learning theories of drug addiction propose that the disorder is, at least in part, attributable to drug effects on accumbal mechanisms that are normally involved in reward-related learning. The neurophysiological mechanisms that might transduce such a drug effect on accumbal mechanisms have yet to be identified. Previous studies showed that a population of accumbal neurons exhibit phasic excitatory responses time locked to cocaine-reinforced lever presses during intravenous cocaine self-administration sessions (neurons referred to as lever-press neurons). Most of the same neurons, like the majority of accumbal neurons, also show a decrease in average firing rate during the drug self-administration session. Evidence indicates that the phasic firing patterns transmit information related to drug-reward-related events. On the other hand, the decreases in average firing reflect a primary pharmacological effect of self-administered cocaine. In the present study, we tested the hypothesis that the phasic firing associated with drug seeking (i.e., signal) is less sensitive than other accumbal firing (i.e., background) to the inhibitory effect of cocaine. During intravenous cocaine self-administration sessions, 45 of 68 neurons showed a decrease in average firing during the self-administration session relative to a predrug baseline period. Fourteen neurons showed both an inhibition in average firing and an excitatory phasic response. For these 14 neurons, signal either remained equal to the average predrug firing rate or exceeded the predrug firing rate during the self-administration session. For the same neurons, background firing generally fell below average predrug firing. The differential changes in signal and background were associated with an increase in the ratio of signal-to-background for the individual neurons. Moreover, the relatively unique resistance of signal to inhibition was associated with an increase in the ratio of signal firing of all lever-press neurons relative to the background firing of all recorded neurons. This type of differential inhibition in signal and background firing might be expected to increase the relative influence of the drug-reward-related signals on accumbal-related neural circuits and differentially influence susceptibility of drug- and non-drug-reward-related synaptic and neural responses to neuroplasticity. It thus represents a mechanism by which inhibitory effects of self-administered drug might amplify the accumbal contribution to behavior and learning and potentially contribute to drug addiction.

Effects of expectations for different reward magnitudes on neuronal activity in primate striatum

In behavioral science, it is well known that humans and nonhuman animals are highly sensitive to differences in reward magnitude when choosing an outcome from a set of alternatives. We know that a realm of behavioral reactions is altered when animals begin to expect different levels of reward outcome. Our present aim was to investigate how the expectation for different magnitudes of reward influences behavior-related neurophysiology in the anterior striatum. In a spatial delayed response task, different instruction pictures are presented to the monkey. Each image represents a different magnitude of juice. By reaching to the spatial location where an instruction picture was presented, animals could receive the particular liquid amount designated by the stimulus. Reliable preferences in reward choice trials and differences in anticipatory licks, performance errors, and reaction times indicated that animals differentially expected the various reward amounts predicted by the instruction cues. A total of 374 of 2,000 neurons in the anterior parts of the caudate nucleus, putamen, and ventral striatum showed five forms of task-related activation during the preparation or execution of movement and activations preceding or following the liquid drop delivery. Approximately one-half of these striatal neurons showed differing response levels dependent on the magnitude of liquid to be received. Results of a linear regression analysis showed that reward magnitude and single cell discharge rate were related in a subset of neurons by a monotonic positive or negative relationship. Overall, these data support the idea that the striatum utilizes expectancies that contain precise information concerning the predicted, forthcoming level of reward in directing general behavioral reactions.

Reward value invariant place responses and reward site associated activity in hippocampal neurons of behaving rats

To investigate the involvement of the hippocampal-accumbens system in goal-oriented displacement behaviors, hippocampal neuronal activity was recorded in rats learning and recalling new distributions of different volumes of liquid reward among the arms of a plus maze. Each arm had a reward box containing a water trough and identical visual cues that could be illuminated independently. As the water-restricted rat successively visited the respective boxes, it received 7, 5, and 3 drops of water, and then 1 drop, provided at 1-s intervals. (Reward distributions were reassigned daily and mid-session.) In the training phase, reward boxes were lit individually. In the recall phase, the lamps on all arms were lit and then turned off as the rat visited the boxes in order of descending value. Neuronal firing rates were analyzed for changes related to reward value or to shifts between learning and recall phases. The principal finding is that place responses remained unchanged after these manipulations and that these neurons showed no evidence of explicit coding of reward value. In addition, two other types of responses appeared while the rat was stationary at the reward boxes awaiting multiple rewards. These were observed primarily in neurons within the dentate gyrus, but also in CA1. Position-selective reward site responses were regular at 20-60 impulses per second, while position-independent discharges bursted irregularly at about 5 impulses per second. Such responses could explain controversial reports of reward dependence in hippocampal neurons. The higher incidence of the latter responses in the temporal ("ventral") hippocampus is consistent with the distinctive anatomical and functional properties of this subregion.

Neural correlates of "hot" and "cold" emotional processing: a multilevel approach to the functional anatomy of emotion

The neural correlates of two hypothesized emotional processing modes, i.e., schematic and propositional modes, were investigated with positron emission tomography. Nineteen subjects performed an emotional mental imagery task while mentally repeating sentences linked to the meaning of the imagery script. In the schematic conditions, participants repeated metaphoric sentences, whereas in the propositional conditions, the sentences were explicit questions about specific emotional appraisals of the imagery scenario. Five types of emotional scripts were proposed to the subjects (happiness, anger, affection, sadness, and a neutral scenario). The results supported the hypothesized distinction between schematic and propositional emotional processing modes. Specifically, schematic mode was associated with increased activity in the ventromedial prefrontal cortex whereas propositional mode was associated with activation of the anterolateral prefrontal cortex. In addition, interaction analyses showed that schematic versus propositional processing of happiness (compared with the neutral scenario) was associated with increased activity in the ventral striatum whereas "schematic anger" was tentatively associated with activation of the ventral pallidum.

Dopamine gating of glutamatergic sensorimotor and incentive motivational input signals to the striatum

Dopamine (DA) neurons of the substantia nigra (SN) and ventral tegmental area (VTA) respond to a wide category of salient stimuli. Activation of SN and VTA DA neurons, and consequent release of nigrostriatal and mesolimbic DA, modulates the processing of concurrent glutamate inputs to dorsal and ventral striatal target regions. According to the view described here, this occurs under conditions of unexpected environmental change regardless of whether that change is rewarding or aversive. Nigrostriatal and mesolimbic DA activity gates the input of sensory, motor, and incentive motivational (e.g. reward) signals to the striatum. In light of recent single-unit and brain imaging data, it is suggested that the striatal reward signals originate in the orbitofrontal cortex and basolateral amygdala (BLA), regions that project strongly to the striatum. A DA signal of salient unexpected event occurrence, from this framework, gates the throughput of the orbitofrontal glutamate reward input to the striatum just as it gates the throughput of corticostriatal sensory and motor signals needed for normal response execution. Processing of these incoming signals is enhanced when synaptic DA levels are high, because DA enhances the synaptic efficacy of strong concurrent glutamate inputs while reducing the efficacy of weak glutamate inputs. The impairments in motor performance and incentive motivational processes that follow from nigrostriatal and mesolimbic DA loss can be understood in terms of a single mechanism: abnormal processing of sensorimotor and incentive motivation-related glutamate input signals to the striatum.

Getting formal with dopamine and reward

Recent neurophysiological studies reveal that neurons in certain brain structures carry specific signals about past and future rewards. Dopamine neurons display a short-latency, phasic reward signal indicating the difference between actual and predicted rewards. The signal is useful for enhancing neuronal processing and learning behavioral reactions. It is distinctly different from dopamine's tonic enabling of numerous behavioral processes. Neurons in the striatum, frontal cortex, and amygdala also process reward information but provide more differentiated information for identifying and anticipating rewards and organizing goal-directed behavior. The different reward signals have complementary functions, and the optimal use of rewards in voluntary behavior would benefit from interactions between the signals. Addictive psychostimulant drugs may exert their action by amplifying the dopamine reward signal.

Visual and anticipatory bias in three cortical eye fields of the monkey during an adaptive decision-making task

To examine the role of three cortical eye fields during internally guided decision-making processes, we recorded neuronal activities in the frontal eye field (FEF), supplementary eye field (SEF), and lateral intraparietal cortex (LIP) using a free-choice delayed saccade task with two synchronized targets. Although the monkeys must perform the task in a time-locked manner, they were free to choose either the receptive field (RF) target or the nonreceptive field (nRF) target to receive reward. In all three areas we found neurons with stronger activation during trials when the monkey was going to make a saccade to the RF target (RF trials) than to the nRF target (nRF trials). Modulation occurred not only during target presentation (visual bias) but also before target presentation (anticipatory bias). The visual bias was evident as an attenuated visual response to the RF stimulus in nRF trials. The anticipatory bias, however, was seen as an enhancement of pretarget activity in the RF trials. We analyzed the activity during the 500 msec before target presentation and found that 22.5% of FEF and 31.3% of LIP neurons and 49.1% of SEF neurons showed higher activity during the RF trials. To more accurately determine when each neuron started to show preferential activity, we used a new inverse interspike interval analysis procedure. Our results suggest that although all three cortical eye fields reflect attentional and intentional aspects of sensorimotor processing, SEF plays an earlier and perhaps more cognitive role in internally guided decision-making processes for saccades.

Nucleus accumbens cell firing during goal-directed behaviors for cocaine vs. 'natural' reinforcement

Numerous investigations indicate that the nucleus accumbens (Acb) is an important neural substrate mediating the reinforcing properties of 'natural' rewards (food or water) as well as abused substances. Here, our electrophysiological studies that examined Acb cell firing within seconds of lever press responding for intravenous cocaine vs. water or food reinforcement in rats are reviewed. Initial investigations revealed that a subset of Acb neurons exhibits four types of firing patterns within seconds of the reinforced response for intravenous cocaine during self-administration sessions. Three of those four cell types were also observed during water reinforcement sessions. In a subsequent study, the activity of the same Acb neurons was examined in rats responding on multiple schedules for either two distinct 'natural' reinforcers (water and food), or one of those 'natural' reinforcers and the intravenous self-administration of cocaine. The results showed that the majority of neurons tested exhibited similar, overlapping neuronal firing patterns across the two 'natural' reinforcer conditions. In contrast, the majority of neurons examined displayed differential, nonoverlapping firing patterns relative to operant responding for water (or food) vs. cocaine reinforcement. Additional studies that examined the role of associative factors on Acb cell firing during cocaine self-administration sessions are reviewed. Collectively, these findings illustrate the dynamic nature of Acb cell firing in behaving animals, and provide insight into how Acb neurons process information about goal-directed behaviors for 'natural' reinforcers vs. abused substances.

Anterior cingulate: single neuronal signals related to degree of reward expectancy

As monkeys perform schedules containing several trials with a visual cue indicating reward proximity, their error rates decrease as the number of remaining trials decreases, suggesting that their motivation and/or reward expectancy increases as the reward approaches. About one-third of single neurons recorded in the anterior cingulate cortex of monkeys during these reward schedules had responses that progressively changed strength with reward expectancy, an effect that disappeared when the cue was random. Alterations of this progression could be the basis for the changes from normal that are reported in anterior cingulate population activity for obsessive-compulsive disorder and drug abuse, conditions characterized by disturbances in reward expectancy.

An examination of nucleus accumbens cell firing during extinction and reinstatement of water reinforcement behavior in rats

Electrophysiological recording procedures were used to examine nucleus accumbens (Acb) cell firing in rats (n = 13) during water reinforcement sessions consisting of three phases. During phase one (maintenance), a lever press resulted in water reinforcement (fixed ratio 1; 0.05 ml/press) paired with an auditory stimulus (0.5 s). Of 128 Acb neurons recorded during maintenance, 40 cells (31%) exhibited one of three types of neuronal firing patterns described previously [J. Neurosci. 14 (12) (1994) 7735-7746; J. Neurosci. 20 (11) (2000) 4255-4266]. Briefly, Acb neurons exhibited increases in firing rate within seconds preceding the reinforced response (type PR) or increases (type RFe) or decreases (type RFi) in activity seconds following response completion. In phase two (extinction), subsequent lever pressing had no programmed consequences (i.e., water reinforcement and the auditory stimulus were not presented). After 30 min of no responding, animals were given water reinforcement/auditory stimulus 'primes' to reestablish lever pressing behavior during the third phase (reinstatement). Results indicated that all types of phasic neurons (PR, RFe and RFi) exhibited an attenuated firing rate during extinction, and in some cases recovery of patterned discharges were observed during reinstatement. No significant changes in cell firing were observed for any cell type during presentation of the stimulus prime used to reestablish operant responding following extinction. These findings are discussed in terms of how Acb neurons process information related to 'natural' reinforcers versus drugs of abuse.

Position sensitivity in phasically discharging nucleus accumbens neurons of rats alternating between tasks requiring complementary types of spatial cues

To determine how hippocampal location-selective discharges might influence downstream structures for navigation, nucleus accumbens neurons were recorded in rats alternating between two tasks guided respectively by lit cues in the maze or by extramaze room cues. Of 144 phasically active neurons, 80 showed significant behavioral correlates including displacements, immobility prior to, or after reward delivery, as well as turning, similar to previous reports. Nine neurons were position-selective, 22 were sensitive to task and platform changes and 40 others were both. Although the accumbens neurons showed the same behavioral correlate in two or four functionally equivalent locations, these responses were stronger at some of these places, evidence for position sensitivity. To test whether position responses were selective for room versus platform cues, the experimental platform was rotated while the rat performed each of the two tasks. This revealed responses to changes in position relative to both platform and room cues, despite the fact that previous studies had shown that place responses of hippocampal neurons recorded in the same task are anchored to room cues only. After these manipulations and shifts between the two tasks, the responses varied among simultaneously recorded neurons, and even in single neurons in alternating visits to reward sites. Again this contrasts with the uniformity of place responses of hippocampal neurons recorded in this same task. Thus accumbens position responses may derive from hippocampal inputs, while responses to context changes are more likely to derive from other signals or intrinsic processing. Considering the accumbens as a limbic-motor interface, we conclude that position-modulated behavioral responses in the accumbens may be intermediate between the allocentric reference frame of position-selective discharges in the hippocampus and the egocentric coding required to organize movement control. The conflicting responses among simultaneously recorded neurons could reflect competition processes serving as substrates for action selection and learning.

Feature-based anticipation of cues that predict reward in monkey caudate nucleus

A subset of caudate neurons fires before cues that instruct the monkey what he should do. To test the hypothesis that the anticipatory activity of such neurons depends on the context of stimulus-reward mapping, we examined their activity while the monkeys performed a memory-guided saccade task in which either the position or the color of a cue indicated presence or absence of reward. Some neurons showed anticipatory activity only when a particular position was associated with reward, while others fired selectively for color-reward associations. The functional segregation suggests that caudate neurons participate in feature-based anticipation of visual information that predicts reward. This neuronal code influences the general activity level in response to visual features without improving the quality of visual discrimination.

Reward signaling by dopamine neurons

Dopamine projections from the midbrain to the striatum and frontal cortex are involved in behavioral reactions controlled by rewards, as inferred from deficits in parkinsonism, schizophrenia, and drug addiction. Recent experiments have shown that dopamine neurons are not directly modulated in relation to movements. Rather, they appear to code the rewarding aspects of environmental stimuli. They show short, phasic increases of activity following primary food and liquid rewards ("unconditioned stimuli") and conditioned, reward-predicting stimuli of visual, auditory, and somatosensory modalities. They also display smaller activation-depression sequences after stimuli resembling rewards and after novel or particularly intense stimuli. Rewards are only reported as far as they occur differently than predicted. According to learning theories, a "prediction error" message may constitute a powerful teaching signal for behavior and learning. The phasic reward message is different from the more tonic enabling function of dopamine that is deficient in Parkinson's disease, indicating that dopamine neurons subserve different functions at different time scales. Neurons in other brain structures, such as the striatum, orbitofrontal cortex, and amygdala, code the quality, quantity, and preference of rewards. The dopamine reward prediction error signal may cooperate with these reward perception signals during the learning and performance of behavioral reactions to motivating environmental stimuli.

Selective activation of accumbens neurons by cocaine-associated stimuli during a water/cocaine multiple schedule

Electrophysiological recording procedures were used to examine the responsiveness of nucleus accumbens (Acb) neurons to stimuli associated with cocaine delivery during a multiple schedule for water reinforcement and cocaine self-administration. Rats (n=9) were trained to press one lever for water reinforcement (0.05 ml/resp.; fixed ratio 1; FR1; 15 min) and a second spatially distinct lever for intravenous cocaine (0.33 mg/infusion; FR1; 2 h). The delivery of each reinforcer was signaled by different auditory stimuli. Of 101 neurons, 52 cells were classified as phasically active, exhibiting one of four well-defined types of patterned discharges relative to the water- or cocaine-reinforced response [J. Neurosci., 14(12) (1994) 7735; J. Neurosci., 20(11) (2000) 4255]. Acb cells were examined in test sessions consisting of 'probe' trials during which the stimulus previously paired with cocaine infusion was randomly presented in a response-independent manner during the self-administration portion of the session. Results showed that only neurons that exhibited changes in firing rate within seconds following the reinforced response for cocaine (but not water) were activated by the stimulus. This finding indicates that the responsiveness of cocaine selective neurons to cocaine-associated stimuli likely represents a conditioned response as opposed to a generalized stimulus-evoked discharge.

A subset of cingulate cortical neurones is specifically activated during alcohol-acquisition behaviour

A new need is associated with the formation of behaviour directed at its satisfaction. In chronically ethanol-treated rabbits a bodily need develops to acquire and consume alcohol. The present study examined the firing properties of single neurones in the cingulate (limbic) cortex of chronically ethanol-treated rabbits. The main questions of this study were: are there neurones in the cingulate cortex which specifically increase their firing during alcohol-acquisition behaviour (AAB)? What is the relationship between the neuronal mechanisms of pre-existing and newly formed behaviour? Adult rabbits were taught to acquire food by pressing pedals. After 9 months of ethanol treatment, the same rabbits were taught to acquire ethanol (15% solution in a 0.5-mL capsule) by means of the same instrumental

METHOD

Activity of the 118 neurones was recorded from the cingulate cortex. The comparison of activity of each neurone in AAB and food-acquisition behaviour (FAB) enabled us to reveal that their subservings overleap substantially but not completely: 41% of 'common neurones' involved in the subserving of both FAB and AAB as well as 5% of 'alcohol-neurones' (alcohol-acquisition specific cells) were found. We think of the latter neurones as units that were specialized during the forming of alcohol-seeking behaviour. Thus, present experiments help us not only to answer the above questions but also to provide an additional insight into the nature of similarity between neuronal mechanisms of long-term memory and long-lived modifications resulting from repeated drug exposure.

Influence of expectation of different rewards on behavior-related neuronal activity in the striatum

This study investigated how different expected rewards influence behavior-related neuronal activity in the anterior striatum. In a spatial delayed-response task, monkeys reached for a left or right target and obtained a small quantity of one of two juices (apple, grenadine, orange, lemon, black currant, or raspberry). In each trial, an initial instruction picture indicated the behavioral target and predicted the reward. Nonmovement trials served as controls for movement relationships. Consistent preferences in special reward choice trials and differences in anticipatory licks, performance errors, and reaction times indicated that animals differentially expected the rewards predicted by the instructions. About 600 of >2,500 neurons in anterior parts of caudate nucleus, putamen, and ventral striatum showed five forms of task-related activations, comprising responses to instructions, spatial or nonspatial activations during the preparation or execution of the movement, and activations preceding or following the rewards. About one-third of the neurons showed different levels of task-related activity depending on which liquid reward was predicted at trial end. Activations were either higher or lower for rewards that were preferred by the animals as compared with nonpreferred rewards. These data suggest that the expectation of an upcoming liquid reward may influence a fraction of task-related neurons in the anterior striatum. Apparently the information about the expected reward is incorporated into the neuronal activity related to the behavioral reaction leading to the reward. The results of this study are in general agreement with an account of goal-directed behavior according to which the outcome should be represented already at the time at which the behavior toward the outcome is performed.

Position and behavioral modulation of synchronization of hippocampal and accumbens neuronal discharges in freely moving rats

To understand how hippocampal signals are processed by downstream neurons, we analyzed the relative timing between neuronal discharges in simultaneous recordings in the hippocampus and nucleus accumbens of rats performing in a plus maze. In all, 154 pairs of cells (composed of 65 hippocampal and 56 accumbens neurons) were examined during the 1 s period prior to reward delivery. Cross-correlation analyses over a +/- 300-ms window with 10-ms bins revealed that 108 pairs had at least one significant histogram bin (P < 0.01). The most frequently occurring peaks of hippocampal firing prior to accumbens discharges appeared at latencies from -30-0 ms, corresponding to published values of the latency of the hippocampal pathway to the nucleus accumbens. Other peaks appeared most often at latencies multiples of about 110 ms prior to and after this, corresponding to theta rhythmicity. Since firing synchronization can result from several types of connectivity patterns (such as common inputs), a group of 18 hippocampus-accumbens pairs was selected as those most likely to have monosynaptic connections. The criterion was the presence of at least one highly significant peak (P < 0.001) at latencies corresponding to field potentials evoked in the accumbens by hippocampal stimulation. A significant peak occurred on all four maze arms for only one of these cell pairs, indicating positional modulation for the others. In addition, behavior dependence of the synchrony between these nucleus accumbens and hippocampus neurons was examined by studying data in relation to three different synchronization points: reward box arrival, box departure, and arrival at the center of the maze. This indicates that the functional connectivity between hippocampal and accumbens neurons was stronger when the rat was near reward areas. Ten of the hippocampal neurons in these 18 cell pairs showed 9-Hz (theta) rhythmic activity in autocorrelation analyses. Of these 10 cells, cross-correlograms from eight hippocampal-accumbens pairs also showed theta rhythmicity. Overall, these results indicate that the synchrony between hippocampus and nucleus accumbens neurons is modulated by spatial position and behavior, and theta rhythm may play an important role for this synchronization.

Involvement of basal ganglia and orbitofrontal cortex in goal-directed behavior

An impressive array of neural processing appears to be dedicated to the extraction of reward-related information from environmental stimuli and use of this information in the generation of goal-directed behaviors. While other structures are certainly involved in these processes, the characteristics of activations seen in mesencephalic dopamine neurons, striatal neurons and neurons of the orbitofrontal cortex provide distinct examples of the different ways in which reward-related information is processed. In addition, the differences in activations seen in these three regions demonstrate the different roles they may play in goal-directed behavior. A principal role played by dopamine neurons is that of a detector of an error in reward prediction. The homogeneity of responsiveness across the population of dopamine neurons indicates that this error signal is widely broadcast to dopamine terminal regions where it could provide a teaching signal for synaptic modifications underlying the learning of goal-directed appetitive behaviors. The responses of these same neurons to conditioned stimuli associated with reward could also serve as a signal of prediction error useful for the learning of sequences of environmental stimuli leading to reward. Dopamine neuron responses to both rewards and conditioned stimuli are not contingent on the behavior executed to obtain the reward and thus appear to reflect a relatively pure signal of a reward prediction error. It is not yet clear whether these activations, and responses to novel stimuli, have an additional function in engaging neural systems involved in the representation and execution of goal-directed behaviors. This representation of goal-directed behaviors may involve the striatal regions studied, where processing of reward-related information appears to be much more heterogeneous. Different subpopulations of striatal neurons are activated at different stages in the course of goal-directed behaviors, with largely separate populations activated following presentation of conditioned stimuli, preceding reinforcers, and following reinforcers. Neurons exhibiting each of these types of activation appear to differentiate between rewarding and non-rewarding outcomes of behavioral acts and, as a population, appear to be biased towards processing reward vs. non-reward. These activations observed in the striatum were often contingent on the behavioral act associated with obtaining reward, reflecting an integration of information not observed in dopamine neurons. Another difference between reward processing in striatal neurons and dopamine neurons is the influence of predictability on neuronal responsiveness. Unlike dopamine neurons, many striatal neurons respond to predicted rewards, although at least some may reflect the relative degree of predictability in the magnitude of the responses to reward. Thus, striatal processing of reward-related information is in some ways more complex than that observed in dopamine neurons, incorporating information on behavior and potentially providing more detailed information regarding predictability. These activations could serve as a component of the neural representation of the goal, and/or the behavioral aspects of goal-directed behaviors. As such they would be of use for the execution of appropriate goal-directed behaviors in response to known environmental stimuli, as well as for generating behaviors in response to novel stimuli that may be associated with desirable goals. Neuronal activations in the orbitofrontal cortex appear to involve less integration of behavioral and reward-related information, but rather incorporate another aspect of reward, the relative motivational significance of different rewards. These activations would serve a function similar to those striatal neurons that encode exclusively reward-related information in situations in which only a single outcome is obtainable. (ABSTRACT TRUNCATED)

Multiple reward signals in the brain

The fundamental biological importance of rewards has created an increasing interest in the neuronal processing of reward information. The suggestion that the mechanisms underlying drug addiction might involve natural reward systems has also stimulated interest. This article focuses on recent neurophysiological studies in primates that have revealed that neurons in a limited number of brain structures carry specific signals about past and future rewards. This research provides the first step towards an understanding of how rewards influence behaviour before they are received and how the brain might use reward information to control learning and goal-directed behaviour.

Learning motivational significance of visual cues for reward schedules requires rhinal cortex

The limbic system is necessary to associate stimuli with their motivational and emotional significance. The perirhinal cortex is directly connected to this system, and neurons in this region carry signals related to a monkey's progress through visually cued reward schedules. This task manipulates motivation by displaying different visual cues to indicate the amount of work remaining until reward delivery. We asked whether rhinal (that is, entorhinal and perirhinal) cortex is necessary to associate the visual cues with reward schedules. When faced with new visual cues in reward schedules, intact monkeys adjusted their motivation in the schedules, whereas monkeys with rhinal cortex removals failed to do so. Thus, the rhinal cortex is critical for forming associations between visual stimuli and their motivational significance.