Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards

Impact of size and delay on neural activity in the rat limbic corticostriatal system

A number of factors influence an animal’s economic decisions. Two most commonly studied are the magnitude of and delay to reward. To investigate how these factors are represented in the firing rates of single neurons, we devised a behavioral task that independently manipulated the expected delay to and size of reward. Rats perceived the differently delayed and sized rewards as having different values and were more motivated under short delay and big-reward conditions than under long delay and small reward conditions as measured by percent choice, accuracy, and reaction time. Since the creation of this task, we have recorded from several different brain areas including, orbitofrontal cortex, striatum, amygdala, substantia nigra pars reticulata, and midbrain dopamine neurons. Here, we review and compare those data with a substantial focus on those areas that have been shown to be critical for performance on classic time discounting procedures and provide a potential mechanism by which they might interact when animals are deciding between differently delayed rewards. We found that most brain areas in the cortico-limbic circuit encode both the magnitude and delay to reward delivery in one form or another, but only a few encode them together at the single neuron level.

Natural rewards, neuroplasticity, and non-drug addictions

There is a high degree of overlap between brain regions involved in processing natural rewards and drugs of abuse. "Non-drug" or "behavioral" addictions have become increasingly documented in the clinic, and pathologies include compulsive activities such as shopping, eating, exercising, sexual behavior, and gambling. Like drug addiction, non-drug addictions manifest in symptoms including craving, impaired control over the behavior, tolerance, withdrawal, and high rates of relapse. These alterations in behavior suggest that plasticity may be occurring in brain regions associated with drug addiction. In this review, I summarize data demonstrating that exposure to non-drug rewards can alter neural plasticity in regions of the brain that are affected by drugs of abuse. Research suggests that there are several similarities between neuroplasticity induced by natural and drug rewards and that, depending on the reward, repeated exposure to natural rewards might induce neuroplasticity that either promotes or counteracts addictive behavior.

Cortico-basal ganglia mechanisms for overcoming innate, habitual and motivational behaviors

Most of the human behaviors are executed automatically under familiar circumstances. These behaviors are prepotent in that they take precedence over any other potential alternatives. Yet, humans are also capable of engaging cognitive resources to inhibit such a prepotent behavior and replace it with an alternative controlled behavior in response to an unforeseen situation. This remarkable capability to switch behaviors in a short period of time is the hallmark of executive functions. In this article, we first argue that the prepotent automaticity could emerge at least in three different domains - innate, habitual and motivational. We then review neurophysiological findings on how the brain might realize its switching functions in each domain, primarily by focusing on the monkey oculomotor system as the experimental model. Emerging evidence now suggests that multiple neuronal populations in the shared cortico-basal ganglia network contribute to overriding prepotent eye movement, be its origin innate, habitual or motivational. This consideration suggests the general versatility of the cortico-basal ganglia network as the neural mechanism whereby humans and other animals keep themselves from becoming subservient to reflex, habit and motivational impulses.

Importance of the temporal structure of movement sequences on the ability of monkeys to use serial order information

The capacity to acquire motor skills through repeated practice of a sequence of movements underlies many everyday activities. Extensive research in humans has dealt with the importance of spatial and temporal factors on motor sequence learning, standing in contrast to the few studies available in animals, particularly in nonhuman primates. In the present experiments, we studied the effect of the serial order of stimuli and associated movements in macaque monkeys overtrained to make arm-reaching movements in response to spatially distinct visual targets. Under different conditions, the temporal structure of the motor sequence was varied by changing the duration of the interval between successive target stimuli or by adding a cue that reliably signaled the onset time of the forthcoming target stimulus. In each condition, the extent to which the monkeys are sensitive to the spatial regularities was assessed by comparing performance when stimulus locations follow a repeating sequence, as opposed to a random sequence. We observed no improvement in task performance on repeated sequence blocks, compared to random sequence blocks, when target stimuli are relatively distant from each other in time. On the other hand, the shortening of the time interval between successive target stimuli or, more efficiently, the addition of a temporal cue before the target stimulus yielded a performance advantage under repeated sequence, reflected in a decrease in the latency of arm and saccadic eye movements accompanied by an increased tendency for eye movements to occur in an anticipatory manner. Contrary to the effects on movement initiation, the serial order of stimuli and movements did not markedly affect the execution of movement. Moreover, the location of a given target in the random sequence influenced task performance based on the location of the preceding target, monkeys being faster in responding as a result of familiarity caused by extensive practice with some target transitions also used in the repeated sequence. This performance advantage was most prominently detectable when temporal prediction of forthcoming target stimuli was optimized. Taken together, the present findings demonstrate that the monkey's capacity to make use of serial order information to speed task performance was dependent on the temporal structure of the motor sequence.

Roles of nucleus accumbens core and shell in incentive-cue responding and behavioral inhibition

The nucleus accumbens (NAc) is involved in many reward-related behaviors. The NAc has two major components, the core and the shell. These two areas have different inputs and outputs, suggesting that they contribute differentially to goal-directed behaviors. Using a discriminative stimulus (DS) task in rats and inactivating the NAc by blocking excitatory inputs with glutamate antagonists, we dissociated core and shell contributions to task performance. NAc core but not shell inactivation decreased responding to a reward-predictive cue. In contrast, inactivation of either subregion induced a general behavioral disinhibition. This reveals that the NAc actively suppresses actions inappropriate to the DS task. Importantly, selective inactivation of the shell but not core significantly increased responding to the nonrewarded cue. To determine whether the different contributions of the NAc core and shell depend on the information encoded in their constituent neurons, we performed electrophysiological recording in rats performing the DS task. Although there was no firing pattern unique to either core or shell, the reward-predictive cue elicited more frequent and larger magnitude responses in the NAc core than in the shell. Conversely, more NAc shell neurons selectively responded to the nonrewarded stimulus. These quantitative differences might account for the different behavioral patterns that require either core or shell. Neurons with similar firing patterns could also have different effects on behavior due to their distinct projection targets.

Potential vulnerabilities of neuronal reward, risk, and decision mechanisms to addictive drugs

How do addictive drugs hijack the brain's reward system? This review speculates how normal, physiological reward processes may be affected by addictive drugs. Addictive drugs affect acute responses and plasticity in dopamine neurons and postsynaptic structures. These effects reduce reward discrimination, increase the effects of reward prediction error signals, and enhance neuronal responses to reward-predicting stimuli, which may contribute to compulsion. Addictive drugs steepen neuronal temporal reward discounting and create temporal myopia that impairs the control of drug taking. Tonically enhanced dopamine levels may disturb working memory mechanisms necessary for assessing background rewards and thus may generate inaccurate neuronal reward predictions. Drug-induced working memory deficits may impair neuronal risk signaling, promote risky behaviors, and facilitate preaddictive drug use. Malfunctioning adaptive reward coding may lead to overvaluation of drug rewards. Many of these malfunctions may result in inadequate neuronal decision mechanisms and lead to choices biased toward drug rewards.

Global and regional putamen volume loss in patients with heart failure

AIMS

Heart failure (HF) is accompanied by diminished cognitive, motor, learning, emotional, and planning deficits, which are associated with increased morbidity and mortality. A basal ganglia structure, the putamen, serves many functions that are affected in HF, but its global or localized structural integrity is unknown. Our aim was to evaluate global and regional putamen volume differences in HF over control subjects.

METHODS AND RESULTS

We collected two high-resolution T1-weighted scans from 16 HF patients (age, 54.1 ± 8.3 years; 12 males; left ventricular ejection fraction, 27.8 ± 6.8%) and 32 control subjects (52.4 ± 7.3 years; 24 males) using a 3.0 T magnetic resonance imaging scanner. After realigning, averaging, and reorienting the T1-weighted volumes into a common space, the structures were manually outlined, tracings were normalized for head size, volumes calculated, and surface models generated. Demographic data were compared between groups with χ(2) and independent samples t-tests, global putamen volumes were evaluated using independent samples t-tests, and regional differences were examined with surface morphometry. No significant differences in age or sex appeared between groups, but body mass index differed significantly (P = 0.008). Heart failure patients showed significantly lower left (controls vs. HF; 4842.1 ± 740.0 vs. 4224.1 ± 894.4 mm(3), P = 0.014) and right (4769.3 ± 651.9 vs. 4193.7 ± 876.2 mm(3), P = 0.014) global putamen volumes than controls, with localized reductions in bilateral rostral, mid-dorsal, and medial-caudal regions (left, P < 0.003; right, P < 0.0002).

CONCLUSION

Putamen structures showed global and localized volume reductions in HF over controls. The localized volume losses suggest deficits in motor and neuropsychological functions, which are evident in HF subjects, and may be due to hypoxic and ischaemic processes targeting these areas.

Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice

In choosing between different rewards expected after unequal delays, humans and animals often prefer the smaller but more immediate reward, indicating that the subjective value or utility of reward is depreciated according to its delay. Here, we show that neurons in the primate caudate nucleus and ventral striatum modulate their activity according to temporally discounted values of rewards with a similar time course. However, neurons in the caudate nucleus encoded the difference in the temporally discounted values of the two alternative targets more reliably than neurons in the ventral striatum. In contrast, neurons in the ventral striatum largely encoded the sum of the temporally discounted values, and therefore, the overall goodness of available options. These results suggest a more pivotal role for the dorsal striatum in action selection during intertemporal choice.

Representation of reward feedback in primate auditory cortex

It is well established that auditory cortex is plastic on different time scales and that this plasticity is driven by the reinforcement that is used to motivate subjects to learn or to perform an auditory task. Motivated by these findings, we study in detail properties of neuronal firing in auditory cortex that is related to reward feedback. We recorded from the auditory cortex of two monkeys while they were performing an auditory categorization task. Monkeys listened to a sequence of tones and had to signal when the frequency of adjacent tones stepped in downward direction, irrespective of the tone frequency and step size. Correct identifications were rewarded with either a large or a small amount of water. The size of reward depended on the monkeys' performance in the previous trial: it was large after a correct trial and small after an incorrect trial. The rewards served to maintain task performance. During task performance we found three successive periods of neuronal firing in auditory cortex that reflected (1) the reward expectancy for each trial, (2) the reward-size received, and (3) the mismatch between the expected and delivered reward. These results, together with control experiments suggest that auditory cortex receives reward feedback that could be used to adapt auditory cortex to task requirements. Additionally, the results presented here extend previous observations of non-auditory roles of auditory cortex and shows that auditory cortex is even more cognitively influenced than lately recognized.

Effects of cocaine rewards on neural representations of cognitive demand in nonhuman primates

RATIONALE

Investigations of the neural consequences of the effects of cocaine on cognition have centered on specific brain circuits including prefrontal cortex, medial temporal lobe and striatum and their roles in controlling drug dependent behavior and addiction. These regions are critical to many aspects of drug abuse; however recent investigations in addicted individuals have reported possible cognitive deficits that impact recovery and other therapeutic interventions.

OBJECTIVES

Therefore a direct assessment of the effects of cocaine as a reward for cognitive function provides a means of determining how brain systems involved such as prefrontal cortex are affected under normal vs. conditions of acute drug exposure as a precursor to the final impaired function in the addicted state.

METHODS

Nonhuman primates (NHPs) were tested in a delayed-match-to-sample decision making task to determine effects of high vs. low cognitive load trials on single neuron activity and fluorodeoxyglucose-positron emission tomography (FDG-PET) determined metabolic activation of prefrontal cortex when juice vs. intravenous cocaine were employed as rewards for successful performance.

RESULTS

Cognitive processing in prefrontal cortex was altered primarily on high load trials in which cocaine was randomly presented as the signaled and delivered reward on particular trials. The detrimental actions of cocaine rewards were also shown to persist and impair task performance on subsequent juice rewarded trials.

CONCLUSIONS

The findings indicate that one of the ways in which cocaine use may disrupt performance of a cognitive task is to alter neural processing in prefrontal cortex when involved in discriminating circumstances on the basis of low vs. high cognitive demand.

The reward circuit: linking primate anatomy and human imaging

Although cells in many brain regions respond to reward, the cortical-basal ganglia circuit is at the heart of the reward system. The key structures in this network are the anterior cingulate cortex, the orbital prefrontal cortex, the ventral striatum, the ventral pallidum, and the midbrain dopamine neurons. In addition, other structures, including the dorsal prefrontal cortex, amygdala, hippocampus, thalamus, and lateral habenular nucleus, and specific brainstem structures such as the pedunculopontine nucleus, and the raphe nucleus, are key components in regulating the reward circuit. Connectivity between these areas forms a complex neural network that mediates different aspects of reward processing. Advances in neuroimaging techniques allow better spatial and temporal resolution. These studies now demonstrate that human functional and structural imaging results map increasingly close to primate anatomy.

The plausibility of sugar addiction and its role in obesity and eating disorders

BACKGROUND & AIMS

To consider the hypothesis that addiction to food, or more specifically sucrose, plays a role in obesity and eating disorders.

METHODS

By considering the relevant literature a series of predictions were examined, derived from the hypothesis that addiction to sucrose consumption can develop. Fasting should increase food cravings, predominantly for sweet items; cravings should occur after an overnight fast; the obese should find sweetness particularly attractive; a high-sugar consumption should predispose to obesity. More specifically predictions based on the hypothesis that addiction to sugar is central to bingeing disorders were developed. Dieting should predate the development of bingeing; dietary style rather than psychological, social and economic factors should be predispose to eating disorders; sweet items should be preferentially consumed while bingeing; opioid antagonists should cause withdrawal symptoms; bingeing should develop at a younger age when there is a greater preference for sweetness.

RESULTS

The above predications have in common that on no occasion was the behaviour predicted by an animal model of sucrose addiction supported by human studies.

CONCLUSION

There is no support from the human literature for the hypothesis that sucrose may be physically addictive or that addiction to sugar plays a role in eating disorders.

Orbitofrontal and anterior cingulate cortex neurons selectively process cocaine-associated environmental cues in the rhesus monkey

Encounters with stimuli associated with drug use are believed to contribute to relapse. To probe the neurobiology of environmentally triggered drug use, we have conducted single-unit recordings in rhesus monkeys during presentation of two distinct types of drug paired cues that differentially support drug-seeking. The animals were highly conditioned to these cues via exposure during self-administration procedures conducted over a 4 year period. The cues studied were a discriminative cue that signaled response-contingent availability of cocaine, and a discrete cue that was temporally paired with the cocaine infusion (0.1 or 0.5 mg/kg). Two cortical regions consistently activated by cocaine-associated cues in human imaging studies are the orbitofrontal (OFC) and anterior cingulate cortex (ACC), though little is known about cortical neuronal activity responses to drug cues. We simultaneously recorded single-unit activity in OFC and ACC as well as in dorsal striatum in rhesus monkeys during cocaine self-administration. Dorsal striatal neurons were less engaged by drug cues than cortical regions. Between OFC and ACC, distinct functionality was apparent in neuronal responses. OFC neurons preferentially responded to the discriminative cue, consistent with a role in cue-induced drug-seeking. In contrast, the ACC did not respond more to the discriminative cue than to the discrete cue. Also distinct from the OFC, ACC showed sustained firing throughout the 18 s duration of the discrete cue. This pattern of sustained activation in ACC is consistent with a role in reward expectation and/or in mediating behavioral effects of discrete cues paired with drug infusions.

Beyond the Reward Pathway: Coding Reward Magnitude and Error in the Rat Subthalamic Nucleus

It was recently shown that subthalamic nucleus (STN) lesions affect motivation for food, cocaine, and alcohol, differentially, according to either the nature of the reward or the preference for it. The STN may thus code a reward according to its value. Here, we investigated how the firing of subthalamic neurons is modulated during expectation of a predicted reward between two possibilities (4 or 32% sucrose solution). The firing pattern of neurons responding to predictive cues and to reward delivery indicates that STN neurons can be divided into subpopulations responding specifically to one reward and less or giving no response to the other. In addition, some neurons (“oops” neurons) specifically encode errors as they respond only during error trials. These results reveal that the STN plays a critical role in ascertaining the value of the reward and seems to encode that value differently depending on the magnitude of the reward. These data highlight the importance of the STN in the reward circuitry of the brain.

The encoding of cocaine vs. natural rewards in the striatum of nonhuman primates: categories with different activations

The behavioral and motivational changes that result from use of abused substances depend upon activation of neuronal populations in the reward centers of the brain, located primarily in the corpus striatum in primates. To gain insight into the cellular mechanisms through which abused drugs reinforce behavior in the primate brain, changes in firing of neurons in the ventral (VStr, nucleus accumbens) and dorsal (DStr, caudate-putamen) striatum to "natural" (juice) vs. drug (i.v. cocaine) rewards were examined in four rhesus monkeys performing a visual Go-Nogo decision task. Task-related striatal neurons increased firing to one or more of the specific events that occurred within a trial represented by (1) Target stimuli (Go trials) or (2) Nogotarget stimuli (Nogo trials), and (3) Reward delivery for correct performance. These three cell populations were further subdivided into categories that reflected firing exclusively on one or the other type of signaled reward (juice or cocaine) trial (20%-30% of all cells), or, a second subpopulation that fired on both (cocaine and juice) types of rewarded trial (50%). Results show that neurons in the primate striatum encoded cocaine-rewarded trials similar to juice-rewarded trials, except for (1) increased firing on cocaine-rewarded trials, (2) prolonged activation during delivery of i.v. cocaine infusion, and (3) differential firing in ventral (VStr cells) vs. dorsal (DStr cells) striatum cocaine-rewarded trials. Reciprocal activations of antithetic subpopulations of cells during different temporal intervals within the same trial suggest a functional interaction between processes that encode drug and natural rewards in the primate brain.

Neuroelectric signatures of reward learning and decision-making in the human nucleus accumbens

Learning that certain actions lead to risky rewards is critical for biological, social, and economic survival, but the precise neural mechanisms of such reward-guided learning remain unclear. Here, we show that the human nucleus accumbens plays a key role in learning about risks by representing reward value. We recorded electrophysiological activity directly from the nucleus accumbens of five patients undergoing deep brain stimulation for treatment of refractory major depression. Patients engaged in a simple reward-learning task in which they first learned stimulus-outcome associations (learning task), and then were able to choose from among the learned stimuli (choosing task). During the learning task, nucleus accumbens activity reflected potential and received reward values both during the cue stimulus and during the feedback. During the choosing task, there was no nucleus accumbens activity during the cue stimulus, but feedback-related activity was pronounced and similar to that during the learning task. This pattern of results is inconsistent with a prediction error response. Finally, analyses of cross-correlations between the accumbens and simultaneous recordings of medial frontal cortex suggest a dynamic interaction between these structures. The high spatial and temporal resolution of these recordings provides novel insights into the timing of activity in the human nucleus accumbens, its functions during reward-guided learning and decision-making, and its interactions with medial frontal cortex.

Pentylenetetrazole-induced status epilepticus following training does not impair expression of morphine-induced conditioned place preference

Learning and memory play an important role in morphine addiction. Status epilepticus (SE) can impair the spatial and emotional learning and memory. However, little is known about the effects of SE on morphine-induced conditioned place preference (CPP). The present study was designed to investigate the effects of SE on morphine CPP, with food CPP being used as a control. The effects of SE on spatial memory in the Morris water maze (MWM) and Y-maze were investigated. SE was induced in adult mice using intraperitoneal injection of pentylenetetrazole; control mice received saline. The data indicated that SE had no effects on the formation of morphine CPP; however, the formation of food CPP was blocked by SE. Meanwhile, spatial memory assayed in the MWM and Y-maze was impaired by SE. In addition, the data demonstrated that SE did not cause a lasting disturbance of motor activity nor a change in the mice's appetite. These results suggested that although SE had no effects on morphine CPP, there was impaired food CPP and spatial memory in both the MWM and the Y-maze. The mechanisms underlying memory process of morphine CPP may be different from other types of memory.

Distinct subsets of nucleus accumbens neurons encode operant responding for ethanol versus water

Subsets of nucleus accumbens (NAc) neurons process information about operant responses for drugs as well as natural rewards (food and water) by excitations and inhibitions in firing rate time-locked to the operant response. The degree to which ensembles of neurons exhibit similar firing patterns when encoding cues and operant responses across different reinforcer conditions will provide critical information regarding the functional organization of this nucleus. The present experiment evaluated the relative contribution of subsets of accumbens neurons that encode distinct features of lever press responding for ethanol vs. water. Electrophysiological recordings (n = 153 neurons) were made in the accumbens of rats trained on concurrent reinforcement schedules for ethanol and water throughout a self-administration session. During operant responding, 52% of neurons exhibited patterned discharges characterized by significant increases or decreases in firing rate of +/- 1 s relative to lever presses for ethanol and/or water. Of these phasic cells, 85% discriminated between presses for ethanol and water (i.e. exhibited firing patterns unique to one reinforcer type), while 15% exhibited identical firing patterns relative to lever presses for both reinforcers. Notably, the data revealed that both high ethanol preference and spatially distinct lever positions contributed to the reinforcer specificity. Together, these data demonstrate that subsets of NAc neurons encode conditioned and instrumental aspects of ethanol vs. water reinforcement in well-trained rats, and that reinforcer preference and spatial cues are important components of this differential information processing.

Prospective and retrospective temporal difference learning

A striking recent finding is that monkeys behave maladaptively in a class of tasks in which they know that reward is going to be systematically delayed. This may be explained by a malign Pavlovian influence arising from states with low predicted values. However, by very carefully analyzing behavioral data from such tasks, La Camera and Richmond (2008) observed the additional important characteristic that subjects perform differently on states in the task that are at equal distances from the future reward, depending on what has happened in the recent past. The authors pointed out that this violates the definition of state value in the standard reinforcement learning models that are ubiquitous as accounts of operant and classical conditioned behavior; they suggested and analyzed an alternative temporal difference (TD) model in which past and future are melded. Here, we show that, in fact, a standard TD model can actually exhibit the same behavior, and that this avoids deleterious consequences for choice. At the heart of the model is the average reward per step, which acts as a baseline for measuring immediate rewards. Relatively subtle changes to this baseline occasioned by the past can markedly influence predictions and thus behavior.

Measuring and modeling the interaction among reward size, delay to reward, and satiation level on motivation in monkeys

Motivation is usually inferred from the likelihood or the intensity with which behavior is carried out. It is sensitive to external factors (e.g., the identity, amount, and timing of a rewarding outcome) and internal factors (e.g., hunger or thirst). We trained macaque monkeys to perform a nonchoice instrumental task (a sequential red-green color discrimination) while manipulating two external factors: reward size and delay-to-reward. We also inferred the state of one internal factor, level of satiation, by monitoring the accumulated reward. A visual cue indicated the forthcoming reward size and delay-to-reward in each trial. The fraction of trials completed correctly by the monkeys increased linearly with reward size and was hyperbolically discounted by delay-to-reward duration, relations that are similar to those found in free operant and choice tasks. The fraction of correct trials also decreased progressively as a function of the satiation level. Similar (albeit noiser) relations were obtained for reaction times. The combined effect of reward size, delay-to-reward, and satiation level on the proportion of correct trials is well described as a multiplication of the effects of the single factors when each factor is examined alone. These results provide a quantitative account of the interaction of external and internal factors on instrumental behavior, and allow us to extend the concept of subjective value of a rewarding outcome, usually confined to external factors, to account also for slow changes in the internal drive of the subject.

Relation of locus coeruleus neurons in monkeys to Pavlovian and operant behaviors

Noradrenaline is released throughout the forebrain from locus coeruleus (LC) projections in close temporal proximity to emotional and goal-directed events. To examine interactive influences of these processes on LC neuronal activity, we used a task where Pavlovian and operant processes vary and can be easily identified. We recorded 69 single LC neurons from two monkeys performing a task where cues indicate the progression through schedules of one, two, or three operant trials. Pavlovian responses and phasic LC activations occur following the appearance of conditioned visual cues (54/69 neurons), especially those at the beginning of new schedules, whether the current trial will be rewarded (single trial schedule) or not (2 or 3 trial schedules), and after visual imperative signals eliciting the operant response (64/69 neurons), whether the current trial will be rewarded or not. The modulation of LC responses seems to be relatively independent of attention or motivation, because the responses do not covary with operant performance in the task. The magnitude of LC responses across the schedules varied in close relation to the intensity of Pavlovian behavior but these responses were also modulated by operant processes. Our conclusion is that LC activation occurs when task-relevant stimuli evoke a conditioned instinctive (Pavlovian) response, with the strength of the activation also being modulated by goal-directed processes. Thus locus coeruleus neurons broadcast information about stimulus-elicited primitive and goal-directed behaviors to forebrain structures important for executive functions and emotions.

A neural correlate of motivational conflict in the superior colliculus of the macaque

Behavior is controlled by both external instructions and internal motives, but the actions demanded by each may be different. A common consequence of such a conflict is a delay in decision making and subsequent motor responses. It is unknown, however, what neural mechanisms underlie motivational conflict and associated response delay. To answer this question, we recorded single-neuron activity in the superior colliculus (SC) as macaque monkeys performed a visually guided, asymmetrically rewarded saccade task. A peripheral spot of light at one of two opposing positions was illuminated to indicate a saccade target. In a given block of trials, one position was associated with a big reward and the other with a small reward. The big-reward position was alternated across blocks. Behavioral analyses revealed that small-reward trials created a conflict between the instructed saccade to one position and the internally motivated, yet invalid saccade to the opposite position. We found that movement neurons in the SC temporally exhibited bursting activity after the appearance of the small-reward target opposite their movement field. This transient activity predicted the amount of response delay for upcoming saccades. Our data suggest that motivational conflict activates movement neurons in both colliculi, thereby delaying saccade initiation through intercollicular inhibitory interactions.

Modeling the violation of reward maximization and invariance in reinforcement schedules

It is often assumed that animals and people adjust their behavior to maximize reward acquisition. In visually cued reinforcement schedules, monkeys make errors in trials that are not immediately rewarded, despite having to repeat error trials. Here we show that error rates are typically smaller in trials equally distant from reward but belonging to longer schedules (referred to as "schedule length effect"). This violates the principles of reward maximization and invariance and cannot be predicted by the standard methods of Reinforcement Learning, such as the method of temporal differences. We develop a heuristic model that accounts for all of the properties of the behavior in the reinforcement schedule task but whose predictions are not different from those of the standard temporal difference model in choice tasks. In the modification of temporal difference learning introduced here, the effect of schedule length emerges spontaneously from the sensitivity to the immediately preceding trial. We also introduce a policy for general Markov Decision Processes, where the decision made at each node is conditioned on the motivation to perform an instrumental action, and show that the application of our model to the reinforcement schedule task and the choice task are special cases of this general theoretical framework. Within this framework, Reinforcement Learning can approach contextual learning with the mixture of empirical findings and principled assumptions that seem to coexist in the best descriptions of animal behavior. As examples, we discuss two phenomena observed in humans that often derive from the violation of the principle of invariance: "framing," wherein equivalent options are treated differently depending on the context in which they are presented, and the "sunk cost" effect, the greater tendency to continue an endeavor once an investment in money, effort, or time has been made. The schedule length effect might be a manifestation of these phenomena in monkeys.

Dissecting motivational circuitry to understand substance abuse

An important goal of cocaine addiction research is to understand the neurobiological mechanisms underlying this disease state. Here, we review studies from our laboratory that examined nucleus accumbens (NAc) cell firing and rapid dopamine signaling using electrophysiological and electrochemical recordings in behaving rodents. A major advantage of these techniques is that they allow for the characterization of NAc activity and rapid dopamine release during specific phases of motivated behavior. Moreover, each approach enables an examination of the dynamic nature of NAc signaling as a function of factors such as hedonics and associative learning. We show that NAc neurons differentially respond to rewarding and aversive stimuli and their predictors in a bivalent manner. This differential responding is modifiable and can be altered by the presentation of other natural rewards or cocaine. Likewise, the dynamic nature of NAc cell firing is also reflected in the differential activation of distinct populations of NAc neurons during goal-directed behaviors for natural versus drug rewards, and the heightened activation of some NAc neurons following cocaine abstinence. Our electrochemical data also show that rapid dopamine signaling in the NAc reflects primary rewards and their predictors and appears to modulate specific NAc neuronal responses. In some cases, these influences are observed in a regionally specific manner that matches previous pharmacological manipulations. Collectively, these findings provide critical insight into the functional organization of the NAc that can be used to guide additional studies aimed at dissecting the neural code underlying compulsive drug-seeking behavior.

Chronic social stress, hedonism and vulnerability to obesity: lessons from rodents

Obesity is a current health pandemia. Determinants of this pathology are rather complex and include genetic, developmental and environmental factors only partially disclosed. Stress related neuroendocrine dysregulation and overconsumption of high palatable high caloric food and are likely to contribute to this modern health threats. Despite the evidence that psychosocial stress is one of the main sources of stress in humans and may play an important role in the development of the stress disorders, including obesity and metabolic syndrome, animal models focusing on the relationship between chronic stress and energy homeostasis are scattered and most of them encompasses physical rather than psychosocial stress. Aim of the present paper is to review rodent studies on the effect of psychosocial stress throughout life on body weight and food intake regulation. In the second part of the review special focus will be given on the mechanisms linking stress and the reward system.

Activity of primate orbitofrontal and dorsolateral prefrontal neurons: effect of reward schedule on task-related activity

Recent studies show that task-related activity in the dorsolateral prefrontal cortex (DLPFC) is modulated by the quality and quantity of the reward, suggesting that the subject's motivational state affects cognitive operations in the DLPFC. The orbito-frontal cortex (OFC) is a possible source of motivational inputs to the DLPFC. However, it is not well known whether these two areas exhibit similar motivational effects on task-related activity. We compared motivational effects on task-related activity in these areas while a monkey performed an oculomotor delayed-response (ODR) task under two reward schedules. In the ODR-1 schedule, reward was given only after the successful completion of four consecutive trials, whereas in the ODR-2 schedule, reward was given after every correct trial. Task-related activities in both areas showed spatial selectivity. The spatial characteristics of task-related activity remained constant in both schedules. Task-related activity in both areas, especially delay-period activity, was also affected by the reward schedule and the magnitude of the activity gradually increased depending on the proximity of the reward trial in the ODR-1 schedule. More task-related OFC activities were affected by reward schedules, whereas more task-related DLPFC activities were affected by spatial factors and reward schedules. These results indicate that the OFC plays a role in monitoring the proximity of the reward trial and detecting reward delivery, whereas the DLPFC plays a role in performing cognitive operations and integrating cognitive and motivational information. These results also indicate that spatial information and the animal's motivational state independently affect neuronal activity in both areas.

Alcohol preference influences the subthalamic nucleus control on motivation for alcohol in rats

In addition to its role in motor and attentional processes, the subthalamic nucleus (STN) has also been recently demonstrated to be involved in motivational function. Indeed, bilateral STN lesions modulate differentially the motivation for natural rewards and drugs of abuse, increasing motivation for food and decreasing motivation for cocaine in rats. Here, we show that in outbred rats, the STN can modulate the motivation for alcohol according to alcohol preference, without affecting alcohol intake. When performed on 'High-Drinker' rats, STN lesions enhanced the breaking point (BP) under a progressive ratio schedule of reinforcement and increased the time spent in the environment previously paired with alcohol access in the place preference paradigm. In contrast, when performed on 'Low-Drinker' rats, STN lesions decreased the BP and increased the time spent in the environment paired with water. These results show that STN lesions enhance the motivation for alcohol in rats showing a high alcohol preference, whereas they decrease it in rats showing a low preference for alcohol. These results suggest that the STN plays a complex role in the reward circuit, that is not limited to a

Order-dependent modulation of directional signals in the supplementary and presupplementary motor areas

To maximize reward and minimize effort, animals must often execute multiple movements in a timely and orderly manner. Such movement sequences must be usually discovered through experience, and during this process, signals related to the animal's action, its ordinal position in the sequence, and subsequent reward need to be properly integrated. To investigate the role of the primate medial frontal cortex in planning and controlling multiple movements, monkeys were trained to produce a series of hand movements instructed by visual stimuli. We manipulated the number of movements in a sequence across trials, making it possible to dissociate the effects of the ordinal position of a given movement and the number of remaining movements necessary to obtain reward. Neurons in the supplementary and presupplementary motor areas modulated their activity according to the number of remaining movements, more often than in relation to the ordinal position, suggesting that they might encode signals related to the timing of reward or its temporally discounted value. In both cortical areas, signals related to the number of remaining movements and those related to movement direction were often combined multiplicatively, suggesting that the gain of the signals related to movements might be modulated by motivational factors. Finally, compared with the supplementary motor area, neurons in the presupplementary motor area were more likely to increase their activity when the number of remaining movements is large. These results suggest that these two areas might play complementary roles in controlling movement sequences.

Telencephalon: Neocortex

The neocortex is an ultracomplex, six-layered structure that develops from the dorsal palliai sector of the telencephalic hemispheres (Figs. 2.24, 2.25, 11.1). All mammals, including monotremes and marsupials, possess a neocortex, but in reptiles, i.e. the ancestors of mammals, only a three-layered neocortical primordium is present [509, 511]. The term neocortex refers to its late phylogenetic appearance, in comparison to the “palaeocortical” olfactory cortex and the “archicortical” hippocampal cortex, both of which are present in all amniotes [509].

The neural correlates of subjective value during intertemporal choice

Neuroimaging studies of decision-making have generally related neural activity to objective measures (such as reward magnitude, probability or delay), despite choice preferences being subjective. However, economic theories posit that decision-makers behave as though different options have different subjective values. Here we use functional magnetic resonance imaging to show that neural activity in several brain regions--particularly the ventral striatum, medial prefrontal cortex and posterior cingulate cortex--tracks the revealed subjective value of delayed monetary rewards. This similarity provides unambiguous evidence that the subjective value of potential rewards is explicitly represented in the human brain.

The nucleus accumbens and reward: neurophysiological investigations in behaving animals

The nucleus accumbens (Acb) is a crucial component of the brain reward system. This report reviews electrophysiological studies that examined Acb cell firing during goal-directed behaviors for natural reinforcers (food, water, sucrose) and drugs of abuse (cocaine, heroin, ethanol). Studies that examined the role of environmental stimuli and operant contingencies on Acb activity during behavior are also explored. Given the extensive literature that links dopamine in the Acb with drug reinforcement, experiments are considered that examined the influence of dopamine in modulating Acb cell firing during drug-seeking behaviors. Finally, because the Acb is one neural substrate of a larger brain reward circuit, the influence of afferent input (hippocampus and prefrontal cortex) on Acb cell firing during behavior is also discussed. These findings provide a unique insight into the cellular mechanisms underlying reward-related processing and goal-directed behaviors and reveal a level of functional organization in the Acb not identified by other experimental approaches.

Cue-evoked encoding of movement planning and execution in the rat nucleus accumbens

The nucleus accumbens is involved in the modulation of motivated behaviour by reward-associated sensory information. However, little is known about the specific nature of the nucleus accumbens' contribution to generating movement. We investigated motor encoding by nucleus accumbens neurons in rats performing a delayed response task that allowed us to dissociate the effects of sensory and motor events on firing. In a subset of neurons, firing in the delay period preceding movement was highly selective; this selectivity was tightly correlated with the direction of the subsequent movement, but not with the sensory properties of the instructive cue. Direction selectivity in this population of neurons developed over the course of the delay period, with the strongest selectivity apparent just prior to movement onset. Selectivity was also apparent in nucleus accumbens neurons during movement, such that firing showed a tight correlation with movement direction, but not the instructive cue presented nor the spatial destination of the movement. These results are consistent with the hypothesis that a subpopulation of nucleus accumbens neurons contributes to the selection and execution of specific motivated behaviours.

Accumbal neurons that are activated during cocaine self-administration are spared from inhibitory effects of repeated cocaine self-administration

Hypoactivity of the accumbens is induced by repeated cocaine exposure and is hypothesized to play a role in cocaine addiction. However, it is difficult to understand how a general hypoactivity of the accumbens, which facilitates multiple types of motivated behaviors, could contribute to the selective increase in drug-directed behavior that defines addiction. Electrophysiological recordings, made during sessions in which rats self-administer cocaine, show that most accumbal neurons that encode events related to drug-directed behavior achieve and maintain higher firing rates during the period of cocaine exposure (Task-Activated neurons) than do other accumbal neurons (Task-Non-Activated neurons). We have hypothesized that this difference in activity makes the neurons that facilitate drug-directed behavior less susceptible than other neurons to the chronic inhibitory effects of cocaine. A sparing of neurons that facilitate drug-directed behavior from chronic hypoactivity might lead to a relative increase in the transmission of neuronal signals that facilitate drug-directed behavior through accumbal circuits and thereby contribute to changes in behavior that characterize addiction (ie differential inhibition hypothesis). A prediction of the hypothesis is that neurons that are activated in relation to task events during cocaine self-administration sessions will show less of a decrease in firing across repeated self-administration sessions than will other neurons. To test this prediction, rats were exposed to 30 daily (6 h/day) cocaine self-administration sessions. Chronic extracellular recordings of single accumbal neurons were made during the second to third session and the 30th session. Between-session comparisons showed that decreases in firing were exhibited by Task-Non-Activated, but not by Task-Activated, neurons. During the day 30 session, the magnitude of the difference in firing rate between the two groups of neurons was positively related to the propensity of animals to seek and take cocaine. The findings of the present study are consistent with a basic prediction of the differential inhibition hypothesis and may be relevant to understanding cocaine addiction.

Dynamic Changes in Representations of Preceding and Upcoming Reward in Monkey Orbitofrontal Cortex

We investigated how orbitofrontal cortex (OFC) contributes to adaptability in the face of changing reward contingencies by examining how reward representations in monkey orbitofrontal neurons change during a visually cued, multi-trial reward schedule task. A large proportion of orbitofrontal neurons were sensitive to events in this task (69/80 neurons in the valid and 48/58 neurons in the random cue context). Neuronal activity depended upon preceding reward, upcoming reward, reward delivery, and schedule state. Preceding reward-dependent activity occurred in both the valid and random cue contexts, whereas upcoming reward-dependent activity was observed only in the valid context. A greater proportion of neurons encoded preceding reward in the random than the valid cue context. The proportion of neurons with preceding reward-dependent activity declined as each trial progressed, whereas the proportion encoding upcoming reward increased. Reward information was represented by ensembles of neurons, the composition of which changed with task context and time. Overall, neuronal activity in OFC adapted to reflect the importance of different types of reward information in different contexts and time periods. This contextual and temporal adaptability is one hallmark of neurons participating in executive functions.

The nucleus accumbens as part of a basal ganglia action selection circuit

BACKGROUND

The nucleus accumbens is the ventral extent of the striatum, the main input nucleus of the basal ganglia. Recent hypotheses propose that the accumbens and its dopamine projection from the midbrain contribute to appetitive behaviors required to obtain reward. However, the specific nature of this contribution is unclear. In contrast, significant advances have been made in understanding the role of the dorsal striatum in action selection and decision making.

OBJECTIVE

In order to develop a hypothesis of the role of nucleus accumbens dopamine in action selection, the physiology and behavioral pharmacology of the nucleus accumbens are compared to those of the dorsal striatum.

HYPOTHESES

Three hypotheses concerning the role of dopamine in these structures are proposed: (1) that dopamine release in the dorsal striatum serves to facilitate the ability to respond appropriately to temporally predictable stimuli (that is, stimuli that are so predictable that animals engage in anticipatory behavior just prior to the stimulus); (2) that dopamine in the nucleus accumbens facilitates the ability to respond to temporally unpredictable stimuli (which require interruption of ongoing behavior); and (3) that accumbens neurons participate in action selection in response to such stimuli by virtue of their direct (monosynaptic inhibitory) and indirect (polysynaptic excitatory) projections to basal ganglia output nuclei.

Cue and reward signals carried by monkey entorhinal cortex neurons during reward schedules

Ablation of entorhinal/perirhinal cortices prevents learning associations between visual stimuli used as cues in reward schedules and the schedule state. Single neurons in perirhinal cortex are sensitive to associations between the cues and the reward schedules. To investigate whether neurons in the entorhinal cortex have similar sensitivities, we recorded single neuronal activity from two rhesus monkeys while the monkeys performed a visually cued reward schedule task. When the cue was related to the reward schedules, the monkeys made progressively fewer errors as the schedule state became closer to the reward state, showing that the monkeys were sensitive to the cue and the schedule state. Of 75 neurons recorded in the entorhinal cortex during task performance, about 30% responded. About half of these responded after cue presentation. When the relation of the cue to the reward schedules was random, the cue-related responses disappeared or lost their selectivity for schedule states. The responses of the entorhinal cortex neurons are similar to responses of perirhinal cortex neurons in that they are selective for the associative relationships between cues and reward schedules. However, they are particularly selective for the first trial of a new schedule, in contrast to perirhinal cortex where responsivity to all schedule states is seen. A different subpopulation of entorhinal neurons responded to the reward, unlike perirhinal neurons which respond solely to the cue. These results indicate that the entorhinal signals carry associative relationships between the visual cues and reward schedules, and between rewards and reward schedules that are not simply derived from perirhinal cortex by feed-forward serial processing.

Rat Nucleus Accumbens Neurons Persistently Encode Locations Associated With Morphine Reward

When rats and mice are free to explore a familiar environment they spend more time in a previously rewarded location. This conditioned place preference (CPP) results from an increased probability of initiating transitions from an unrewarded location to one previously paired with reward. We recorded nucleus accumbens (NAc) neurons while rats explored a three-room in-line apparatus. Before place conditioning, approximately equal proportions of NAc neurons show excitations or inhibitions when the rat is in each of the rooms (morphine paired, center or saline paired). Conditioning increased the proportion of neurons inhibited while the rat was in the morphine room and neurons excited in the saline room. Many of the neurons in these two groups responded during room transitions. Furthermore, the postconditioning increase in the population of neurons with room-selective responding persisted for several weeks after the last morphine treatment. This long-lasting change in population responses of NAc neurons to initially neutral locations is a neural correlate of the change in location preference manifest as CPP.

Mode changes in activity of single neurons in anterior insular cortex across trials during multi-trial reward schedules

We previously showed that spike count response distributions in anterior cingulate neurons can be fitted by a mixture of a few Poisson distributions in our reward schedule task. Here we report that the neuronal responses in insular cortex, an area connected to anterior cingulate cortex, can also be nicely fitted. The ratio of Poisson distributions changed with schedule progress, suggesting that neuronal responses in these areas fall into discrete firing modes. More insular neurons show mode changes across the schedules. The selection of firing modes might be related to cognitive processes, but seems independent across the two areas.

Effects of reward expectancy on sequential eye movements in monkeys

Desirability of an action, often referred to as utility or value, is determined by various factors, such as the probability and timing of expected reward. We investigated how performance of monkeys in an oculomotor serial reaction time task is influenced by multiple motivational factors. The animals produced a series of visually-guided eye movements, while the sequence of target locations and the location of the rewarded target were systematically manipulated. The results show that error rates as well as saccade latencies were consistently influenced by the number of remaining movements necessary to obtain a reward. In addition, when the animal produced multiple saccades before fixating a given target, the first saccade tended to be directed towards the rewarded location, suggesting that saccades to rewarded location and visual target might be programmed concurrently. These results show that monkeys can utilize information about the required sequence of movements to update their subjective values.

Functional differences between macaque prefrontal cortex and caudate nucleus during eye movements with and without reward

The prefrontal cortex and the basal ganglia form mutually connected networks and are thought to play essential roles together in guiding goal-directed behaviors. Yet, these structures seem to have independent pathways to motor outputs as well, suggesting differential contributions to goal-directed behaviors. We hypothesized that the prefrontal cortex guides actions to a direction required by external demands and the basal ganglia guide actions to an internally motivated direction. To test this hypothesis, we used a task in which monkeys were required to make a memory-guided saccade to a direction indicated by a visual cue while only one direction was associated with reward. We observed a functional dissociation between the lateral prefrontal cortex (LPFC), which commonly represented the cue direction, and the caudate nucleus (CD), which commonly represented the reward-associated direction. Furthermore, cue-directed and reward-directed signals were integrated differently in the two areas; when the cue direction and the reward direction were opposite, LPFC neurons maintained tuning to the cue direction, whereas CD neurons lost the tuning. Different types of spatial tuning in the two brain areas may contribute to different types of goal-directed behavior.

Firing Patterns of Accumbal Neurons During a Pavlovian-Conditioned Approach Task

The nucleus accumbens (NAc) is necessary for the expression of Pavlovian-conditioned approach behavior but not for the expression of instrumental behavior conditioned in sessions that set a low response requirement. Although numerous studies have characterized firing patterns of NAc neurons in relation to instrumental behavior, very little is known about how NAc neurons encode information in Pavlovian tasks. In the present study, recordings of accumbal firing patterns were made during sessions in which rats performed a Pavlovian-conditioned approach task. Most of the recorded neurons (74/83, 89%) exhibited significant responses during the conditioned stimulus (CS) presentation and/or the reward exposure. The reward responses were prevalent, predominantly inhibitory, and comparable to reward responses observed in various types of behavioral paradigms, including instrumental tasks. The CS responses could be segregated into multiple subtypes on the basis of directionality, onset latency, and duration. Several characteristics of the CS firing patterns were unique relative to cue responses observed previously during alternative types of conditioning sessions. It is possible that the novel firing patterns correspond to the differential contributions of the accumbens to Pavlovian-conditioned approach behavior and instrumentally conditioned behavior. Regardless, the novel patterns of firing add to existing evidence that characterization of accumbal firing patterns in Pavlovian tasks may provide additional information about the neurophysiological mechanisms that mediate accumbal contributions to behavior.

Dopamine neuronal responses in monkeys performing visually cued reward schedules

Dopamine neurons are important for reward-related behaviours. They have been recorded during classical conditioning and operant tasks with stochastic reward delivery. However, daily behaviour, although frequently complex in the number of steps, is often very predictable. We studied the responses of 75 dopamine neurons during schedules of trials in which the events and related reward contingencies could be well-predicted, within and across trials. In this visually cued reward schedule task, a visual cue tells the monkeys exactly how many trials, 1, 2, 3, or 4, must be performed to obtain a reward. The number of errors became larger as the number of trials remaining before the reward increased. Dopamine neurons frequently responded to the cues at the beginning and end of the schedules. Approximately 75% of the first-cue responsive neurons did not distinguish among the schedules that were beginning even though the cues were different. Approximately half of the last-cue responsive neurons depended on which schedule was ending, even though the cue signalling the last trial was the same in all schedules. Thus, the responses were related to what the monkey knew about the relation between the cues and the schedules, not the identity of the cues. These neurons also frequently responded to the go signal and/or to the OK signal indicating the end of a correctly performed trial whether a reward was forthcoming or not, and to the reward itself. Thus, dopamine neurons seem to respond to behaviourally important, i.e. salient, events even when the events have been well-predicted.

Activity in the lateral intraparietal area predicts the goal and latency of saccades in a free-viewing visual search task

The purpose of saccadic eye movements is to facilitate vision, by placing the fovea on interesting objects in the environment. Eye movements are not made for reward, and they are rarely restricted. Despite this, most of our knowledge about the neural genesis of eye movements comes from experiments in which specific eye movements are rewarded or restricted. Such experiments have demonstrated that activity in the lateral intraparietal (LIP) area of the monkey correlates with the monkey's planning of a memory-guided saccade or deciding where, on the basis of motion information, to make a saccade. However, other experiments have shown that neural activity in LIP can easily be dissociated from the generation of saccadic eye movements, especially when sophisticated behavioral paradigms dissociate the monkey's locus of attention from the goal of an intended saccade. In this study, we trained monkeys to report the results of a visual search task by making a nontargeting hand movement. Once the task began, the monkeys were entirely free to move their eyes, and rewards were not contingent on the monkeys making specific eye movements. We found that neural activity in LIP predicted not only the goal of the monkey's saccades but also their saccadic latencies.

Behavioral theories and the neurophysiology of reward

The functions of rewards are based primarily on their effects on behavior and are less directly governed by the physics and chemistry of input events as in sensory systems. Therefore, the investigation of neural mechanisms underlying reward functions requires behavioral theories that can conceptualize the different effects of rewards on behavior. The scientific investigation of behavioral processes by animal learning theory and economic utility theory has produced a theoretical framework that can help to elucidate the neural correlates for reward functions in learning, goal-directed approach behavior, and decision making under uncertainty. Individual neurons can be studied in the reward systems of the brain, including dopamine neurons, orbitofrontal cortex, and striatum. The neural activity can be related to basic theoretical terms of reward and uncertainty, such as contiguity, contingency, prediction error, magnitude, probability, expected value, and variance.

Basal ganglia orient eyes to reward

Expectation of reward motivates our behaviors and influences our decisions. Indeed, neuronal activity in many brain areas is modulated by expected reward. However, it is still unclear where and how the reward-dependent modulation of neuronal activity occurs and how the reward-modulated signal is transformed into motor outputs. Recent studies suggest an important role of the basal ganglia. Sensorimotor/cognitive activities of neurons in the basal ganglia are strongly modulated by expected reward. Through their abundant outputs to the brain stem motor areas and the thalamocortical circuits, the basal ganglia appear capable of producing body movements based on expected reward. A good behavioral measure to test this hypothesis is saccadic eye movement because its brain stem mechanism has been extensively studied. Studies from our laboratory suggest that the basal ganglia play a key role in guiding the gaze to the location where reward is available. Neurons in the caudate nucleus and the substantia nigra pars reticulata are extremely sensitive to the positional difference in expected reward, which leads to a bias in excitability between the superior colliculi such that the saccade to the to-be-rewarded position occurs more quickly. It is suggested that the reward modulation occurs in the caudate where cortical inputs carrying spatial signals and dopaminergic inputs carrying reward-related signals are integrated. These data support a specific form of reinforcement learning theories, but also suggest further refinement of the theory.

Neuronal signals in the monkey basolateral amygdala during reward schedules

The amygdala is critical for connecting emotional reactions with environmental events. We recorded neurons from the basolateral complex of two monkeys while they performed visually cued schedules of sequential color discrimination trials, with both valid and random cues. When the cues were valid, the visual cue, which was present throughout each trial, indicated how many trials remained to be successfully completed before a reward. Seventy-six percent of recorded neurons showed response selectivity, with the selectivity depending on some aspects of the current schedule. After a reward, when the monkeys knew that the upcoming cue would be valid, 88 of 246 (36%) neurons responded between schedules, seemingly anticipating the receiving information about the upcoming schedule length. When the cue appeared, 102 of 246 (41%) neurons became selective, at this point encoding information about whether the current trial was the only trial required or how many more trials are needed to obtain a reward. These cue-related responses had a median latency of 120 ms (just between the latencies in inferior temporal visual area TE and perirhinal cortex). When the monkey was releasing a touch bar to complete the trial correctly, 71 of 246 (29%) neurons responded, with responses in the rewarded trials being similar no matter which schedule was ending, thus being sensitive to the reward contingency. Finally, 39 of 246 (16%) neurons responded around the reward. We suggest that basolateral amygdala, by anticipating and then delineating the schedule and representing reward contingency, provide contextual information that is important for adjusting motivational level as a function of immediate behavior goals.

Reward-period Activity in Primate Dorsolateral Prefrontal and Orbitofrontal Neurons Is Affected by Reward Schedules

Reward-period activity observed in the dorsolateral prefrontal cortex (DLPFC) and the orbitofrontal cortex (OFC) is thought to represent the detection of reward delivery. To investigate whether this activity plays the same role in these areas, we examined this activity under different reward schedules and whether the reward schedule has similar effects on this activity in each of these areas. A monkey performed an oculomotor delayed-response (ODR) task under two reward schedules. In the ODR-1 schedule, the monkey received a large amount of reward only after four successful trials, whereas in the ODR-2 schedule, it received a small amount of reward after every successful trial. Although reward-period activity was observed in both areas, more neurons exhibited this activity in the OFC. Reward-period activity was modulated by the proximity to reward delivery in both areas and this feature was observed more frequently in the OFC. The onset time of this activity also gradually advanced depending on the proximity to reward delivery. Moreover, many OFC neurons with this activity responded to free reward delivery. These results indicate that reward-period activity in the OFC represents the detection of reward delivery and that the gradual change in the magnitude and the onset time of this activity represents the expectation of reward delivery. Similar features of reward-period activity were observed in DLPFC neurons, although a significant number of DLPFC neurons did not respond to free reward delivery and no advance was observed in the onset time of this activity. These results suggest that reward-period activity in the DLPFC participates in whether or not correct performance was achieved. Thus, although similar reward-period activity was observed in both areas, the activity in the OFC represents the detection of reward delivery and is affected by the monkey's motivational state, whereas that in the DLPFC seems to participate in monitoring whether or not the necessary performance is achieved.

The effect of the adenosine A(2A) antagonist KW-6002 on motor and motivational processes in the rat

RATIONALE

It is well established that humans and rats respond to an imperative stimulus more rapidly as a function of the foreperiod preceding the target, and with this decrease in mean response time, there is also an increase in anticipatory (prior to the signal) responses. These changes reflect enhanced motor readiness. Also, reaction times are quicker when the cost of reward (amount of work required) is minimum. Antagonism of the adenosine A(2A) receptor has been shown to effect motor-related processes.

OBJECTIVE

This study examined the behavioural effects of systemic administration of the adenosine A(2A) antagonist KW-6002 in a cued reaction time task in the rat. The purpose of this study is to ascertain whether KW-6002 would enhance motor readiness and effect performance as a function of reward cost.

METHODS

Rats were trained on a visually cued reaction time task with variable foreperiods, and the effects of three doses of KW-6002 (0.3, 1.0 and 3.0 mg/kg systemically, compared to vehicle) were examined.

RESULTS

Increasing doses of KW-6002 resulted in faster reaction times and an increase in the number of anticipatory responses. KW-6002 enhanced the foreperiod effect on reaction time distributions and anticipatory responses. In addition, KW-6002 had differential effects on performance between rewarded and unrewarded trials.

CONCLUSION

Antagonism of adenosine A(2A) receptors by systemic KW-6002 speeds reaction time and enhanced motor preparatory processes as well as interacting with motivational processes.