Neurobehavioural mechanisms of reward and motivation

Functional magnetic resonance imaging of brain reward circuitry in the human

To produce behavior, motivational states necessitate at least three fundamental operations, including (1) selection of objectives focused on goal-objects, (2) compilation of goal-object information, and (3) determination of physical plans for securing goal-objects. The second of these general operations has been theorized to involve three subprocesses: (a) feature detection and other perceptual processing of putative goal-object "rewards," (b) valuation of goal-object worth in the context of potential hedonic deficit states, and (c) extraction of incidence and temporal data regarding the goal-object. A number of subcortical brain regions appear to be involved in these three informational subprocesses, in particular, the amygdala, sublenticular extended amygdala (SLEA) of the basal forebrain, and nucleus accumbens/subcallosal cortex (NAc/SCC). Components of the amygdala, SLEA, and NAc/SCC together constitute the larger anatomic structure of the extended amygdala. Functional magnetic resonance imaging (fMRI) studies of humans have recently begun to localize these subcortical regions within the extended amygdala during specific experimental conditions. In this manuscript, two human cocaine- infusion studies and one cognitive psychology experiment are reviewed in relation to their pattern of fMRI activation within regions of the extended amygdala. Activation in the NAc/SCC, in particular, is evaluated in relation to a hypothesis that one function of the NAc/SCC and associated brain regions is the evaluation of goal-object incidence data for the computation of conditional probabilities regarding goal-object availability. Further work is warranted to test hypothesized functions for all regions within the extended amygdala and integrate them toward an understanding of motivated behavior.

Convergence and segregation of ventral striatal inputs and outputs

The ventral striatum, which prominently includes the nucleus accumbens (Acb), is a heterogeneous area. Within the Acb of rats, a peripherally located shell and a centrally situated core can be recognized that have different connectional, neurochemical, and functional identities. Although the Acb core resembles in many respects the dorsally adjacent caudate-putamen complex in its striatal character, the Acb shell has, in addition to striatal features, a more diverse array of neurochemical characteristics, and afferent and efferent connections. Inputs and outputs of the Acb, in particular of the shell, are inhomogeneously distributed, resulting in a mosaical arrangement of concentrations of afferent fibers and terminals and clusters of output neurons. To determine the precise relationships between the distributional patterns of various afferents (e.g., from the prefrontal cortex, the basal amygdaloid complex, the hippocampal formation, and the midline/intralaminar thalamic nuclei) and efferents to the ventral pallidum and mesencephalon, neuroanatomical anterograde and retrograde tracing experiments were carried out. The results of the double anterograde, double retrograde, and anterograde/retrograde tracing experiments indicate that various parts of the shell (dorsomedial, ventromedial, ventral, and lateral) and the core (medial and lateral) have different input-output characteristics. Furthermore, within these Acb regions, various populations of neurons can be identified, arranged in a cluster-like fashion, onto which specific sets of afferents converge and that project to particular output stations, distinct from the input-output relationships of neighboring, cluster-like neuronal populations. These results support the idea that the nucleus accumbens may consist of a collection of neuronal ensembles with different input-output relationships and, presumably, different functional characteristics.

A role for AMPA/kainate receptors in conditioned place preference induced by diazepam in the rat

There is increasing evidence for a role of glutamate receptors in the reinforcing properties of dependence producing drugs such as the psychostimulants and opiates. Activation of AMPA/kainate receptors are implicated in the acquisition of amphetamine-induced reinforcement but a role for this receptor in benzodiazepine-induced reinforcement has not been examined. In the present study the ability of the orally active AMPA/kainate antagonist GYKI 52466 was assessed for its ability to block the reinforcing properties of diazepam in a conditioned place preference paradigm. Diazepam (2.5 and 5.0 mg/kg, i.p.) produced a robust place preference and GYKI 52466 inhibited the acquisition of place preference conditioning-induced by diazepam. These results suggest that glutamatergic pathways are an important component of the circuitry involved in the acquisition of a benzodiazepine induced place preference.

Properties and plasticity of excitatory synapses on dopaminergic and GABAergic cells in the ventral tegmental area

Excitatory inputs to the ventral tegmental area (VTA) influence the activity of both dopaminergic (DA) and GABAergic (GABA) cells, yet little is known about the basic properties of excitatory synapses on these two cell types. Using a midbrain slice preparation and whole-cell recording techniques, we found that excitatory synapses on DA and GABA cells display several differences. Synapses on DA cells exhibit a depression in response to repetitive activation, are minimally affected by the GABAB receptor agonist baclofen, and express NMDA receptor-dependent long-term potentiation (LTP). In contrast, synapses on GABA cells exhibit a facilitation in response to repetitive activation, are depressed significantly by baclofen, and do not express LTP. The relative contribution of NMDA and non-NMDA receptors to the synaptic currents recorded from the two cell types is the same as is the depression of synaptic transmission elicited by the application of adenosine, serotonin, or methionine enkephalin (met-enkephalin). The significant differences in the manner in which excitatory synaptic inputs to DA and GABA cells in the VTA can be modulated have potentially important implications for understanding the behavior of VTA neurons during normal behavior and during pathological states such as addiction.

Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by D-amphetamine

Dopamine release within the nucleus accumbens (NAcc) has been associated with both the rewarding and locomotor-stimulant effects of abused drugs. The functions of the NAcc core and shell were investigated in mediating amphetamine-potentiated conditioned reinforcement and locomotion. Rats were initially trained to associate a neutral stimulus (Pavlovian CS) with food reinforcement (US). After excitotoxic lesions that selectively destroyed either the NAcc core or shell, animals underwent additional CS-US training sessions and then were tested for the acquisition of a new instrumental response that produced the CS acting as a conditioned reinforcer (CR). Animals were infused intra-NAcc with D-amphetamine (0, 1, 3, 10, or 20 microg) before each session. Shell lesions affected neither Pavlovian nor instrumental conditioning but completely abolished the potentiative effect of intra-NAcc amphetamine on responding with CR. Core-lesioned animals were impaired during the Pavlovian retraining sessions but showed no deficit in the acquisition of responding with CR. However, the selectivity in stimulant-induced potentiation of the CR lever was reduced, as intra-NAcc amphetamine infusions dose-dependently increased responding on both the CR lever and a nonreinforced (control) lever. Shell lesions produced hypoactivity and attenuated amphetamine-induced activity. In contrast, core lesions resulted in hyperactivity and enhanced the locomotor-stimulating effect of amphetamine. These results indicate a functional dissociation of subregions of the NAcc; the shell is a critical site for stimulant effects underlying the enhancement of responding with CR and locomotion after intra-NAcc injections of amphetamine, whereas the core is implicated in mechanisms underlying the expression of CS-US associations.

The effect of lithium on methamphetamine-induced regional Fos protein expression in the rat brain

Lithium has been used widely for the treatment of manic states. Since amphetamines produce effects in humans similar to the symptoms of idiopathic mania, amphetamine administration to animals has been proposed as a model of this condition. To investigate the neurobiologic substrates of the antimanic effects of chronic lithium administration, we investigated its effects on methamphetamine-induced regional Fos protein expression in the rat brain. Chronic lithium administration (14 days; serum lithium concentration, 0.41+/-0.02 mEq/l) significantly reduced the number of neuronal nuclei showing immunoreactivity induced by methamphetamine (2mg/kg) in the prefrontal cortex, caudate/putamen, nucleus accumbens, and central nucleus of the amygdala. These results indicate the structural basis in CNS which is responsible for the antimanic effect of lithium.

Pharmacotherapy of adolescent attention deficit hyperactivity disorder: challenges, choices and caveats

A recent increase in stimulant treatment of adolescents with attention deficit hyperactivity disorder (ADHD) has been documented. Challenges in treating adolescent ADHD with methylphenidate or dextroamphetamine include compliance with frequent dosing, abuse potential and wear-off or rebound effects. Co-morbid anxiety, occurring in at least 30 percent of ADHD youths, is associated with lower rate of response to stimulants. The effective alternatives, tricyclic antidepressants or pemoline, are each associated with rare but serious toxicity. Bupropion has recently proven effective in controlled trials. Other noradrenergic or dopamine-enhancing agents such as venlafaxine and nicotine show some benefit in open trials. The need for more options in pharmacotherapy of ADHD is evidenced by rapid adoption in clinical practice of alternative and adjunctive medication despite lack of controlled research on efficacy and safety. The indications for long-term stimulant treatment of ADHD present some controversy, and highlight a need for more research on safety and efficacy through the lifespan. Thresholds for diagnosis are much lower with DSM than with ICD, and thresholds for treatment are contentious, given the performance-enhancing effects of stimulants in normal students. The endpoint for treatment is unclear, as stimulants are also effective in adult ADHD. Based on short- and intermediate-term studies to date, stimulant medication is clearly more efficacious than cognitive and behavioral strategies for the symptoms of ADHD. Longer term research is needed to determine whether sustained stimulant therapy will reduce the adverse emotional, behavioral and academic consequences of inattention and impulsivity in adolescents and adults.

Activation of reward circuitry in human opiate addicts

The neurobiological mechanisms of opiate addictive behaviour in humans are unknown. A proposed model of addiction implicates ascending brainstem neuromodulatory systems, particularly dopamine. Using functional neuroimaging, we assessed the neural response to heroin and heroin-related cues in established opiate addicts. We show that the effect of both heroin and heroin-related visual cues are maximally expressed in the sites of origin of ascending midbrain neuromodulatory systems. These context-specific midbrain activations predict responses to salient visual cues in cortical and subcortical regions implicated in reward-related behaviour. These findings implicate common neurobiological processes underlying drug and drug-cue-related effects.

Amygdala activity related to enhanced memory for pleasant and aversive stimuli

Pleasant or aversive events are better remembered than neutral events. Emotional enhancement of episodic memory has been linked to the amygdala in animal and neuropsychological studies. Using positron emission tomography, we show that bilateral amygdala activity during memory encoding is correlated with enhanced episodic recognition memory for both pleasant and aversive visual stimuli relative to neutral stimuli, and that this relationship is specific to emotional stimuli. Furthermore, data suggest that the amygdala enhances episodic memory in part through modulation of hippocampal activity. The human amygdala seems to modulate the strength of conscious memory for events according to emotional importance, regardless of whether the emotion is pleasant or aversive.

Psychostimulant-induced Fos protein expression in the thalamic paraventricular nucleus

Lesions of glutamatergic afferents to the nucleus accumbens have been reported to block psychostimulant-induced behavioral sensitization. However, thalamic glutamatergic projections to the nucleus accumbens have received little attention in the context of psychostimulant actions. We examined the effects of acute amphetamine and cocaine administration on expression of Fos protein in the thalamic paraventricular nucleus (PVT), which provides glutamatergic inputs to the nucleus accumbens and also receives dopaminergic afferents. Immunoblot and immunohistochemical studies revealed that both psychostimulants dose-dependently increased PVT Fos expression. PVT neurons retrogradely labeled from the nucleus accumbens were among the PVT cells that showed a Fos response to amphetamine. D2 family dopamine agonists, including low doses of the D3-preferring agonist 7-OH-DPAT, increased the numbers of Fos-like-immunoreactive neurons in the PVT. Conversely, the effects of cocaine and amphetamine on PVT Fos expression were blocked by pretreatment with the dopamine D2/3 antagonist raclopride. Because PVT neurons express D3 but not other dopamine receptor transcripts, it appears that psychostimulants induce Fos in PVT neurons through a D3 dopamine receptor. We suggest that the PVT may be an important part of an extended circuit subserving both the arousing properties and reinforcing aspects of psychostimulants.

Facilitation of sexual behavior and enhanced dopamine efflux in the nucleus accumbens of male rats after D-amphetamine-induced behavioral sensitization

Behavioral sensitization caused by repeated and intermittent administration of psychostimulants, such as cocaine and D-amphetamine, is accompanied by enhanced function in limbic-motor circuitry that is involved in the generation of motivated behavior. The present microdialysis study investigated the effect of D-amphetamine-induced sensitization on dopamine (DA) efflux in the nucleus accumbens (NAC) of male rats during sexual behavior. Male rats were given one injection of D-amphetamine (1.5 mg/kg, i.p.) or saline every other day for a total of 10 injections. Three weeks after discontinuation of drug treatment, rats were tested for sexual behavior during a test in which microdialysis was performed. There was an augmented efflux of DA in the NAC of D-amphetamine-sensitized rats compared with nonsensitized control rats when a receptive female was present behind a screen (35 vs 17%). Sensitized rats exhibited facilitated sexual behavior when the screen was removed, as indicated by a significantly shorter latency to mount and an overall increase in the amount of copulatory behavior. Although there was a significant increase in NAC DA concentrations from baseline in both sensitized and nonsensitized rats during copulation, there was a greater increase in DA efflux in the NAC of sensitized rats during the first 10 min copulatory sample (60 vs 37%). These results demonstrate that behavioral sensitization caused by repeated psychostimulant administration can "cross-sensitize" to a natural behavior, such as sex, and that increased NAC DA release may contribute to the facilitation of appetitive and consummatory aspects of this behavior.

Modulation by dopamine D1-like receptors of synaptic transmission and NMDA receptors in rat nucleus accumbens is attenuated by the protein kinase C inhibitor Ro 32-0432

Dopamine, acting at a D1-like receptor, depresses the release of glutamate in the nucleus accumbens (NAcc) in brain slices, thereby reducing the amplitude of the excitatory postsynaptic current (EPSC). This effect depends upon an inhibitory feedback action of adenosine, liberated following facilitation of postsynaptic NMDA receptors by D1 receptor activation, an action independent of adenylyl cyclase stimulation or cyclic AMP-dependent protein kinase (PKA; Harvey, J., Lacey, M.G., 1997. J. Neurosci. 17, 5271). Using whole-cell recording from NAcc neurones, the dopamine depression of the EPSC was blocked by pre-treatment of brain slices with the selective protein kinase C (PKC) inhibitor Ro 32-0432, but only minimally attenuated by intracellular dialysis of single cells with Ro 32-0432 in the recording pipette. With synaptic transmission blocked by tetrodotoxin, inward currents caused by application of NMDA were enhanced by the D1 receptor agonist SKF 81297A in half the cells tested. In a separate population of cells dialysed intracellularly with Ro 32-0432, SKF 81297A was without effect on NMDA current amplitude. These findings indicate a functional role for phospholipase C-coupled D1-like receptors in both modulating synaptic transmission in NAcc and potentiating NMDA receptors on a subset of NAcc neurones, via PKC activation.

Neuronal spike activity in the nucleus accumbens of behaving rats during ethanol self-administration

Many lines of evidence support the importance of the nucleus accumbens (NAC) for ethanol-reinforced behavior. The nature of the neuronal activity that occurs in this region during ethanol self-administration is not known. We recorded from ensembles of single-units primarily located within the shell of the NAC during operant responding for oral ethanol solutions by well-trained rats. Of 90 units recorded from seven sessions from seven rats, 41 (46%) did not exhibit significant changes in relation to the experimental events. Of the 49 units (54%) that did exhibit significant phasic changes, alterations in firing rate occurred in relation to the following experimental events: operant response (63%), tone stimulus (20%), and ethanol delivery (63%). In addition, changes in spike activity during the intervals between the three experimental events were noted in 33% of the units. Most units (55% of responsive units) responded to multiple experimental events. Thus different but overlapping populations of neurons in the NAC represent each event that occurs along the temporal dimension of a single trial performed to obtain ethanol reward. The data suggest that the NAC plays a crucial role in linking together conditioned and unconditioned internal and external stimuli with motor plans to allow for ethanol-seeking behavior to occur.

Emotion, motivation, and anxiety: brain mechanisms and psychophysiology

The organization of response systems in emotion is founded on two basic motive systems, appetitive and defensive. The subcortical and deep cortical structures that determine primary motivated behavior are similar across mammalian species. Animal research has illuminated these neural systems and defined their reflex outputs. Although motivated behavior is more complex and varied in humans, the simpler underlying response patterns persist in affective expression. These basic phenomena are elucidated here in the context of affective perception. Thus, the research examines human beings watching uniquely human stimuli--primarily picture media (but also words and sounds) that prompt emotional arousal--showing how the underlying motivational structure is apparent in the organization of visceral and behavioral responses, in the priming of simple reflexes, and in the reentrant processing of these symbolic representations in the sensory cortex. Implications of the work for understanding pathological emotional states are discussed, emphasizing research on psychopathy and the anxiety disorders.

Effects of dopamine antagonists and accumbens dopamine depletions on time-constrained progressive-ratio performance

Four experiments were conducted to determine the effects of dopamine (DA) antagonists and DA depletions on progressive-ratio responding for food reinforcement. On this schedule, ratio requirement increased by one response after each reinforcer was obtained, and rats were tested in 30-min sessions. Response rates and highest ratio completed were reduced in a dose-related manner by systemic injections of the D1 antagonist SCH 23390, and also by the D2 antagonists haloperidol and raclopride. Drug-treated rats also showed reductions in time to complete the last ratio, demonstrating that they had stopped responding before the end of the session. DA depletions produced by injections of 6-OHDA directly into the nucleus accumbens substantially decreased both the number of responses and the highest ratio completed. The deficits in response number and highest ratio completed induced by DA depletions persisted through the first 3 weeks of postsurgical testing, with some recovery by the fourth week. However, the deficits resulting from dopamine depletions were largely a manifestation of a decrease in response rate; although time to complete the last ratio was significantly reduced by dopamine depletions in the first few days of testing, rats recovered on this measure by the fifth day after surgery. Although previous work has shown that performance on several schedules (e.g., continuous, low value ratios, variable interval) is relatively unaffected by accumbens DA depletions, the present data demonstrate that such depletions do produce a substantial and persistent impairment of progressive ratio response output. Rats with accumbens DA depletions appear to have deficits in maintaining the high work output necessary for responding at large ratio values. The relative sparing of responding on some simple schedules, together with the present progressive ratio results, suggest that rats with accumbens DA depletions remain directed toward the acquisition and consumption of food, but they show deficits in work output for food.

What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?

What roles do mesolimbic and neostriatal dopamine systems play in reward? Do they mediate the hedonic impact of rewarding stimuli? Do they mediate hedonic reward learning and associative prediction? Our review of the literature, together with results of a new study of residual reward capacity after dopamine depletion, indicates the answer to both questions is 'no'. Rather, dopamine systems may mediate the incentive salience of rewards, modulating their motivational value in a manner separable from hedonia and reward learning. In a study of the consequences of dopamine loss, rats were depleted of dopamine in the nucleus accumbens and neostriatum by up to 99% using 6-hydroxydopamine. In a series of experiments, we applied the 'taste reactivity' measure of affective reactions (gapes, etc.) to assess the capacity of dopamine-depleted rats for: 1) normal affect (hedonic and aversive reactions), 2) modulation of hedonic affect by associative learning (taste aversion conditioning), and 3) hedonic enhancement of affect by non-dopaminergic pharmacological manipulation of palatability (benzodiazepine administration). We found normal hedonic reaction patterns to sucrose vs. quinine, normal learning of new hedonic stimulus values (a change in palatability based on predictive relations), and normal pharmacological hedonic enhancement of palatability. We discuss these results in the context of hypotheses and data concerning the role of dopamine in reward. We review neurochemical, electrophysiological, and other behavioral evidence. We conclude that dopamine systems are not needed either to mediate the hedonic pleasure of reinforcers or to mediate predictive associations involved in hedonic reward learning. We conclude instead that dopamine may be more important to incentive salience attributions to the neural representations of reward-related stimuli. Incentive salience, we suggest, is a distinct component of motivation and reward. In other words, dopamine systems are necessary for 'wanting' incentives, but not for 'liking' them or for learning new 'likes' and 'dislikes'.

The identification of abnormal behaviour and behavioural problems in stabled horses and their relationship to horse welfare: a comparative review

Many behaviours in domestic animals, such as the 'stable vices' of horses, are treated because they are considered undesirable for economic or cultural reasons, and not because the activity affects the horse's quality of life. The impact of a behaviour on the human reporter is not a function of its impact on the animal performer, and an understanding of the causes and effects of the particular activity is necessary to assess the costs and benefits of treatment. Where the behaviour is a sign of poor welfare, such as an inadequate environment, treatment can best be achieved by removing these underlying causal factors. Pharmacological or physical prevention of a behaviour can be justified only if the behaviour causes harm to the performer or to others. In these cases, prevention of the behaviour without addressing its causes is no cure and may result in its perseverance in a modified form or the disruption of the animal's ability to adapt to its environment. Where the behavioural 'problem' causes no harm and is not related to poor housing, then the education of the reporter, rather than treatment of the performer, may be the best solution.

Pharmacological stimuli decreasing nucleus accumbens dopamine can act as positive reinforcers but have a low addictive potential

Opioid peptides, through mu and delta receptors, play an important part in reward. In contrast, the role of kappa receptors is more controversial. We examined the possible positive reinforcing effects of a selective kappa agonist, RU 51599, by studying intravenous self-administration in the rat. The effect of RU 51599 on dopamine release in the nucleus accumbens was also studied, as opioids and dopamine seem to interact in the mediation of reward. The behavioural and dopaminergic effects of RU 51599 were compared with those of the mu agonist heroin. Rats self-administered both RU 51599 (6.5, 20 and 60 microg/inj) and heroin (30 microg/inj) at low ratio requirement. When the ratio requirement, i.e. the number of responses necessary to receive one drug infusion, was increased, self-administration of RU 51599 rapidly extinguished, whereas self-administration of heroin was maintained. Intravenous infusion of RU 51599 (100, 200 and 400 microg) dose-dependently decreased (25, 30 and 40%, respectively) extracellular concentrations of dopamine, as measured by means of microdialysis in freely moving rats. In contrast, heroin increased accumbens dopamine (130% over baseline). These results indicate that kappa receptors, similarly to mu ones, can mediate positive reinforcing effects of opioid peptides. However, the strength of the reinforcement is very low for kappa receptors. This suggests that changes in accumbens dopamine do not correlate with the capacity of a stimulus to induce reward or aversion. In contrast, a parallel seems to exist between an increase in accumbens dopamine and the drive to reach or obtain a positive reinforcer.

The adaptive nature of the human neurocognitive architecture: an alternative model

The model of the human neurocognitive architecture proposed by evolutionary psychologists is based on the presumption that the demands of hunter-gatherer life generated a vast array of cognitive adaptations. Here we present an alternative model. We argue that the problems inherent in the biological markets of ancestral hominids and their mammalian predecessors would have required an adaptively flexible, on-line information-processing system, and would have driven the evolution of a functionally plastic neural substrate, the neocortex, rather than a confederation of evolutionarily prespecified social cognitive adaptations. In alignment with recent neuroscientific evidence, we suggest that human cognitive processes result from the activation of constructed cortical representational networks, which reflect probabilistic relationships between sensory inputs, behavioral responses, and adaptive outcomes. The developmental construction and experiential modification of these networks are mediated by subcortical circuitries that are responsive to the life history regulatory system. As a consequence, these networks are intrinsically adaptively constrained. The theoretical and research implications of this alternative evolutionary model are discussed.

Expectation of reward modulates cognitive signals in the basal ganglia

Action is controlled by both motivation and cognition. The basal ganglia may be the site where these kinds of information meet. Using a memory-guided saccade task with an asymmetric reward schedule, we show that visual and memory responses of caudate neurons are modulated by expectation of reward so profoundly that a neuron's preferred direction often changed with the change in the rewarded direction. The subsequent saccade to the target was earlier and faster for the rewarded direction. Our results indicate that the caudate contributes to the determination of oculomotor outputs by connecting motivational values (for example, expectation of reward) to visual information.

Dopamine facilitates long-term depression of glutamatergic transmission in rat prefrontal cortex

Using sharp-electrode intracellular recordings, we studied the dopaminergic facilitation of synaptic plasticity in layer I-II afferents--layer V neuron glutamatergic synapses in rat prefrontal cortex in vitro. Tetanic stimulation (100 pulses at 50 Hz, four times at 0.1 Hz) to layer I-II afferents induced N-methyl-D-aspartate receptor-independent long-term depression (>40 min) of the glutamatergic synapses when the stimulation was coupled with a bath-application of dopamine. Tetanic stimulation alone did not induce lasting synaptic changes. Dopamine application alone transiently depressed synaptic responses, which fully recovered within 30 min. Pharmacological analyses with antagonists suggested that dopamine action on either D1-like or D2-like receptors can facilitate the induction of long-term depression. However, results with agonists were not fully consistent with the antagonist results: while a D2 agonist mimicked the facilitatory dopamine effect, D1 agonists failed to mimic the effect. We also analysed the synaptic responses during tetanus and found that dopamine prolongs membrane depolarization during high-frequency inputs. Postsynaptic membrane depolarization is indeed critical for long-term depression induction in the presence of dopamine, since postsynaptic hyperpolarization during tetanus blocked the dopaminergic facilitation of long-term depression induction. Postsynaptic injection of the Ca2+ chelator bis-(o-aminophenoxy)-N,N,N',N'-tetra-acetic acid (100 mM in the electrode) also blocked long-term depression induction. Our results show that dopamine lowers the threshold for long-term depression induction in rat prefrontal glutamatergic transmission. A possible underlying mechanism of this dopaminergic facilitation is the enhancement of postsynaptic depolarization during tetanus by dopamine, which may increase the amount of Ca2+ entry from voltage-gated channels to the level sufficient for plasticity induction.

Dopamine neurons report an error in the temporal prediction of reward during learning

Many behaviors are affected by rewards, undergoing long-term changes when rewards are different than predicted but remaining unchanged when rewards occur exactly as predicted. The discrepancy between reward occurrence and reward prediction is termed an 'error in reward prediction'. Dopamine neurons in the substantia nigra and the ventral tegmental area are believed to be involved in reward-dependent behaviors. Consistent with this role, they are activated by rewards, and because they are activated more strongly by unpredicted than by predicted rewards they may play a role in learning. The present study investigated whether monkey dopamine neurons code an error in reward prediction during the course of learning. Dopamine neuron responses reflected the changes in reward prediction during individual learning episodes; dopamine neurons were activated by rewards during early trials, when errors were frequent and rewards unpredictable, but activation was progressively reduced as performance was consolidated and rewards became more predictable. These neurons were also activated when rewards occurred at unpredicted times and were depressed when rewards were omitted at the predicted times. Thus, dopamine neurons code errors in the prediction of both the occurrence and the time of rewards. In this respect, their responses resemble the teaching signals that have been employed in particularly efficient computational learning models.

Predictive reward signal of dopamine neurons

The effects of lesions, receptor blocking, electrical self-stimulation, and drugs of abuse suggest that midbrain dopamine systems are involved in processing reward information and learning approach behavior. Most dopamine neurons show phasic activations after primary liquid and food rewards and conditioned, reward-predicting visual and auditory stimuli. They show biphasic, activation-depression responses after stimuli that resemble reward-predicting stimuli or are novel or particularly salient. However, only few phasic activations follow aversive stimuli. Thus dopamine neurons label environmental stimuli with appetitive value, predict and detect rewards and signal alerting and motivating events. By failing to discriminate between different rewards, dopamine neurons appear to emit an alerting message about the surprising presence or absence of rewards. All responses to rewards and reward-predicting stimuli depend on event predictability. Dopamine neurons are activated by rewarding events that are better than predicted, remain uninfluenced by events that are as good as predicted, and are depressed by events that are worse than predicted. By signaling rewards according to a prediction error, dopamine responses have the formal characteristics of a teaching signal postulated by reinforcement learning theories. Dopamine responses transfer during learning from primary rewards to reward-predicting stimuli. This may contribute to neuronal mechanisms underlying the retrograde action of rewards, one of the main puzzles in reinforcement learning. The impulse response releases a short pulse of dopamine onto many dendrites, thus broadcasting a rather global reinforcement signal to postsynaptic neurons. This signal may improve approach behavior by providing advance reward information before the behavior occurs, and may contribute to learning by modifying synaptic transmission. The dopamine reward signal is supplemented by activity in neurons in striatum, frontal cortex, and amygdala, which process specific reward information but do not emit a global reward prediction error signal. A cooperation between the different reward signals may assure the use of specific rewards for selectively reinforcing behaviors. Among the other projection systems, noradrenaline neurons predominantly serve attentional mechanisms and nucleus basalis neurons code rewards heterogeneously. Cerebellar climbing fibers signal errors in motor performance or errors in the prediction of aversive events to cerebellar Purkinje cells. Most deficits following dopamine-depleting lesions are not easily explained by a defective reward signal but may reflect the absence of a general enabling function of tonic levels of extracellular dopamine. Thus dopamine systems may have two functions, the phasic transmission of reward information and the tonic enabling of postsynaptic neurons.

Evaluation of unconditioned novelty-seeking and d-amphetamine-conditioned motivation in mice

Following repeated association between psychostimulant drugs and a distinct environment, contextual cues acquire the ability to elicit a conditioned approach response. Further, both rats and mice have a natural drive to seek for the experience of novelty, and a previously unknown environment is able to elicit an unconditioned approach response. Both the experience of novelty and amphetamine (AMPH)-conditioned effects have been associated in rodents with the activation of brain meso-limbic dopaminergic pathways. This study assessed the relative strength of AMPH-conditioned and novelty-induced unconditioned motivations in mice. During the pretreatment period, mice were randomly assigned to three different treatment history groups, and received d-AMPH (0, 2, or 10 mg/kg i.p. once/day) injections for 3 days in the presence of a familiar environment. Following a 48-h washout from the last drug injection, animals were placed in the familiar and pretreatment-paired environment and challenged with either SAL (to evaluate conditioning) or a standard AMPH dose (2 mg/kg, to assess either acute drug effects or carryover influences of each animal's treatment history with the same drug). Following the opening of a partition, animals showed both a clear-cut preference for a novel environment as well as a marked novelty-induced hyperactivity. Interestingly, when mice were tested in a drug-free state in this free-choice paradigm, they expressed neither conditioning to the drug-associated environment nor carry-over effects on the novelty-induced hyperactivity profile. On the other hand, mice injected with AMPH showed a mixed profile, with AMPH 2 treatment history mice showing a conditioned preference for the familiar and drug-paired environment, whereas AMPH 10 animals preferred to spend more time in the novel compartment. Both AMPH doses were associated with an increased locomotion, whereas only the AMPH 10 dose resulted in a stereotyped behavioral syndrome, possibly reminiscent of an aversive "poor welfare" condition. Thus, as a function of the drug dosage, differential positive or negative incentive properties are suggested to be evoked by the AMPH-conditioned environment. In conclusion, a reliable and useful experimental paradigm has been developed to investigate the issue of vulnerability to a variety of habit-forming agents or emotional experiences whose positive reinforcing properties may rely on a common neurobiological mechanism.

Basolateral amygdala stimulation evokes glutamate receptor-dependent dopamine efflux in the nucleus accumbens of the anaesthetized rat

Afferents from the basolateral amygdala and dopamine projections from the ventral tegmental area to the nucleus accumbens have both been implicated in reward-related processes. The present study used in vivo chronoamperometry with stearate-graphite paste electrodes in urethane-anaesthetized rats to determine how basolateral amygdala efferents to the nucleus accumbens synaptically regulate dopamine efflux. Repetitive-pulse (20 Hz for 10 s) electrical stimulation of the basolateral amygdala evoked a complex pattern of changes in monitored dopamine oxidation currents in the nucleus accumbens related to dopamine efflux. These changes were characterized by an initial increase that was time-locked to stimulation, a secondary decrease below baseline, followed by a prolonged increase in the dopamine signal above baseline. The effects of burst-patterned stimulation (100 Hz, 5 pulses/burst, 1-s interburst interval, 40 s) of the basolateral amygdala on the basal accumbens dopamine signal were similar to those evoked by 20 Hz stimulation, with the lack of a secondary suppressive component. Infusions of the ionotropic glutamate receptor antagonists (+/-)-2-amino-5-phosphonopentanoic acid (APV) or 6,7-dinitroquinoxaline-2,3-dione (DNQX) into the nucleus accumbens dose-dependently blocked or attenuated the initial and prolonged increases in the dopamine signal following 20 Hz or burst-patterned basolateral amygdala stimulation. Infusions of the metabotropic glutamate receptor antagonist (+)-alpha-methyl-4-carboxyphenylglycine selectively blocked the intermediate suppressive effect of 20 Hz basolateral amygdala stimulation on dopamine oxidation currents. Blockade of glutamate receptors or inhibition of dopamine neuronal activity via infusions of either APV + DNQX, lidocaine or gamma-hydroxybutyric acid, respectively, into the ventral tegmental area did not effect the pattern of changes in the accumbens dopamine signal evoked by basolateral amygdala stimulation. These data suggest that the glutamatergic basolateral amygdala inputs to nucleus accumbens dopamine terminals synaptically facilitate or depress dopamine efflux, and these effects are independent of dopamine neuronal firing activity. Moreover, these results imply that changes in nucleus accumbens dopamine levels following presentation of reward-related stimuli may be mediated, in part, by the basolateral amygdala.

Goal-directed instrumental action: contingency and incentive learning and their cortical substrates

Instrumental behaviour is controlled by two systems: a stimulus-response habit mechanism and a goal-directed process that involves two forms of learning. The first is learning about the instrumental contingency between the response and reward, whereas the second consists of the acquisition of incentive value by the reward. Evidence for contingency learning comes from studies of reward devaluation and from demonstrations that instrumental performance is sensitive not only the probability of contiguous reward but also to the probability of unpaired rewards. The process of incentive learning is evident in the acquisition of control over performance by primary motivational states. Preliminary lesion studies of the rat suggest that the prelimbic area of prefrontal cortex plays a role in the contingency learning, whereas the incentive learning for food rewards involves the insular cortex.

Reward prediction in primate basal ganglia and frontal cortex

Reward information is processed in a limited number of brain structures, including fronto-basal ganglia systems. Dopamine neurons respond phasically to primary rewards and reward-predicting stimuli depending on reward unpredictability but without discriminating between rewards. These responses reflect 'errors' in the prediction of rewards in correspondence to learning theories and thus may constitute teaching signals for appetitive learning. Neurons in the striatum (caudate, putamen, ventral striatum) code reward predictions in a different manner. They are activated during several seconds when animals expect predicted rewards. During learning, these activations occur initially in rewarded and unrewarded trials and become subsequently restricted to rewarded trials. This occurs in parallel with the adaptation of reward expectations by the animals, as inferred from their behavioral reactions. Neurons in orbitofrontal cortex respond differentially to stimuli predicting different liquid rewards, without coding spatial or visual features. Thus, different structures process reward information processed in different ways. Whereas dopamine neurons emit a reward teaching signal without indicating the specific reward, striatal neurons adapt expectation activity to new reward situations, and orbitofrontal neurons process the specific nature of rewards. These reward signals need to cooperate in order for reward information to be used for learning and maintaining approach behavior.

The role of accumbens dopamine in lever pressing and response allocation: effects of 6-OHDA injected into core and dorsomedial shell

Three experiments investigated the behavioral effects of injections of the neurotoxic agent 6-hydroxydopamine (6-OHDA) into the core or shell of the nucleus accumbens. In the first experiment, it was observed that injections of 6-OHDA into either core or shell had no significant effect on variable interval 30-s responding. In Experiment 2, responding on a fixed ratio 5 (FR5) schedule was impaired by 6-OHDA injections in the core, but not the shell. Rats with core injections of 6-OHDA showed significant alterations in the relative distribution of interresponse times, which were indicative of reductions in the maximal rate of responding and increases in the number of pauses. In the third experiment, rats were tested using a lever-pressing/chow-feeding procedure, in which a preferred food (Bioserve pellets) was available by pressing a lever on a FR5 schedule, but a less preferred food (lab chow) was also available concurrently in the test chamber. Untreated rats usually pressed the lever at high rates to obtain the food pellets and ate little of the lab chow. After training, dopamine depletions were produced by injections of 6-OHDA directly into the core or dorsomedial shell subregions. Injections of 6-OHDA into the core significantly decreased lever pressing for food pellets, increased lab chow consumption, and decreased the relative amount of food obtained by lever pressing. Dorsomedial shell injections of 6-OHDA had no significant effects on either lever pressing or lab chow consumption. Neurochemical results indicate that injections of 6-OHDA in the shell produced substantial depletions in the shell that were somewhat selective; however, injections of 6-OHDA into the core tended to deplete both core and shell. Correlational analyses revealed that decreases in FR5 lever pressing were associated with dopamine levels in the core, but not the shell. The present results indicate that substantial depletions of dopamine in the dorsomedial shell are not sufficient for suppressing reinforced lever pressing, and indicate that dopamine depletions must include the core area to impair performance on these tasks. The lack of effect of accumbens dopamine depletions on VI30 responding are consistent with the notion that accumbens dopamine depletions affect responding on schedules that generate a high rate of responding (FR5), but not those that generate a moderate rate of responding (e.g., VI30 s). The results of the concurrent FR5/chow-feeding experiment indicate that rats with accumbens dopamine depletions remain directed towards the acquisition and consumption of food. These results suggest that dopamine in the core region of accumbens sets constraints upon the selection of food-related behaviors, and that core dopamine depletions alter the relative allocation of food-related responses.

Metabotropic glutamate receptors depress glutamate-mediated synaptic input to rat midbrain dopamine neurones in vitro

1. Glutamate (AMPA receptor-mediated) excitatory postsynaptic potentials (e.p.s.ps.), evoked by electrical stimulation rostral to the recording site, were examined by intracellular microelectrode recording from dopamine neurones in parasagittal slices of rat ventral midbrain. 2. The e.p.s.p. was depressed by the group III metabotropic glutamate (mGlu) receptor agonist L-2-amino-4-phosphonobutyric acid (L-AP4; 0.01-30 microM) by up to 60% with an EC50 of 0.82 microM. The depression induced by L-AP4 (3 microM) was reversed by the group III preferring mGlu receptor antagonist, alpha-methyl-4-phosphonophenylglycine (MPPG; 250 microM). 3. The group I and II mGlu agonist, 1S,3R-aminocyclopentanedicarboxylic acid (ACPD; 3-30 microM) also depressed the e.p.s.p. in a concentration-dependent manner. The effect of ACPD (10 microM) was reversed by (+)-alpha-methyl-4-carboxyphenylglycine (MCPG; 1 mM; 4 cells). This effect of ACPD was also partially antagonized (by 50.3+/-15.7%, 4 cells) by MPPG (250 microM). 4. The selective agonist at group I mGlu receptors, dihydroxyphenylglycine (DHPG; 100 microM), decreased e.p.s.p. amplitude by 27.1+/-8.2% (7 cells), as did the group II mGlu receptor-selective agonist (1S,1R,2'R,3'R)-2-(2,3-dicarboxycyclopropyl)glycine (DCG-IV; 1 microM) by 26.7+/-4.3% (5 cells). 5. DHPG (10-100 microM) caused a depolarization of the recorded cell, as did ACPD (3-30 microM), whereas no such postsynaptic effect of either L-AP4 or DCG-IV was observed. 6. These results provide evidence for the presence of presynaptic inhibitory metabotropic glutamate autoreceptors from the mGlu receptor groups II and III on descending glutamatergic inputs to midbrain dopamine neurones. Group I mGlu receptors mediate a postsynaptic depolarization, and can also depress glutamatergic transmission, but may not necessarily be localized presynaptically. These sites represent novel drug targets for treatment of schizophrenia and movement disorders of basal ganglia origin.

Real time measurement of stimulated dopamine release in the conscious rat using fast cyclic voltammetry: dopamine release is not observed during intracranial self stimulation

Fast cyclic voltammetry (FCV) was used to measure real time release of electrically stimulated endogenous dopamine in the nucleus accumbens (NAc) of conscious freely moving rats for up to 17 days. The method of electrode construction, implantation, electrical stimulation and recording of changes of extracellular dopamine concentration in the conscious rat are described. Rats trained on a continuous reinforcement schedule to perform intracranial self stimulation (ICSS) were implanted with electrodes for FCV. During ICSS, no faradaic signal was observed at an electrode implanted in the NAc. Decreasing the intensity of the stimulating current abolished ICSS, increasing the stimulating current disrupted ICSS. Operator delivered electrical stimulations using currents greater than those needed for ICSS yielded dopamine signals. It is concluded that during ICSS, sufficient dopamine does not reach the extracellular fluid space to yield a faradaic signal detectable by FCV.

Mice deficient in G(olf) are anosmic

We have used gene targeting to examine the role of the G alpha subunit, G(olf), in olfactory signal transduction. Mice homozygous for a null mutation in G(olf) show a striking reduction in the electrophysiological response of primary olfactory sensory neurons to a wide variety of odors. Despite this profound diminution in response to odors, the topographic map of primary sensory projections to the olfactory bulb remains unaltered in G(olf) mutants. Greater than 75% of the G(olf) mutant mice are unable to nurse and die within 2 days after birth. Rare surviving homozygotes mate and are fertile, but mutant females exhibit inadequate maternal behaviors. Surviving homozygous mutant mice also exhibit hyperactive behaviors. These behavioral phenotypes, taken together with the patterns of G(olf) expression, suggest that G(olf) is required for olfactory signal transduction and may also function as an essential signaling molecule more centrally in the brain.

Sex-linked behavioural differences in mice expressing a human insulin transgene in the medial habenula

We previously reported that a human insulin transgene was specifically expressed in the medial habenula of the adult mouse brain, and that this expression was ascribed to the delta-168 transgene. The present study analyses the possible behavioural consequences of this insulin transgene expression using measures of food intake, spontaneous activity, emotional reactivity, learning and extinction performance of an operant task. The delta-168 transgenic mice did not differ from the C57BL/6 control mice as concerns food intake, behaviour in the open field, or emotional response in an elevated plus maze. On the other hand, measures of locomotor activity in a circular corridor revealed a significantly faster decline of spontaneous locomotor activity in male as compared to female delta-168 transgenic mice. Moreover, as compared to female transgenic mice, male transgenic mice exhibited a deficit in the rate of acquisition and an acceleration of the rate of extinction of a bar press response in a Skinner box. In contrast, the behaviour of female transgenic mice did not differ from either male or female C57BL/6 control mice. The results of the present study demonstrate that the behavioural modifications observed in delta-168 transgenic mice are sex-linked and suggest that these behavioural differences result from changes in the interaction (interface) between motivational and motor mechanisms mediated via the striato-habenulo-mesencephalic system.

Glucocorticoids as a biological substrate of reward: physiological and pathophysiological implications

The observations presented in this review suggest that glucocorticoids are one of the biological substrates of reward. These hormones are secreted in response to rewarding stimuli, such as food, a receptive sexual partner or drugs of abuse. Furthermore, manipulations of the secretion of glucocorticoids modify reward-related behaviours, and administration of these hormones, in the range of physiological stress levels, has positive reinforcing effects. The rewarding effects of glucocorticoids are probably mediated by a glucocorticoid-induced stimulation of the mesencephalic dopaminergic transmission, one of the principal neural substrates of reward. It is proposed that the rewarding effects of glucocorticoids play the role of counteracting the aversive effects of external aggressions, allowing a better coping with threatening situations. However, a sustained increase in the secretion of these hormones, or an hypersensitivity to their rewarding effects, could determine reward-related pathologies, such as a predisposed state to develop drug-abuse. In conclusion, through their reward-related effects, glucocorticoids may play a key role in tuning adaptation to stress and in determining reward-related behavioral pathologies.

The acoustic startle response in rats--circuits mediating evocation, inhibition and potentiation

This review describes the neuronal mechanisms underlying the mediation and modulation of the acoustic startle response (ASR) in rats. The combination of anatomical, physiological and behavioral methods has identified pathways which mediate and modulate the ASR. The ASR is mediated by a relatively simple, oligosynaptic pathway located in the lower brainstem which activates spinal and cranial motor neurons. An important element of the pathway which mediates the ASR is the caudal nucleus of the pontine reticular formation (PnC). Interestingly, this nucleus is also the target of input from various brain nuclei which are involved in the modulation (e.g. fear-potentiation, sensitization, habituation, prepulse inhibition and pleasure-attenuation) of the ASR. Hence, the PnC can be described as a sensorimotor interface, where the transition of sensory input into the motor output can be directly influenced by excitatory or inhibitory afferents. On the basis of these facts we conclude that the ASR may be a valuable model for the study of general principles of sensorimotor-motivational information processing at the behavioral and neurophysiological level in mammals.

Response-reinforcement learning is dependent on N-methyl-D-aspartate receptor activation in the nucleus accumbens core

The nucleus accumbens, a site within the ventral striatum, is best known for its prominent role in mediating the reinforcing effects of drugs of abuse such as cocaine, alcohol, and nicotine. Indeed, it is generally believed that this structure subserves motivated behaviors, such as feeding, drinking, sexual behavior, and exploratory locomotion, which are elicited by natural rewards or incentive stimuli. A basic rule of positive reinforcement is that motor responses will increase in magnitude and vigor if followed by a rewarding event. It is likely, therefore, that the nucleus accumbens may serve as a substrate for reinforcement learning. However, there is surprisingly little information concerning the neural mechanisms by which appetitive responses are learned. In the present study, we report that treatment of the nucleus accumbens core with the selective competitive N-methyl-D-aspartate (NMDA) antagonist 2-amino-5-phosphonopentanoic acid (AP-5; 5 nmol/0.5 microl bilaterally) impairs response-reinforcement learning in the acquisition of a simple lever-press task to obtain food. Once the rats learned the task, AP-5 had no effect, demonstrating the requirement of NMDA receptor-dependent plasticity in the early stages of learning. Infusion of AP-5 into the accumbens shell produced a much smaller impairment of learning. Additional experiments showed that AP-5 core-treated rats had normal feeding and locomotor responses and were capable of acquiring stimulus-reward associations. We hypothesize that stimulation of NMDA receptors within the accumbens core is a key process through which motor responses become established in response to reinforcing stimuli. Further, this mechanism, may also play a critical role in the motivational and addictive properties of drugs of abuse.

Neuroscience and appetitive behavior research: 25 years

Neuroscience techniques have made major contributions to the understanding of appetitive behavior. Highlights in six areas are summarized to illustrate progress during the 25 years of the Columbia Appetitive Behavior Seminar: (1) discovery of angiotensin and aldosterone in the control of thirst and salt appetite; (2) electrophysiological decoding of chemoreceptive information in the brain; (3) a new foundation in the hypothalamus built on peptides, such as neuropeptide Y and galanin, interacting with monoamines and steroids in the control of appetite for macronutrients; (4) discovery of numerous peptides that mediate and integrate satiety, such as cholecystokinin, insulin, leptin and enterostatin, and other systems that suppress eating during illness; (5) better understanding of appetite suppressant drugs, and (6) exploration of a circuit that translates hypothalamic signals into behavioral action through connections to brainstem reflex arcs and forebrain instrumental response systems.

The nucleus accumbens and learning and memory

Recent research on the nucleus accumbens (NA) indicates that this brain region is involved in learning and memory processes in a way that is separable from its other well-known roles in behavior, such as motivation, reward, and locomotor activity. These findings have suggested that 1) the NA may be involved in declarative, or hippocampal formation-dependent learning and memory, and not in several other non-declarative forms of learning and memory, and 2) the NA may be selectively involved in certain stages of learning and memory. These characteristics suggest that the NA may be part of a larger striatal system which subserves acquisition and consolidation, but is not a site of long-term storage, of different forms of learning and memory.

A postsynaptic interaction between dopamine D1 and NMDA receptors promotes presynaptic inhibition in the rat nucleus accumbens via adenosine release

The mechanism underlying dopamine D1 receptor-mediated attenuation of glutamatergic synaptic input to nucleus accumbens (NAcc) neurons was investigated in slices of rat forebrain, using whole-cell patch-clamp recording. The depression by dopamine of EPSCs evoked by single-shock cortical stimulation was stimulus-dependent. Synaptic activation of NMDA-type glutamate receptors was critical for this effect, because dopamine-induced EPSC depressions were blocked by the competitive NMDA receptor antagonist D/L-2-amino-5-phosphonopentanoate (AP5). Application of NMDA also depressed the EPSC, and both this effect and the dopamine depressions were blocked by the A1 receptor antagonist 8-cyclopentyl-1,3-dipropylxanthine (DPCPX), implicating adenosine release in the EPSC depression. A1 receptor agonists also depressed EPSCs by a presynaptic action, causing increased paired-pulse facilitation, but this was insensitive to AP5. Activation of D1 receptors enhanced both postsynaptic inward currents evoked by NMDA application and the isolated NMDA receptor-mediated component of synaptic transmission. The biochemical processes underlying the dopamine-induced EPSC depression did not involve either protein kinase A or the production of cAMP and its metabolites, because this effect was resistant to the protein kinase inhibitors H89 and H7 and the cAMP-specific phosphodiesterase inhibitor rolipram. We conclude that activation of postsynaptic D1 receptors enhances the synaptic activation of NMDA receptors in nucleus accumbens neurons, thereby promoting a transsynaptic feedback inhibition of glutamatergic synaptic transmission via release of adenosine. Unusually for D1 receptors, this phenomenon occurs independently of adenylyl cyclase stimulation. This process may contribute to the locomotor stimulant action of dopaminergic agents in the NAcc.

The neural basis of associative reward learning in honeybees

Appetitive learning of food-predicting stimuli, an essential part of foraging behavior in honeybees, follows the rules of associative learning. In the learning of odors as reward-predicting stimuli, an individual neuron, one of a small group of large ascending neurons that serve principal brain neuropiles, mediates the reward and has experience-dependent response properties. This implies that this neuron functions as an integral part of associative memory, might underlie more complex features of learning, and could participate in the implementation of learning rules. Moreover, its structural properties suggest that it organizes the interaction of functionally different neural nets during learning and experience-dependent behavior.

Dopamine neurons and their role in reward mechanisms

Information related to rewards is processed by a limited number of brain structures. Recent studies have demonstrated that dopamine neurons respond to appetitive events, such as primary rewards and reward-predicting stimuli. Rather than responding unconditionally, these neurons signal deviations from the prediction of future appetitive events. These reward-related responses correspond formally to concepts of behavioral and computational learning theories and may thus constitute teaching signals for appetitive learning.