Opposing roles of D1 and D2 receptors in appetitive conditioning

Pharmacological modulation of dopamine D1 and D2 receptors reveals distinct neural networks related to probabilistic learning in non-human primates

The neurotransmitter dopamine (DA) has a multifaceted role in healthy and disordered brains through its action on multiple subtypes of dopaminergic receptors. How modulation of these receptors controls behavior by altering connectivity across intrinsic brain-wide networks remains elusive. Here we performed parallel behavioral and resting-state functional MRI experiments after administration of two different DA receptor antagonists in macaque monkeys. Systemic administration of SCH-23390 (D1 antagonist) disrupted probabilistic learning when subjects had to learn new stimulus-reward associations and diminished functional connectivity (FC) in cortico-cortical and fronto-striatal connections. By contrast, haloperidol (D2 antagonist) improved learning and broadly enhanced FC in cortical connections. Further comparison between the effect of SCH-23390/haloperidol on behavioral and resting-state FC revealed specific cortical and subcortical networks associated with the cognitive and motivational effects of DA, respectively. Thus, we reveal the distinct brain-wide networks that are associated with the dopaminergic control of learning and motivation via DA receptors.

Blocking D2/D3 dopamine receptors in male participants increases volatility of beliefs when learning to trust others

The ability to learn about other people is crucial for human social functioning. Dopamine has been proposed to regulate the precision of beliefs, but direct behavioural evidence of this is lacking. In this study, we investigate how a high dose of the D2/D3 dopamine receptor antagonist sulpiride impacts learning about other people's prosocial attitudes in a repeated Trust game. Using a Bayesian model of belief updating, we show that in a sample of 76 male participants sulpiride increases the volatility of beliefs, which leads to higher precision weights on prediction errors. This effect is driven by participants with genetically conferred higher dopamine availability (Taq1a polymorphism) and remains even after controlling for working memory performance. Higher precision weights are reflected in higher reciprocal behaviour in the repeated Trust game but not in single-round Trust games. Our data provide evidence that the D2 receptors are pivotal in regulating prediction error-driven belief updating in a social context.

Critical periods when dopamine controls behavioral responding during Pavlovian learning

RATIONALE

Learning the association between rewards and predictive cues is critical for appetitive behavioral responding. The mesolimbic dopamine system is thought to play an integral role in establishing these cue-reward associations. The dopamine response to cues can signal differences in reward value, though this emerges only after significant training. This suggests that the dopamine system may differentially regulate behavioral responding depending on the phase of training.

OBJECTIVES

The purpose of this study was to determine whether antagonizing dopamine receptors elicited different effects on behavior depending on the phase of training or the type of Pavlovian task.

METHODS

Separate groups of male rats were trained on Pavlovian tasks in which distinct audio cues signaled either differences in reward size or differences in reward rate. The dopamine receptor antagonist flupenthixol was systemically administered prior to either the first ten sessions of training (acquisition phase) or the second ten sessions of training (expression phase), and we monitored the effect of these manipulations for an additional ten training sessions.

RESULTS

We identified acute effects of dopamine receptor antagonism on conditioned responding, the latency to respond, and post-reward head entries in both Pavlovian tasks. Interestingly, dopamine receptor antagonism during the expression phase produced persistent deficits in behavioral responding only in rats trained on the reward size Pavlovian task.

CONCLUSIONS

Together, our results illustrate that dopamine's control over behavior in Pavlovian tasks depends upon one's prior training experience and the information signaled by the cues.

Striatal dopamine signals are region specific and temporally stable across action-sequence habit formation

Habits are automatic, inflexible behaviors that develop slowly with repeated performance. Striatal dopamine signaling instantiates this habit-formation process, presumably region specifically and via ventral-to-dorsal and medial-to-lateral signal shifts. Here, we quantify dopamine release in regions implicated in these presumed shifts (ventromedial striatum [VMS], dorsomedial striatum [DMS], and dorsolateral striatum [DLS]) in rats performing an action-sequence task and characterize habit development throughout a 10-week training. Surprisingly, all regions exhibited stable dopamine dynamics throughout habit development. VMS and DLS signals did not differ between habitual and non-habitual animals, but DMS dopamine release increased during action-sequence initiation and decreased during action-sequence completion in habitual rats, whereas non-habitual rats showed opposite effects. Consistently, optogenetic stimulation of DMS dopamine release accelerated habit formation. Thus, we demonstrate that dopamine signals do not shift regionally during habit formation and that dopamine in DMS, but not VMS or DLS, determines habit bias, attributing "habit functions" to a region previously associated exclusively with non-habitual behavior.

Dopamine D1 and D2 Receptors Are Important for Learning About Neutral-Valence Relationships in Sensory Preconditioning

Dopamine neurotransmission has been ascribed multiple functions with respect to both motivational and associative processes in reward-based learning, though these have proven difficult to tease apart. In order to better describe the role of dopamine in associative learning, this series of experiments examined the potential of dopamine D1- and D2-receptor antagonism (or combined antagonism) to influence the ability of rats to learn neutral valence stimulus-stimulus associations. Using a sensory preconditioning task, rats were first exposed to pairings of two neutral stimuli (S2-S1). Subsequently, S1 was paired with a mild foot-shock and resulting fear to both S1 (directly conditioned) and S2 (preconditioned) was examined. Initial experiments demonstrated the validity of the procedure in that measures of sensory preconditioning were shown to be contingent on pairings of the two sensory stimuli. Subsequent experiments indicated that systemic administration of dopamine D1- or D2-receptor antagonists attenuated learning when administered prior to S2-S1 pairings. However, the administration of a more generic D1R/D2R antagonist was without effect. These effects remained constant regardless of the affective valence of the conditioning environment and did not differ between male and female rats. The results are discussed in the context of recent suggestions that dopaminergic systems encode more than a simple reward prediction error, and provide potential avenues for future investigation.

The role of the nucleus accumbens in learned approach behavior diminishes with training

Nucleus accumbens dopamine plays a key role in reward-directed approach. Past findings suggest that dopamine's role in the expression of learned behavior diminishes with extended training. However, little is known about the central substrates that mediate the shift to dopamine-independent reward approach. In the present study, rats approached and inserted the head into a reward compartment in response to a cue signaling food delivery. On days 4 and 5 of 28-trial-per-day sessions, D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390) infused to the NAc core reduced the probability and speed of cued approach. The disruptive effect of D1 receptor blockade was specific to the nucleus accumbens core and not seen with drug infusions to nearby dopamine target regions. In rats that received drug infusions after extended training (days 10 or 11), accumbens core D1 receptor blockade produced little effect on the expression of the same behavior. These results could have been due to a continued accumbens mediation of cued approach even after the behavior had become independent of accumbens D1 receptors. However, accumbens core ionotropic glutamate receptor blockade disrupted cued approach during early but not late stages of training, similar to the effects of D1 antagonist infusions. The results suggest that with extended training, a nucleus accumbens D1-dependent behavior becomes less dependent not only on nucleus accumbens D1 transmission but also on excitatory transmission in the nucleus accumbens. These findings fill an important gap in a growing literature on reorganization of striatal function over the course of training.

Dopamine signals related to appetitive and aversive events in paradigms that manipulate reward and avoidability

Using environmental cues to acquire good and avoid harmful things is critical for survival. Rewarding and aversive outcomes both drive behavior through reinforcement learning and sometimes occur together in the environment, but it remains unclear how these signals are encoded within the brain and if signals for positive and negative reinforcement are encoded similarly. Recent studies demonstrate that the dopaminergic system and interconnected brain regions process both positive and negative reinforcement necessary for approach and avoidance behaviors, respectively. Here, we review these data with a special focus on behavioral paradigms that manipulate both expected reward and the avoidability of aversive events to reveal neural correlates related to value, prediction error encoding, motivation, and salience.

Differential involvement of dopamine receptor subtypes in the acquisition of Pavlovian sign-tracking and goal-tracking responses

RATIONALE

Previous work has identified that different forms of Pavlovian conditioned approach, sign-tracking and goal-tracking, are governed by distinct neurochemical mechanisms when compared in animals predisposed to learning one form vs. the other.

OBJECTIVES

The present study aimed to investigate whether these are also neurochemically distinct processes in a population of animals capable of developing either response when this is manipulated via the use of distinct conditioned stimuli (CS).

METHODS

Rats were trained on one of two Pavlovian conditioning procedures in which the CS was either a lever, which elicits sign-tracking, or an auditory click, which elicits goal-tracking. The differential involvement of dopamine D1- and D2-receptors (D1R; D2R) in the acquisition of approach types was investigated via systemic administration of antagonists selective to one or both receptor subtypes during Pavlovian training.

RESULTS

Results indicate that dopaminergic signalling is important for the acquisition of both sign-tracking and goal-tracking responses. However, whilst development of sign-tracking to a lever depends on activity at both D1R and D2R, development of goal-tracking in response to a click was shown to depend only on activity at D1R.

CONCLUSIONS

We suggest that the importance of D1R activity in both sign- and goal-tracking acquisition reflects a general role in learning Pavlovian associations, which aligns with data implicating dopamine in prediction error processes. In contrast, the selective involvement of D2R activity in sign-tracking acquisition may reflect its importance in motivational processes such as incentive salience attribution.

Cell type-specific activation of mitogen-activated protein kinase in D1 receptor-expressing neurons of the nucleus accumbens potentiates stimulus-reward learning in mice

Medium spiny neurons (MSN) in the nucleus accumbens (NAc) are a fundamental component of various aspects of motivated behavior. Although mitogen-activated protein kinase (MAPK) signaling plays a crucial role in several types of learning, the cell type-specific role of MAPK pathway in stimulus-reward learning and motivation remains unclear. We herein investigated the role of MAPK in accumbal MSNs in reward-associated learning and memory. During the acquisition of Pavlovian conditioning, the number of phosphorylated MAPK1/3-positive cells was increased significantly and exclusively in the NAc core by 7-days of extensive training. MAPK signaling in the respective D1R- and D2R-MSNs was manipulated by transfecting an adeno-associated virus (AAV) plasmid into the NAc of Drd1a-Cre and Drd2-Cre transgenic mice. Potentiation of MAPK signaling shifted the learning curve of Pavlovian conditioning to the left only in Drd1a-Cre mice, whereas such manipulation in D2R-MSNs had negligible effects. In contrast, MAPK manipulation in D2R-MSNs of the NAc core significantly increased motivation for food rewards as found in Drd1a-Cre mice. These results suggest that MAPK signaling in the D1R-MSNs of NAc core plays an important role in stimulus-reward learning, while MAPK signaling in both D1R- and D2R-MSNs is involved in motivation for natural rewards.

Circuit Mechanisms of Sensorimotor Learning

The relationship between the brain and the environment is flexible, forming the foundation for our ability to learn. Here we review the current state of our understanding of the modifications in the sensorimotor pathway related to sensorimotor learning. We divide the process into three hierarchical levels with distinct goals: (1) sensory perceptual learning, (2) sensorimotor associative learning, and (3) motor skill learning. Perceptual learning optimizes the representations of important sensory stimuli. Associative learning and the initial phase of motor skill learning are ensured by feedback-based mechanisms that permit trial-and-error learning. The later phase of motor skill learning may primarily involve feedback-independent mechanisms operating under the classic Hebbian rule. With these changes under distinct constraints and mechanisms, sensorimotor learning establishes dedicated circuitry for the reproduction of stereotyped neural activity patterns and behavior.

Toward isolating the role of dopamine in the acquisition of incentive salience attribution

Stimulus-reward learning has been heavily linked to the reward-prediction error learning hypothesis and dopaminergic function. However, some evidence suggests dopaminergic function may not strictly underlie reward-prediction error learning, but may be specific to incentive salience attribution. Utilizing a Pavlovian conditioned approach procedure consisting of two stimuli that were equally reward-predictive (both undergoing reward-prediction error learning) but functionally distinct in regard to incentive salience (levers that elicited sign-tracking and tones that elicited goal-tracking), we tested the differential role of D1 and D2 dopamine receptors and nucleus accumbens dopamine in the acquisition of sign- and goal-tracking behavior and their associated conditioned reinforcing value within individuals. Overall, the results revealed that both D1 and D2 inhibition disrupted performance of sign- and goal-tracking. However, D1 inhibition specifically prevented the acquisition of sign-tracking to a lever, instead promoting goal-tracking and decreasing its conditioned reinforcing value, while neither D1 nor D2 signaling was required for goal-tracking in response to a tone. Likewise, nucleus accumbens dopaminergic lesions disrupted acquisition of sign-tracking to a lever, while leaving goal-tracking in response to a tone unaffected. Collectively, these results are the first evidence of an intraindividual dissociation of dopaminergic function in incentive salience attribution from reward-prediction error learning, indicating that incentive salience, reward-prediction error, and their associated dopaminergic signaling exist within individuals and are stimulus-specific. Thus, individual differences in incentive salience attribution may be reflective of a differential balance in dopaminergic function that may bias toward the attribution of incentive salience, relative to reward-prediction error learning only.

Safety out of control: dopamine and defence

We enjoy a sophisticated understanding of how animals learn to predict appetitive outcomes and direct their behaviour accordingly. This encompasses well-defined learning algorithms and details of how these might be implemented in the brain. Dopamine has played an important part in this unfolding story, appearing to embody a learning signal for predicting rewards and stamping in useful actions, while also being a modulator of behavioural vigour. By contrast, although choosing correct actions and executing them vigorously in the face of adversity is at least as important, our understanding of learning and behaviour in aversive settings is less well developed. We examine aversive processing through the medium of the role of dopamine and targets such as D2 receptors in the striatum. We consider critical factors such as the degree of control that an animal believes it exerts over key aspects of its environment, the distinction between 'better' and 'good' actual or predicted future states, and the potential requirement for a particular form of opponent to dopamine to ensure proper calibration of state values.

Dissociable hippocampal and amygdalar D1-like receptor contribution to discriminated Pavlovian conditioned approach learning

Pavlovian conditioning is an elementary form of reward-related behavioral adaptation. The mesolimbic dopamine system is widely considered to mediate critical aspects of reward-related learning. For example, initial acquisition of positively-reinforced operant behavior requires dopamine (DA) D1 receptor (D1R) activation in the basolateral amygdala (BLA), central nucleus of the amygdala (CeA), and the ventral subiculum (vSUB). However, the role of D1R activation in these areas on appetitive, non-drug-related, Pavlovian learning is not currently known. In separate experiments, microinfusions of the D1-like receptor antagonist SCH-23390 (3.0 nmol/0.5 μL per side) into the amygdala and subiculum preceded discriminated Pavlovian conditioned approach (dPCA) training sessions. D1-like antagonism in all three structures impaired the acquisition of discriminated approach, but had no effect on performance after conditioning was asymptotic. Moreover, dissociable effects of D1-like antagonism in the three structures on components of discriminated responding were obtained. Lastly, the lack of latent inhibition in drug-treated groups may elucidate the role of D1-like in reward-related Pavlovian conditioning. The present data suggest a role for the D1 receptors in the amygdala and hippocampus in learning the significance of conditional stimuli, but not in the expression of conditional responses.

Microstructural analysis of rat ethanol and water drinking patterns using a modified operant self-administration model

BACKGROUND

Ethanol drinking pattern has emerged as an important factor in the development, maintenance, and health consequences of alcohol use disorders in humans. The goal of these studies was to further our understanding of this important factor through refinement of an operant rodent model of ethanol consumption capable of drinking pattern microstructural analysis. We evaluated measures of total consumption, appetitive behavior, and drinking microstructure for ethanol and water at baseline and assessed alterations induced by two treatments previously shown to significantly alter gross ethanol appetitive and consummatory behaviors in opposing directions.

METHODS

Male Long-Evans rats were trained on an FR1 operant paradigm which allowed for continuous liquid access until an 8 second pause in consumption resulted in termination of liquid access. Total appetitive and consummatory behaviors were assessed in addition to microstructural drinking pattern for both ethanol and water during a five day baseline drinking period, after chronic intermittent ethanol vapor exposure, and following administration of a cannabinoid receptor antagonist SR141716a.

RESULTS

As in previous operant studies, ethanol vapor exposure resulted in increases in ethanol-directed responding, total consumption, and rate of intake. Further, striking differential alterations to ethanol and water bout size, duration, and lick pattern occurred consistent with alterations in hedonic evaluation. Vapor additionally specifically reduced the number of ethanol-directed lever presses which did not result in subsequent consumption. SR141716a administration reversed many of these effects.

CONCLUSIONS

The addition of microstructural analysis to operant self-administration by rodents provides a powerful and translational tool for the detection of specific alterations in ethanol drinking pattern which may enable insights into neural mechanisms underlying specific components of drug consumption.

Parallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards

The basal ganglia control body movements, value processing and decision-making. Many studies have shown that the inputs and outputs of each basal ganglia structure are topographically organized, which suggests that the basal ganglia consist of separate circuits that serve distinct functions. A notable example is the circuits that originate from the rostral (head) and caudal (tail) regions of the caudate nucleus, both of which target the superior colliculus. These two caudate regions encode the reward values of visual objects differently: flexible (short-term) values by the caudate head and stable (long-term) values by the caudate tail. These value signals in the caudate guide the orienting of gaze differently: voluntary saccades by the caudate head circuit and automatic saccades by the caudate tail circuit. Moreover, separate groups of dopamine neurons innervate the caudate head and tail and may selectively guide the flexible and stable learning/memory in the caudate regions. Studies focusing on manual handling of objects also suggest that rostrocaudally separated circuits in the basal ganglia control the action differently. These results suggest that the basal ganglia contain parallel circuits for two steps of goal-directed behaviour: finding valuable objects and manipulating the valuable objects. These parallel circuits may underlie voluntary behaviour and automatic skills, enabling animals (including humans) to adapt to both volatile and stable environments. This understanding of the functions and mechanisms of the basal ganglia parallel circuits may inform the differential diagnosis and treatment of basal ganglia disorders.

The form of a conditioned stimulus can influence the degree to which it acquires incentive motivational properties

There is considerable individual variation in the extent to which food- and drug-associated cues (conditioned stimuli, CSs) acquire incentive salience, as indicated by whether they elicit approach towards them, and/or act as conditioned reinforcers. Here we asked whether this variation is influenced by properties of the CS itself. In rats, we assessed both the attractiveness and conditioned reinforcing properties of two CSs: a manipulable lever CS versus an auditory (tone) CS. There was considerable individual variation in the extent to which a lever CS acquired incentive motivational properties, as indicated by whether it became attractive (evoked a sign-tracking or goal-tracking conditioned response) or acted as a conditioned reinforcer. However, with a tone CS all rats learned a goal-tracking response, and the tone CS was an equally effective conditioned reinforcer in sign-trackers and goal-trackers. Even when presented in compound (a lever-tone CS), the two elements of the compound differentially acquired motivational properties. In contrast, amphetamine and stress potentiated the conditioned reinforcing properties of both visual and auditory CSs similarly in rats that primarily sign-tracked or goal-tracked. We conclude that variation in the to the ability of CSs to acquire incentive salience, and thus their ability to act as incentive stimuli capable of motivating behavior, is determined in part by properties of the CS itself.

Order of exposure to pleasant and unpleasant odors affects autonomic nervous system response

When mammals are exposed to an odor, that odor is expected to elicit a physiological response in the autonomic nervous system. An unpleasant aversive odor causes non-invasive stress, while a pleasant odor promotes healing and relaxation in mammals. We hypothesized that pleasant odors might reduce a stress response previously induced by an aversive predator odor. Rats were thus exposed to pleasant and unpleasant odors in different orders to determine whether the order of odor exposure had an effect on the physiological response in the autonomic nervous system. The first trial examined autonomic nerve activity via sympathetic and parasympathetic nerve response while the second trial examined body temperature response. Initial exposure to a pleasant odor elicited a positive response and secondary exposure to an unpleasant odor elicited a negative response, as expected. However, we found that while initial exposure to an unpleasant odor elicited a negative stress response, subsequent secondary exposure to a pleasant odor not only did not alleviate that negative response, but actually amplified it. These findings were consistent for both the autonomic nerve activity response trial and the body temperature response trial. The trial results suggest that exposure to specific odors does not necessarily result in the expected physiological response and that the specific order of exposure plays an important role. Our study should provide new insights into our understanding of the physiological response in the autonomic nervous system related to odor memory and discrimination and point to areas that require further research.

Effect of dopamine and serotonin receptor antagonists on fencamfamine-induced abolition of latent inhibition

The purpose of this investigation was to verify the role of dopamine and serotonin receptors in the effect of fencamfamine (FCF) on latent inhibition. FCF is a psychomotor stimulant with an indirect dopaminergic action. Latent inhibition is a model of attention. Latent inhibition is blocked by dopaminergic agents and facilitated by dopamine receptor agonists. FCF has been shown to abolish latent inhibition. The serotonergic system may also participate in the neurochemical mediation of latent inhibition. The selective dopamine D(1) receptor antagonist SCH 23390 (7-chloro-3-methyl-1-phenyl-1,2,4,5-tetrahydro-3-benzazepin-8-ol), D(2) receptor antagonists pimozide (PIM) and methoclopramide (METH), and serotonin 5-HT(2A/C) receptor antagonist ritanserin (RIT) were used in the present study. Latent inhibition was evaluated using a conditioned emotional response procedure. Male Wistar rats that were water-restricted were subjected to a three-phase procedure: preexposure to a tone, tone-shock conditioning, and a test of the effect of the tone on licking frequency. All of the drugs were administered before the preexposure and conditioning phases. The results showed that FCF abolished latent inhibition, and this effect was clearly antagonized by PIM and METH and moderately attenuated by SCH 23390. At the doses used in the present study, RIT pretreatment did not affect latent inhibition and did not eliminate the effect of FCF, suggesting that the FCF-induced abolition of latent inhibition is not mediated by serotonin 5-HT(2A/C) receptors. These results suggest that the effect of FCF on latent inhibition is predominantly related to dopamine D(2) receptors and that dopamine D(2) receptors participate in attention processes.

Dopamine D1 receptor involvement in latent inhibition and overshadowing

Latent inhibition (LI) manifests as poorer conditioning to a stimulus that has previously been experienced without consequence. There is good evidence of dopaminergic modulation of LI, as the effect is reliably disrupted by the indirect dopamine (DA) agonist amphetamine. The disruptive effects of amphetamine on LI are reversed by both typical and atypical antipsychotics, which on their own are able to facilitate LI. However, the contribution of different DA receptors to these effects is poorly understood. Amphetamine effects on another stimulus selection procedure, overshadowing, have been suggested to be D1-mediated. Thus, in the current experiments, we systematically investigated the role of D1 receptors in LI. First, we tested the ability of the full D1 agonist SKF 81297 to abolish LI and compared the effects of this drug on LI and overshadowing. Subsequently, we examined whether the D1 antagonist SCH 23390 can lead to the emergence of LI under conditions that do not produce the effect in normal animals (weak pre-exposure). Finally, we tested the ability of SCH 23390 to block amphetamine-induced disruption of LI. We found little evidence that direct stimulation of D1 receptors abolishes LI (although there was some attenuation of LI at 0.4 mg/kg SKF 81297). Similarly, SCH 23390 failed to enhance LI. However, SCH 23390 did block amphetamine-induced disruption of LI. These data indicate that, while LI may be unaffected by selective manipulation of activity at D1 receptors, the effects of amphetamine on LI are to some extent dependent on actions at D1 receptors.

Dissociating the cognitive effects of levodopa versus dopamine agonists in a neurocomputational model of learning in Parkinson's disease

BACKGROUND/AIMS

Levodopa and dopamine agonists have different effects on the motor, cognitive, and psychiatric aspects of Parkinson's disease (PD).

METHODS

Using a computational model of basal ganglia (BG) and prefrontal cortex (PFC) dopamine, we provide a theoretical synthesis of the dissociable effects of these dopaminergic medications on brain and cognition. Our model incorporates the findings that levodopa is converted by dopamine cells into dopamine, and thus activates prefrontal and striatal D(1) and D(2) dopamine receptors, whereas antiparkinsonian dopamine agonists directly stimulate D(2) receptors in the BG and PFC (although some have weak affinity to D(1) receptors).

RESULTS

In agreement with prior neuropsychological studies, our model explains how levodopa enhances, but dopamine agonists impair or have no effect on, stimulus-response learning and working memory.

CONCLUSION

Our model explains how levodopa and dopamine agonists have differential effects on motor and cognitive processes in PD.

Dose dependent dopaminergic modulation of reward-based learning in Parkinson's disease

Learning to select optimal behavior in new and uncertain situations is a crucial aspect of living and requires the ability to quickly associate stimuli with actions that lead to rewarding outcomes. Mathematical models of reinforcement-based learning to select rewarding actions distinguish between (1) the formation of stimulus-action-reward associations, such that, at the instant a specific stimulus is presented, it activates a specific action, based on the expectation that that particular action will likely incur reward (or avoid punishment); and (2) the comparison of predicted and actual outcomes to determine whether the specific stimulus-action association yielded the intended outcome or needs revision. Animal electrophysiology and human fMRI studies converge on the notion that dissociable neural circuitries centered on the striatum are differentially involved in different components of this learning process. The modulatory role of dopamine (DA) in these respective circuits and component processes is of particular relevance to the study of reward-based learning in patients diagnosed with Parkinson's disease (PD). Here we show that the first component process, learning to predict which actions yield reward (supported by the anterior putamen and associated motor circuitry) is impaired when PD patients are taken off their DA medication, whereas DA medication has no systematic effects on the second processes, outcome evaluation (supported by caudate and ventral striatum and associated frontal circuitries). However, the effects of DA medication on these processes depend on dosage, with larger daily doses leading to a decrease in predictability of stimulus-action-reward relations and increase in reward-prediction errors.

Molecular substrates of action control in cortico-striatal circuits

The purpose of this review is to describe the molecular mechanisms in the striatum that mediate reward-based learning and action control during instrumental conditioning. Experiments assessing the neural bases of instrumental conditioning have uncovered functional circuits in the striatum, including dorsal and ventral striatal sub-regions, involved in action-outcome learning, stimulus-response learning, and the motivational control of action by reward-associated cues. Integration of dopamine (DA) and glutamate neurotransmission within these striatal sub-regions is hypothesized to enable learning and action control through its role in shaping synaptic plasticity and cellular excitability. The extracellular signal regulated kinase (ERK) appears to be particularly important for reward-based learning and action control due to its sensitivity to combined DA and glutamate receptor activation and its involvement in a range of cellular functions. ERK activation in striatal neurons is proposed to have a dual role in both the learning and performance factors that contribute to instrumental conditioning through its regulation of plasticity-related transcription factors and its modulation of intrinsic cellular excitability. Furthermore, perturbation of ERK activation by drugs of abuse may give rise to behavioral disorders such as addiction.

Differential effects of dopamine receptor D1-type and D2-type antagonists and phase of the estrous cycle on social learning of food preferences, feeding, and social interactions in mice

The neurobiological bases of social learning, by which an animal can 'exploit the expertise of others' and avoid the disadvantages of individual learning, are only partially understood. We examined the involvement of the dopaminergic system in social learning by administering a dopamine D1-type receptor antagonist, SCH23390 (0.01, 0.05, and 0.1 mg/kg), or a D2-type receptor antagonist, raclopride (0.1, 0.3, and 0.6 mg/kg), to adult female mice prior to socially learning a food preference. We found that while SCH23390 dose-dependently inhibited social learning without affecting feeding behavior or the ability of mice to discriminate between differently flavored diets, raclopride had the opposite effects, inhibiting feeding but leaving social learning unaffected. We showed that food odor, alone or in a social context, was insufficient to induce a food preference, proving the specifically social nature of this paradigm. The estrous cycle also affected social learning, with mice in proestrus expressing the socially acquired food preference longer than estrous and diestrous mice. This suggests gonadal hormone involvement, which is consistent with known estrogenic regulation of female social behavior and estrogen receptor involvement in social learning. Furthermore, a detailed ethological analysis of the social interactions during which social learning occurs showed raclopride- and estrous phase-induced changes in agonistic behavior, which were not directly related to effects on social learning. Overall, these results suggest a differential involvement of the D1-type and D2-type receptors in the regulation of social learning, feeding, and agonistic behaviors that are likely mediated by different underlying states.

The canary in the coalmine: the sensitivity of mesolimbic dopamine to environmental adversity during development

The hypothalamic-pituitary-adrenal axis has been the focus of extensive research with regard to the phenotypic plasticity this system shows in response to environmental influences on mammalian development. This review proposes that the mesolimbic dopamine system is similarly reactive to indicators of environmental adversity during development. Physical, physiological, and toxicological stressors encountered during perinatal development have been routinely demonstrated to affect dopamine neurophysiology, most likely through consequent exposure to maternal glucocorticoids or a reduction in oxygen supply. However, findings remain inconsistent with regard to the nature of impact these events have on the dopamine system. Both hyper- and hypo-dopaminergic changes have been noted. This review argues that the directionality of change is a function of chronicity and severity of the insult, and that both resultant phenotypes are adaptive developmental responses, despite their potential for conferring vulnerability for psychopathology in humans.

Dopamine agonists increase perseverative instrumental responses but do not restore habit formation in a rat model of Parkinsonism

Dopamine (DA) deafferentation of the dorsolateral striatum has been shown to prevent habit development, leaving instrumental behavior under action-outcome control that is persistently sensitive to modification of the motivational value of the reward. The present experiment further explored the basis of this dysfunction by examining the ability of intrastriatal DA agonist injections (D1 SKF 38393 or D2/D3 Quinpirole) during overtraining of a signaled instrumental task to restore habit formation in rats subjected to bilateral 6-hydroxydopamine (6-OHDA) lesions of the nigrostriatal dopaminergic pathway. Overtraining was followed by a test of goal sensitivity by satiety-specific devaluation of the reward. The results confirmed the impaired shift in performance from action to habit in control lesioned rats. However, lesioned rats repeatedly injected with quinpirole D2/D3 agonist showed an increase in non-rewarded instrumental responses (intertrials periods) during overtraining, suggesting the development of perseverative behavior. Following the procedure of devaluation, quinpirole D2/D3 agonist treatment, and to a lesser extent SKF 38393 D1 agonist, caused the persistence of sensitivity to reward devaluation, indicating clear goal-directed behavior despite extended training. This absence of restoration of habit formation by DA agonist treatment is discussed in the light of DA agonist effects in Parkinson patients.

A molecular dissociation between cued and contextual appetitive learning

In appetitive Pavlovian learning, animals learn to associate discrete cues or environmental contexts with rewarding outcomes, and these cues and/or contexts can potentiate an ongoing instrumental response for reward. Although anatomical substrates underlying cued and contextual learning have been proposed, it remains unknown whether specific molecular signaling pathways within the striatum underlie one form of learning or the other. Here, we show that while the striatum-enriched isoform of adenylyl cyclase (AC5) is required for cued appetitive Pavlovian learning, it is not required for contextual appetitive learning. Mice lacking AC5 (AC5KO) could not learn an appetitive Pavlovian learning task in which a discrete signal light predicted reward delivery, yet they could form associations between context and either natural or drug reward, which could in turn elicit Pavlovian approach behavior. However, unlike wild-type (WT) mice, AC5KO mice could not use these Pavlovian conditioned stimuli to potentiate ongoing instrumental behavior in a Pavlovian-to-instrumental transfer paradigm. These data suggest that AC5 is specifically required for learning associations between discrete cues and outcomes in which the temporal relationship between conditioned stimulus (CS) and unconditioned stimulus (US) is essential, while alternative signaling mechanisms may underlie the formation of associations between context and reward. In addition, loss of AC5 compromises the ability of both contextual and discrete cues to modulate instrumental behavior.

Dopamine receptors in the learning, memory and drug reward circuitry

As primary targets of a variety of abused drugs G-protein-coupled dopamine receptors in the brain play an important role in mediating the various drug-induced alterations in neural and psychological processes thought to underlie the transition from voluntary drug use to habitual and progressively compulsive drug-taking. This review considers the functional involvement of the five major dopamine receptor subtypes in drug reinforcement and reward and discusses the development of addiction as a series of learning transitions from initial goal-directed behaviour to pathological stimulus-response habits in which drug-seeking behaviours are automatically elicited and maintained by cues and stimuli associated with drug rewards.

Motor-skill learning in a novel running-wheel task is dependent on D1 dopamine receptors in the striatum

Evidence indicates that dopamine receptors regulate processes of procedural learning in the sensorimotor striatum. Our previous studies revealed that the indirect dopamine receptor agonist cocaine alters motor-skill learning-associated gene regulation in the sensorimotor striatum. Cocaine-induced gene regulation in the striatum is principally mediated by D1 dopamine receptors. We investigated the effects of cocaine and striatal D1 receptor antagonism on motor-skill learning. Rats were trained on a running wheel (40-60 min, 2-5 days) to learn a new motor skill, that is, the ability to control the movement of the wheel. Immediately before each training session, the animals received an injection of vehicle or cocaine (25 mg/kg, i.p.), and/or the D1 receptor antagonist SCH-23390 (0, 3, 10 microg/kg, i.p., or 0, 0.3, 1 microg, intrastriatal via chronically implanted cannula). The animal's ability to control/balance the moving wheel (wheel skill) was tested before and repeatedly after the training. Normal wheel-skill memory lasted for at least 4 weeks. Cocaine administered before the training tended to attenuate skill learning. Systemic administration of SCH-23390 alone also impaired skill learning. However, cocaine given in conjunction with the lower SCH-23390 dose (3 microg/kg) reversed the inhibition of skill learning produced by the D1 receptor antagonist, enabling intact skill performance during the whole post-training period. In contrast, when cocaine was administered with the higher SCH-23390 dose (10 microg/kg), skill performance was normalized 1-6 days after the training, but these rats lost their improved wheel skill by day 18 after the training. Similar effects were produced by SCH-23390 (0.3-1 microg) infused into the striatum. Our results indicate that cocaine interferes with normal motor-skill learning, which seems to be dependent on optimal D1 receptor signaling. Furthermore, our findings demonstrate that D1 receptors in the striatum are critical for consolidation of long-term skill memory.

Differential involvement of dopamine receptors in conditioned suppression induced by cocaine

Cocaine-paired stimuli can suppress food-reinforced operant behavior in rats, providing an animal model of conditioned drug effects. To study the neuropharmacological basis of this phenomenon, we examined the effects of various dopamine receptor antagonists on the acquisition and expression of cocaine-induced conditioned suppression in rats. Superimposed on an ongoing baseline of food-reinforced operant responding, a stimulus was paired with response-independent cocaine (3.0 mg/kg, i.v.) during each of 8 training sessions. To study acquisition, independent groups of rats were given saline, the dopamine D(1)-like receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH 23390) (0.001-0.03 mg/kg, i.p.), or the dopamine D(2)-like receptor antagonist eticlopride (0.001-0.03 mg/kg, i.p.) prior to each training session. To study expression, independent groups of rats were trained first, then given saline, SCH 23390, eticlopride, or N-[4-(4-(2-methoxyphenyl)piperazinyl)butyl]-2-naphthamide (BP 897) (a dopamine D(3) partial receptor agonist; 0.1-1.0 mg/kg, i.p.) before test sessions in which the stimulus was presented without cocaine. Pre-treatment with either SCH 23390 or eticlopride during acquisition reduced the direct suppressant effects of cocaine, but conditioning was blocked only in rats that were treated with SCH 23390 during acquisition training. Expression of conditioning was attenuated only by eticlopride. Thus, dopamine at least partially mediates both the acquisition and expression of cocaine-induced conditioned suppression, with activation of dopamine D(1)- and D(2)-like receptors underlying these respective processes.

AMPA receptor antagonists reverse effects of extended habit training on signaled food approach responding in rats

RATIONALE

Dopamine D1 receptor stimulation is critically involved in early appetitive phases of learning in various behavioral paradigms. However, extended habit training was previously shown to reduce the ability of dopamine D1 receptor antagonists such as R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH-23390) to disrupt behavioral performance.

OBJECTIVE

The present study aimed to evaluate whether coadministration of glutamate alpha-amino-3-hydroxy-5-methyl-isoxazole-4-propionic acid (AMPA) receptor antagonists restores sensitivity to acute blockade of D1 receptors.

MATERIALS AND METHODS

Adult male Wistar rats were presented with 45-mg food pellets delivered to the food tray, which was immediately preceded by a 400-ms tone (2.8 kHz, 78 dB). During each training and test session, there were 28 food-tone presentations with an average intertrial interval of 70 s, and each head entry into the food tray was recorded. Drug tests were conducted on either day 3 or 9 of the training using independent groups of animals. The main dependent variable was the number of trials during which no head-entry response was made during the 10-s period immediately after the food delivery.

RESULTS

Longer training duration enhanced the resistance of the signaled food approach behavior to extinction and to disrupting effects of supplementary food ration. Similarly, acute administration of SCH-23390 (0.04-0.16 mg/kg) dose-dependently reduced the number of omitted trials when given before the test session on day 3 but much less so when injected on day 9. AMPA receptor antagonists, NBQX (10 mg/kg) or GYKI-52466 (3-10 mg/kg), had no effects on their own but significantly enhanced the disrupting effects of SCH-23390 (0.08 and 0.16 mg/kg) when given on day 9 but not on day 3 of the training.

CONCLUSIONS

These results indicate that AMPA receptor blockade restores sensitivity to appetitive behavior-disrupting effects of SCH-23390 in subjects exposed to extended training protocol.

Dopamine D2/D3 receptors play a specific role in the reversal of a learned visual discrimination in monkeys

Converging evidence supports a role for mesocorticolimbic dopaminergic systems in a subject's ability to shift behavior in response to changing stimulus-reward contingencies. To characterize the dopaminergic mechanisms involved in this function, we quantified the effects of subtype-specific dopamine (DA) receptor antagonists on acquisition, retention, and reversal of a visual discrimination task in non-human primates (Chlorocebus aethiops sabaeus). We used a modified Wisconsin General Test Apparatus that was equipped with three food boxes, each fitted with a lid bearing a unique visual cue; one of the cues concealed a food reward, whereas the other two concealed an empty box. The monkeys were trained first to acquire a novel discrimination (eg A(+), B(-), C(-)) in a single session, before experiencing either a reversal of the discrimination (eg A(-), B(+), C(-)) or the acquisition of a completely new discrimination (eg D(+), E(-), F(-)), on the following day. Systemic administration of the D(2)/D(3) receptor antagonist raclopride (0.001-0.03 mg/kg) failed to significantly affect the performance of reversal learning when reversal sessions were run without a retention session. But, raclopride (0.03 mg/kg) significantly impaired performance under the reversal condition when reversal sessions were run right after a retention session; however, it did not affect acquisition of a novel visual discrimination. Specifically, raclopride significantly increased the number of reversal errors made before reaching the performance criterion in the reversal, but not in new learning sessions. In contrast, the D(1)/D(5) receptor antagonist SCH 23390 did not significantly modulate acquisition of a novel discrimination or reversal learning at doses (0.001-0.03 mg/kg, i.m.) that did not suppress behavior generally. In addition, none of the drug treatments affected retention of a previously learned discrimination. The results strongly suggest that D(2)/D(3) receptors, but not D(1)/D(5) receptors, selectively mediate reversal learning, without affecting the capacity to learn a new stimulus-reward association. These data support the hypothesis that phasic DA release, acting through D(2)-like receptors, mediates behavioral flexibility.

Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens

The ability to predict favorable outcomes using environmental cues is an essential part of learned behavior. Dopamine neurons in the midbrain encode such stimulus-reward relationships in a manner consistent with contemporary learning models, but it is unclear how encoding this translates into actual dopamine release in target regions. Here, we sampled dopamine levels in the rat nucleus accumbens on a rapid (100 ms) timescale using electrochemical technology during a classical conditioning procedure. Early in conditioning, transient dopamine-release events signaled a primary reward, but not predictive cues. After repeated cue-reward pairings, dopamine signals shifted in time to predictive cue onset and were no longer observed at reward delivery. In the absence of stimulus-reward conditioning, there was no shift in the dopamine signal. Consistent with proposed roles in reward prediction and incentive salience, these results indicate that rapid dopamine release provides a reward signal that is dynamically modified by associative learning.

The nucleus accumbens and Pavlovian reward learning

The ability to form associations between predictive environmental events and rewarding outcomes is a fundamental aspect of learned behavior. This apparently simple ability likely requires complex neural processing evolved to identify, seek, and use natural rewards and redirect these activities based on updated sensory information. Emerging evidence from both animal and human research suggests that this type of processing is mediated in part by the nucleus accumbens (NAc) and a closely associated network of brain structures. The NAc is required for a number of reward-related behaviors and processes specific information about reward availability, value, and context. In addition, this structure is critical for the acquisition and expression of most Pavlovian stimulus-reward relationships, and cues that predict rewards produce robust changes in neural activity in the NAc. Although processing within the NAc may enable or promote Pavlovian reward learning in natural situations, it has also been implicated in aspects of human drug addiction, including the ability of drug-paired cues to control behavior. This article provides a critical review of the existing animal and human literature concerning the role of the NAc in Pavlovian learning with nondrug rewards and considers some clinical ture concerning the role of the NAc in Pavlovian learning with nondrug implications of these findings.

A "good parent" function of dopamine: transient modulation of learning and performance during early stages of training

While extracellular dopamine (DA) concentrations are increased by a wide category of salient stimuli, there is evidence to suggest that DA responses to primary and conditioned rewards may be distinct from those elicited by other types of salient events. A reward-specific mode of neuronal responding would be necessary if DA acts to strengthen behavioral response tendencies under particular environmental conditions or to set current environmental inputs as goals that direct approach responses. As described in this review, DA critically mediates both the acquisition and expression of learned behaviors during early stages of training, however, during later stages, at least some forms of learned behavior become independent of (or less dependent upon) DA transmission for their expression.

Feeding behavior of Aplysia: a model system for comparing cellular mechanisms of classical and operant conditioning

Feeding behavior of Aplysia provides an excellent model system for analyzing and comparing mechanisms underlying appetitive classical conditioning and reward operant conditioning. Behavioral protocols have been developed for both forms of associative learning, both of which increase the occurrence of biting following training. Because the neural circuitry that mediates the behavior is well characterized and amenable to detailed cellular analyses, substantial progress has been made toward a comparative analysis of the cellular mechanisms underlying these two forms of associative learning. Both forms of associative learning use the same reinforcement pathway (the esophageal nerve, En) and the same reinforcement transmitter (dopamine, DA). In addition, at least one cellular locus of plasticity (cell B51) is modified by both forms of associative learning. However, the two forms of associative learning have opposite effects on B51. Classical conditioning decreases the excitability of B51, whereas operant conditioning increases the excitability of B51. Thus, the approach of using two forms of associative learning to modify a single behavior, which is mediated by an analytically tractable neural circuit, is revealing similarities and differences in the mechanisms that underlie classical and operant conditioning.

The serial reaction time task in the rat: effects of D1 and D2 dopamine-receptor antagonists

Sequential behaviour, probably reflecting procedural learning, has intensively been investigated in humans and monkeys using so-called serial reaction time tasks (SRTT), where serial stimuli are either presented in a random or sequential fashion. Learning of sequences is typically inferred from faster reaction times to such sequences as compared to random blocks of stimuli. Work with such tasks has shown that sequential behaviour seems to be mediated by specific brain systems, including the basal ganglia and the neurotransmitter dopamine. We have recently developed a rat version of the human serial reaction time task, in which rats have to respond to visual stimuli in one of four spatial locations by nose-poking in order to obtain food reward under a fixed ratio schedule (FR13). Here, we used a test version where random and sequential condition phases (10 min each) were alternated within-sessions. In support of our previous work, we found that well-trained (i.e. skilled) rats display superior performance under sequential than random conditions, namely, faster reaction times and higher response accuracies. Furthermore, we investigated the effects of selective dopamine-receptor blockade, by systemically administering SKF 83566, a D1 antagonist (.05-.15 mg/kg), or raclopride, a D2 antagonist (.05-.20 mg/kg), in two separate experiments. Both antagonists impaired responding to the conditioned visual stimuli in a dose-related way, i.e. they decreased, or even blocked, nose-poke rates. In those rats, which kept responding, the speeding of reaction times during sequential conditions was no longer observed with the D1 antagonist, whereas the enhancements in accuracy were preserved, or even enhanced as compared to vehicle. The D2 antagonist also impaired instrumental behaviour, but did not alter sequence effects on accuracy or reaction times. In contrast to responses to the conditioned stimuli, reaction times to the unconditioned stimuli (food pellets) were not substantially affected by either drug. These results are discussed with respect to methodological factors, and the possible role of dopamine for instrumental behaviour, in general, and sequential behaviour, in specific.

Firing Patterns of Accumbal Neurons During a Pavlovian-Conditioned Approach Task

The nucleus accumbens (NAc) is necessary for the expression of Pavlovian-conditioned approach behavior but not for the expression of instrumental behavior conditioned in sessions that set a low response requirement. Although numerous studies have characterized firing patterns of NAc neurons in relation to instrumental behavior, very little is known about how NAc neurons encode information in Pavlovian tasks. In the present study, recordings of accumbal firing patterns were made during sessions in which rats performed a Pavlovian-conditioned approach task. Most of the recorded neurons (74/83, 89%) exhibited significant responses during the conditioned stimulus (CS) presentation and/or the reward exposure. The reward responses were prevalent, predominantly inhibitory, and comparable to reward responses observed in various types of behavioral paradigms, including instrumental tasks. The CS responses could be segregated into multiple subtypes on the basis of directionality, onset latency, and duration. Several characteristics of the CS firing patterns were unique relative to cue responses observed previously during alternative types of conditioning sessions. It is possible that the novel firing patterns correspond to the differential contributions of the accumbens to Pavlovian-conditioned approach behavior and instrumentally conditioned behavior. Regardless, the novel patterns of firing add to existing evidence that characterization of accumbal firing patterns in Pavlovian tasks may provide additional information about the neurophysiological mechanisms that mediate accumbal contributions to behavior.

Dopamine D(2) receptor activation modulates perceived odor intensity

Dopaminergic modulation affects odor detection thresholds and olfactory discrimination capabilities in rats. The authors show that dopamine D(2) receptor modulation affects odor discrimination capabilities in a manner similar to the modulation of stimulus intensity. Performance in a simultaneous odor discrimination task was systematically altered by manipulations of both odorant concentration and D(2) receptor activation (agonist quinpirole, 0.025-0.5 mg/kg; antagonist spiperone, 0.5 mg/kg). Rats' discrimination performance systematically improved at higher odor concentrations. Blockade of D(2) receptors improved performance equivalent to increasing odor concentration by 2 log units, whereas activation of D(2) receptors reduced odor discrimination performance in a dose-dependent manner. Bulbar dopamine release may serve a gain control function in the olfactory system, optimizing its sensitivity to changes in the chemosensory environment.

Dopamine receptor blockade and extinction differentially affect behavioral variability

Dopamine (DA) neurons respond to unexpected food delivery and are inhibited during the omission of expected reward. DA receptor blockade mimics some, but not all, aspects of non-reward (extinction) conditions. It was therefore of interest to ask whether DA receptor blockade produces extinction-like increases in behavioral variability in addition to its well-known operant response-suppressing effects. In the current experiment, rats were trained drug-free on an operant task in which they pressed on a keyboard. Two of the keys led to food on a continuous reinforcement schedule. Both response rates and behavioral variability were measured. Test day administration of D(1) and D(2) antagonists SCH23390 and raclopride, like extinction, suppressed responding but, unlike extinction, did not lead to an increase in variability.

A mechanistic account of striatal dopamine function in human cognition: Psychopharmacological studies with cabergoline and haloperidol

The authors test a neurocomputational model of dopamine function in cognition by administering to healthy participants low doses of D2 agents cabergoline and haloperidol. The model suggests that DA dynamically modulates the balance of Go and No-Go basal ganglia pathways during cognitive learning and performance. Cabergoline impaired, while haloperidol enhanced, Go learning from positive reinforcement, consistent with presynaptic drug effects. Cabergoline also caused an overall bias toward Go responding, consistent with postsynaptic action. These same effects extended to working memory and attentional domains, supporting the idea that the basal ganglia/dopamine system modulates the updating of prefrontal representations. Drug effects interacted with baseline working memory span in all tasks. Taken together, the results support a unified account of the role of dopamine in modulating cognitive processes that depend on the basal ganglia.

Nucleus accumbens dopamine differentially mediates the formation and maintenance of monogamous pair bonds

The involvement of dopamine within the nucleus accumbens in the formation and maintenance of pair bonds was assessed in a series of experiments using the monogamous prairie vole. We show that dopamine transmission that promotes pair bond formation occurs within the rostral shell of the nucleus accumbens, but not in its core or caudal shell. Within this specific brain region, D1- and D2-like receptor activation produced opposite effects: D1-like activation prevented pair bond formation, whereas D2-like activation facilitated it. After extended cohabitation with a female, male voles showed behavior indicative of pair bond maintenance-namely, selective aggression towards unfamiliar females. These voles also showed a significant upregulation in nucleus accumbens D1-like receptors, and blockade of these receptors abolished selective aggression. Thus, neuroplastic reorganization of the nucleus accumbens dopamine system is responsible for the enduring nature of monogamous pair bonding. Finally, we show that this system may also contribute to species-specific social organization.

Reinforcement in an in vitro analog of appetitive classical conditioning of feeding behavior in Aplysia: blockade by a dopamine antagonist

In a recently developed in vitro analog of appetitive classical conditioning of feeding in Aplysia, the unconditioned stimulus (US) was electrical stimulation of the esophageal nerve (En). This nerve is rich in dopamine (DA)-containing processes, which suggests that DA mediates reinforcement during appetitive conditioning. To test this possibility, methylergonovine was used to antagonize DA receptors. Methylergonovine (1 nM) blocked the pairing-specific increase in fictive feeding that is usually induced by in vitro classical conditioning. The present results and previous observation that methylergonovine also blocks the effects of contingent reinforcement in an in vitro analog of appetitive operant conditioning suggest that DA mediates reinforcement for appetitive associative conditioning of feeding in Aplysia.

Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens

Recent research has implicated the nucleus accumbens (NAc) in consolidating recently acquired goal-directed appetitive memories, including spatial learning and other instrumental processes. However, an important but unresolved issue is whether this forebrain structure also contributes to the consolidation of fundamental forms of appetitive learning acquired by Pavlovian associative processes. In addition, although dopaminergic and glutamatergic influences in the NAc have been implicated in instrumental learning, it is unclear whether similar mechanisms operate during Pavlovian conditioning. To evaluate these issues, the effects of posttraining intra-NAc infusions of D1, D2, and NMDA receptor antagonists, as well as d-amphetamine, were determined on Pavlovian autoshaping in rats, which assesses learning by discriminated approach behavior to a visual conditioned stimulus predictive of food reward. Intracerebral infusions were given either immediately after each conditioning session to disrupt early memory consolidation or after a delay of 24 h. Findings indicate that immediate, but not delayed, infusions of both D1 (SCH 23390) and NMDA (AP-5) receptor antagonists significantly impair learning on this task. By contrast, amphetamine and the D2 receptor antagonist sulpiride were without significant effect. These findings provide the most direct demonstration to date that D1 and NMDA receptors in the NAc contribute to, and are necessary for, the early consolidation of appetitive Pavlovian learning.

Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism

Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both frontal-like and implicit learning impairments. Dopaminergic medication alleviates some cognitive deficits but impairs those that depend on intact areas of the BG, apparently due to DA “overdose.” These findings are difficult to accommodate with verbal theories of BG/DA function, owing to complexity of system dynamics: DA dynamically modulates function in the BG, which is itself a modulatory system. This article presents a neural network model that instantiates key biological properties and provides insight into the underlying role of DA in the BG during learning and execution of cognitive tasks. Specifically, the BG modulates the execution of “actions” (e.g., motor responses and working memory updating) being considered in different parts of the frontal cortex. Phasic changes in DA, which occur during error feedback, dynamically modulate the BG threshold for facilitating/suppressing a cortical command in response to particular stimuli. Reduced dynamic range of DA explains Parkinson and DA overdose deficits with a single underlying dysfunction, despite overall differences in raw DA levels. Simulated Parkinsonism and medication effects provide a theoretical basis for behavioral data in probabilistic classification and reversal tasks. The model also provides novel testable predictions for neuropsychological and pharmacological studies, and motivates further investigation of BG/DA interactions with the prefrontal cortex in working memory.

Alterations in food intake by opioid and dopamine signaling pathways between the ventral tegmental area and the shell of the nucleus accumbens

Reward is an important factor motivating food intake in satiated animals. Two sites involved in the reward response are the ventral tegmental area (VTA) and the nucleus accumbens shell region (sNAcc), between which communication is partially regulated by opioids and dopamine (DA). Previous studies have shown that the mu-opioid agonist Tyr-D-Ala-Gly-MePhe-Gly(ol)-enkephalin (DAMGO) dose-dependently enhances food intake in satiated animals when injected into either the VTA or the sNAcc. The enhanced intake elicited by DAMGO injected into the sNAcc was dose-dependently blocked by injection of naltrexone (NTX) bilaterally into the VTA, indicating an opioid-dependent signaling pathway from the sNAcc to the VTA in mediation of food intake. In the present study, we cannulated animals bilaterally in both the VTA and the sNAcc to further study the nature of opioid- and DA-dependent communication between the sites. Food intake elicited by DAMGO (2 or 5 nmol) injected unilaterally into the VTA was dose-dependently diminished by bilateral injection of NTX (2.5, 5, and 25 g/side) or the D1 antagonist SCH 23390 (3, 1, 0.3, 0.15, 0.05, and 0.015 nmol/side) into the sNAcc. When DAMGO (5 nmol) was injected into the sNAcc, the resulting food intake was decreased by doses of SCH 23390 ranging from 0.05 to 100 nmol/side injected bilaterally into the VTA, but not by equimolar doses of Raclopride, a D2 antagonist. These results, combined with previous findings, suggest a signaling pathway between the VTA and the sNAcc in which opioids and DA facilitate feeding in an interdependent manner.

How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades

How does the brain learn to balance between reactive and planned behaviors? The basal ganglia (BG) and frontal cortex together allow animals to learn planned behaviors that acquire rewards when prepotent reactive behaviors are insufficient. This paper proposes a new model, called TELOS, to explain how laminar circuitry of the frontal cortex, exemplified by the frontal eye fields, interacts with the BG, thalamus, superior colliculus, and inferotemporal and parietal cortices to learn and perform reactive and planned eye movements. The model is formulated as fourteen computational hypotheses. These specify how strategy priming and action planning (in cortical layers III, Va and VI) are dissociated from movement execution (in layer Vb), how the BG help to choose among and gate competing plans, and how a visual stimulus may serve either as a movement target or as a discriminative cue to move elsewhere. The direct, indirect and hyperdirect pathways through the BG are shown to enable complex gating functions, including deferred execution of selected plans, and switching among alternative sensory-motor mappings. Notably, the model can learn and gate the use of a What-to-Where transformation that enables spatially invariant object representations to selectively excite spatially coded movement plans. Model simulations show how dopaminergic reward and non-reward signals guide monkeys to learn and perform saccadic eye movements in the fixation, single saccade, overlap, gap, and delay (memory-guided) saccade tasks. Model cell activation dynamics quantitatively simulate seventeen established types of dynamics exhibited by corresponding real cells during performance of these tasks.

Opposing effects of D1 and D2 receptor activation on odor discrimination learning

Dopaminergic modulation of cortical activity has been implicated in the formation of reward associations. There is abundant evidence for dopaminergic effects on olfactory processing. Using an olfactory discrimination task, the authors show that D1 and D2 dopamine receptors can regulate rats' olfactory discrimination capacities and that the effects of receptor activation functionally oppose one another. Injection of either the D1 agonist SKF 38393 (10 mg/kg) or the D2 antagonist spiperone (0.62 mg/kg) facilitated the discrimination of similar odorants but had no effect on the discrimination of dissimilar odorants, whereas both the D, antagonist SCH 23390 (0.025 mg/kg) and the D2 agonist quinpirole (0.2 mg/kg) significantly impaired rats' ability to discriminate similar and dissimilar odorants.