Responses of monkey midbrain dopamine neurons during delayed alternation performance

Memory-specific encoding activities of the ventral tegmental area dopamine and GABA neurons

Although the midbrain dopamine (DA) system plays a crucial role in higher cognitive functions, including updating and maintaining short-term memory, the encoding properties of the somatic spiking activity of ventral tegmental area (VTA) DA neurons for short-term memory computations have not yet been identified. Here, we probed and analyzed the activity of optogenetically identified DA and GABA neurons while mice engaged in short-term memory-dependent behavior in a T-maze task. Single-neuron analysis revealed that significant subpopulations of DA and GABA neurons responded differently between left and right trials in the memory delay. With a series of control behavioral tasks and regression analysis tools, we show that firing rate differences are linked to short-term memory-dependent decisions and cannot be explained by reward-related processes, motivated behavior, or motor-related activities. This evidence provides novel insights into the mnemonic encoding activities of midbrain DA and GABA neurons.

Neural inhibition as implemented by an actor-critic model involves the human dorsal striatum and ventral tegmental area

Inhibition is implicated across virtually all human experiences. As a trade-off of being very efficient, this executive function is also prone to many errors. Rodent and computational studies show that midbrain regions play crucial roles during errors by sending dopaminergic learning signals to the basal ganglia for behavioural adjustment. However, the parallels between animal and human neural anatomy and function are not determined. We scanned human adults while they performed an fMRI inhibitory task requiring trial-and-error learning. Guided by an actor-critic model, our results implicate the dorsal striatum and the ventral tegmental area as the actor and the critic, respectively. Using a multilevel and dimensional approach, we also demonstrate a link between midbrain and striatum circuit activity, inhibitory performance, and self-reported autistic and obsessive-compulsive subclinical traits.

Dopamine receptor activation regulates reward expectancy signals during cognitive control in primate prefrontal neurons

Dopamine neurons respond to reward-predicting cues but also modulate information processing in the prefrontal cortex essential for cognitive control. Whether dopamine controls reward expectation signals in prefrontal cortex that motivate cognitive control is unknown. We trained two male macaques on a working memory task while varying the reward size earned for successful task completion. We recorded neurons in lateral prefrontal cortex while simultaneously stimulating dopamine D1 receptor (D1R) or D2 receptor (D2R) families using micro-iontophoresis. We show that many neurons predict reward size throughout the trial. D1R stimulation showed mixed effects following reward cues but decreased reward expectancy coding during the memory delay. By contrast, D2R stimulation increased reward expectancy coding in multiple task periods, including cueing and memory periods. Stimulation of either dopamine receptors increased the neurons' selective responses to reward size upon reward delivery. The differential modulation of reward expectancy by dopamine receptors suggests that dopamine regulates reward expectancy necessary for successful cognitive control.

Defining the interconnectivity of the medial prefrontal cortex and ventral midbrain

Dysfunction in dopamine (DA) signaling contributes to neurological disorders ranging from drug addiction and schizophrenia to depression and Parkinson’s Disease. How might impairment of one neurotransmitter come to effect these seemingly disparate diseases? One potential explanation is that unique populations of DA-releasing cells project to separate brain regions that contribute to different sets of behaviors. Though dopaminergic cells themselves are spatially restricted to the midbrain and constitute a relatively small proportion of all neurons, their projections influence many brain regions. DA is particularly critical for the activity and function of medial prefrontal cortical (mPFC) ensembles. The midbrain and mPFC exhibit reciprocal connectivity – the former innervates the mPFC, and in turn, the mPFC projects back to the midbrain. Viral mapping studies have helped elucidate the connectivity within and between these regions, which likely have broad implications for DA-dependent behaviors. In this review, we discuss advancements in our understanding of the connectivity between the mPFC and midbrain DA system, focusing primarily on rodent models.

Mesoaccumbal Dopamine Heterogeneity: What Do Dopamine Firing and Release Have to Do with It?

Ventral tegmental area (VTA) dopamine (DA) neurons are often thought to uniformly encode reward prediction errors. Conversely, DA release in the nucleus accumbens (NAc), the prominent projection target of these neurons, has been implicated in reinforcement learning, motivation, aversion, and incentive salience. This contrast between heterogeneous functions of DA release versus a homogeneous role for DA neuron activity raises numerous questions regarding how VTA DA activity translates into NAc DA release. Further complicating this issue is increasing evidence that distinct VTA DA projections into defined NAc subregions mediate diverse behavioral functions. Here, we evaluate evidence for heterogeneity within the mesoaccumbal DA system and argue that frameworks of DA function must incorporate the precise topographic organization of VTA DA neurons to clarify their contribution to health and disease.

Dopamine firing plays a dual role in coding reward prediction errors and signaling motivation in a working memory task

Little is known about how dopamine (DA) neuron firing rates behave in cognitively demanding decision-making tasks. Here, we investigated midbrain DA activity in monkeys performing a discrimination task in which the animal had to use working memory (WM) to report which of two sequentially applied vibrotactile stimuli had the higher frequency. We found that perception was altered by an internal bias, likely generated by deterioration of the representation of the first frequency during the WM period. This bias greatly controlled the DA phasic response during the two stimulation periods, confirming that DA reward prediction errors reflected stimulus perception. In contrast, tonic dopamine activity during WM was not affected by the bias and did not encode the stored frequency. More interestingly, both delay-period activity and phasic responses before the second stimulus negatively correlated with reaction times of the animals after the trial start cue and thus represented motivated behavior on a trial-by-trial basis. During WM, this motivation signal underwent a ramp-like increase. At the same time, motivation positively correlated with accuracy, especially in difficult trials, probably by decreasing the effect of the bias. Overall, our results indicate that DA activity, in addition to encoding reward prediction errors, could at the same time be involved in motivation and WM. In particular, the ramping activity during the delay period suggests a possible DA role in stabilizing sustained cortical activity, hypothetically by increasing the gain communicated to prefrontal neurons in a motivation-dependent way.

A Comparison of Dopaminergic and Cholinergic Populations Reveals Unique Contributions of VTA Dopamine Neurons to Short-Term Memory

We systematically compare the contributions of two dopaminergic and two cholinergic ascending populations to a spatial short-term memory task in rats. In ventral tegmental area dopamine (VTA-DA) and nucleus basalis cholinergic (NB-ChAT) populations, trial-by-trial fluctuations in activity during the delay period relate to performance with an inverted-U, despite the fact that both populations have low activity during that time. Transient manipulations reveal that only VTA-DA neurons, and not the other three populations we examine, contribute causally and selectively to short-term memory. This contribution is most significant during the delay period, when both increases and decreases in VTA-DA activity impair short-term memory. Our results reveal a surprising dissociation between when VTA-DA neurons are most active and when they have the biggest causal contribution to short-term memory, and they also provide support for classic ideas about an inverted-U relationship between neuromodulation and cognition.

Desire or Dread from Nucleus Accumbens Inhibitions: Reversed by Same-Site Optogenetic Excitations

Microinjections of a glutamate AMPA antagonist (DNQX) in medial shell of nucleus accumbens (NAc) can cause either intense appetitive motivation (i.e., 'desire') or intense defensive motivation (i.e., 'dread'), depending on site along a flexible rostrocaudal gradient and on environmental ambience. DNQX, by blocking excitatory AMPA glutamate inputs, is hypothesized to produce relative inhibitions of NAc neurons. However, given potential alternative explanations, it is not known whether neuronal inhibition is in fact necessary for NAc DNQX microinjections to generate motivations. Here we provide a direct test of whether local neuronal inhibition in NAc is necessary for DNQX microinjections to produce either desire or dread. We used optogenetic channelrhodopsin (ChR2) excitations at the same local sites in NAc as DNQX microinjections to oppose relative neuronal inhibitions induced by DNQX in female and male rats. We found that same-site ChR2 excitation effectively reversed the ability of NAc DNQX microinjections to generate appetitive motivation, and similarly reversed ability of DNQX microinjections to generate defensive motivation. Same-site NAc optogenetic excitations also attenuated recruitment of Fos expression in other limbic structures throughout the brain, which was otherwise elevated by NAc DNQX microinjections that generated motivation. However, to successfully reverse motivation generation, an optic fiber tip for ChR2 illumination needed to be located within <1 mm of the corresponding DNQX microinjector tip; that is, both truly at the same NAc site. Thus, we confirm that localized NAc neuronal inhibition is required for AMPA-blocking microinjections in medial shell to induce either positively-valenced 'desire' or negatively-valenced 'dread'.SIGNIFICANCE STATEMENT A major hypothesis posits neuronal inhibitions in nucleus accumbens generate intense motivation. Microinjections in nucleus accumbens of glutamate antagonist, DNQX, which might suppress local neuronal firing, generate either appetitive or defensive motivation, depending on site and environmental factors. Is neuronal inhibition in nucleus accumbens required for such pharmacologically-induced motivations? Here we demonstrate that neuronal inhibition is necessary to generate appetitive or defensive motivations, using local optogenetic excitations to oppose putative DNQX-induced inhibitions. We show that excitation at the same site prevents DNQX microinjections from recruiting downstream limbic structures into neurobiological activation, and simultaneously prevents generation of either appetitive or defensive motivated behaviors. These results may be relevant to roles of nucleus accumbens mechanisms in pathological motivations, including addiction and paranoia.

Disrupted reinforcement learning during post-error slowing in ADHD

ADHD is associated with altered dopamine regulated reinforcement learning on prediction errors. Despite evidence of categorically altered error processing in ADHD, neuroimaging advances have largely investigated models of normal reinforcement learning in greater detail. Further, although reinforcement leaning critically relies on ventral striatum exerting error magnitude related thresholding influences on substantia nigra (SN) and dorsal striatum, these thresholding influences have never been identified with neuroimaging. To identify such thresholding influences, we propose that error magnitude related activities must first be separated from opposite activities in overlapping neural regions during error detection. Here we separate error detection from magnitude related adjustment (post-error slowing) during inhibition errors in the stop signal task in typically developing (TD) and ADHD adolescents using fMRI. In TD, we predicted that: 1) deactivation of dorsal striatum on error detection interrupts ongoing processing, and should be proportional to right frontoparietal response phase activity that has been observed in the SST; 2) deactivation of ventral striatum on post-error slowing exerts thresholding influences on, and should be proportional to activity in dorsal striatum. In ADHD, we predicted that ventral striatum would instead correlate with heightened amygdala responses to errors. We found deactivation of dorsal striatum on error detection correlated with response-phase activity in both groups. In TD, post-error slowing deactivation of ventral striatum correlated with activation of dorsal striatum. In ADHD, ventral striatum correlated with heightened amygdala activity. Further, heightened activities in locus coeruleus (norepinephrine), raphe nucleus (serotonin) and medial septal nuclei (acetylcholine), which all compete for control of DA, and are altered in ADHD, exhibited altered correlations with SN. All correlations in TD were replicated in healthy adults. Results in TD are consistent with dopamine regulated reinforcement learning on post-error slowing. In ADHD, results are consistent with heightened activities in the amygdala and non-dopaminergic neurotransmitter nuclei preventing reinforcement learning.

Dopamine and Cognitive Control in Prefrontal Cortex

Cognitive control, the ability to orchestrate behavior in accord with our goals, depends on the prefrontal cortex. These cognitive functions are heavily influenced by the neuromodulator dopamine. We review here recent insights exploring the influence of dopamine on neuronal response properties in prefrontal cortex (PFC) during ongoing behaviors in primates. This review suggests three major computational roles of dopamine in cognitive control: (i) gating sensory input, (ii) maintaining and manipulating working memory contents, and (iii) relaying motor commands. For each of these roles, we propose a neuronal microcircuit based on known mechanisms of action of dopamine in PFC, which are corroborated by computational network models. This conceptual approach accounts for the various roles of dopamine in prefrontal executive functioning.

Non-motor Characterization of the Basal Ganglia: Evidence From Human and Non-human Primate Electrophysiology

Although the basal ganglia have been implicated in a growing list of human behaviors, they include some of the least understood nuclei in the brain. For several decades studies have employed numerous methodologies to uncover evidence pointing to the basal ganglia as a hub of both motor and non-motor function. Recently, new electrophysiological characterization of the basal ganglia in humans has become possible through direct access to these deep structures as part of routine neurosurgery. Electrophysiological approaches for identifying non-motor function have the potential to unlock a deeper understanding of pathways that may inform clinical interventions and particularly neuromodulation. Various electrophysiological modalities can also be combined to reveal functional connections between the basal ganglia and traditional structures throughout the neocortex that have been linked to non-motor behavior. Several reviews have previously summarized evidence for non-motor function in the basal ganglia stemming from behavioral, clinical, computational, imaging, and non-primate animal studies; in this review, instead we turn to electrophysiological studies of non-human primates and humans. We begin by introducing common electrophysiological methodologies for basal ganglia investigation, and then we discuss studies across numerous non-motor domains–emotion, response inhibition, conflict, decision-making, error-detection and surprise, reward processing, language, and time processing. We discuss the limitations of current approaches and highlight the current state of the information.

Peromyscus maniculatus bairdii as a naturalistic mammalian model of obsessive-compulsive disorder: current status and future challenges

Obsessive-compulsive disorder (OCD) is a prevalent and debilitating condition, characterized by intrusive thoughts and repetitive behavior. Animal models of OCD arguably have the potential to contribute to our understanding of the condition. Deer mice (Permomyscus maniculatus bairdii) are characterized by stereotypic behavior which is reminiscent of OCD symptomology, and which may serve as a naturalistic animal model of this disorder. Moreover, a range of deer mouse repetitive behaviors may be representative of different compulsive-like phenotypes. This paper will review work on deer mouse behavior, and evaluate the extent to which this serves as a valid and useful model of OCD. We argue that findings over the past decade indicate that the deer mouse model has face, construct and predictive validity.

The phasic dopamine signal maturing: from reward via behavioural activation to formal economic utility

The phasic dopamine reward prediction error response is a major brain signal underlying learning, approach and decision making. This dopamine response consists of two components that reflect, initially, stimulus detection from physical impact and, subsequenttly, reward valuation; dopamine activations by punishers reflect physical impact rather than aversiveness. The dopamine reward signal is distinct from earlier reported and recently confirmed phasic changes with behavioural activation. Optogenetic activation of dopamine neurones in monkeys causes value learning and biases economic choices. The dopamine reward signal conforms to formal economic utility and thus constitutes a utility prediction error signal. In these combined ways, the dopamine reward prediction error signal constitutes a potential neuronal substrate for the crucial economic decision variable of utility.

Pontomesencephalic Tegmental Afferents to VTA Non-dopamine Neurons Are Necessary for Appetitive Pavlovian Learning

The ventral tegmental area (VTA) receives phenotypically distinct innervations from the pedunculopontine tegmental nucleus (PPTg). While PPTg-to-VTA inputs are thought to play a critical role in stimulus-reward learning, direct evidence linking PPTg-to-VTA phenotypically distinct inputs in the learning process remains lacking. Here, we used optogenetic approaches to investigate the functional contribution of PPTg excitatory and inhibitory inputs to the VTA in appetitive Pavlovian conditioning. We show that photoinhibition of PPTg-to-VTA cholinergic or glutamatergic inputs during cue presentation dampens the development of anticipatory approach responding to the food receptacle during the cue. Furthermore, we employed in vivo optetrode recordings to show that photoinhibition of PPTg cholinergic or glutamatergic inputs significantly decreases VTA non-dopamine (non-DA) neural activity. Consistently, photoinhibition of VTA non-DA neurons disrupts the development of cue-elicited anticipatory approach responding. Taken together, our study reveals a crucial regulatory mechanism by PPTg excitatory inputs onto VTA non-DA neurons during appetitive Pavlovian conditioning.

Dopamine reward prediction-error signalling: a two-component response

Environmental stimuli and objects, including rewards, are often processed sequentially in the brain. Recent work suggests that the phasic dopamine reward prediction-error response follows a similar sequential pattern. An initial brief, unselective and highly sensitive increase in activity unspecifically detects a wide range of environmental stimuli, then quickly evolves into the main response component, which reflects subjective reward value and utility. This temporal evolution allows the dopamine reward prediction-error signal to optimally combine speed and accuracy.

Reward functions of the basal ganglia

Besides their fundamental movement function evidenced by Parkinsonian deficits, the basal ganglia are involved in processing closely linked non-motor, cognitive and reward information. This review describes the reward functions of three brain structures that are major components of the basal ganglia or are closely associated with the basal ganglia, namely midbrain dopamine neurons, pedunculopontine nucleus, and striatum (caudate nucleus, putamen, nucleus accumbens). Rewards are involved in learning (positive reinforcement), approach behavior, economic choices and positive emotions. The response of dopamine neurons to rewards consists of an early detection component and a subsequent reward component that reflects a prediction error in economic utility, but is unrelated to movement. Dopamine activations to non-rewarded or aversive stimuli reflect physical impact, but not punishment. Neurons in pedunculopontine nucleus project their axons to dopamine neurons and process sensory stimuli, movements and rewards and reward-predicting stimuli without coding outright reward prediction errors. Neurons in striatum, besides their pronounced movement relationships, process rewards irrespective of sensory and motor aspects, integrate reward information into movement activity, code the reward value of individual actions, change their reward-related activity during learning, and code own reward in social situations depending on whose action produces the reward. These data demonstrate a variety of well-characterized reward processes in specific basal ganglia nuclei consistent with an important function in non-motor aspects of motivated behavior.

Learning touch preferences with a tactile robot using dopamine modulated STDP in a model of insular cortex

Neurorobots enable researchers to study how behaviors are produced by neural mechanisms in an uncertain, noisy, real-world environment. To investigate how the somatosensory system processes noisy, real-world touch inputs, we introduce a neurorobot called CARL-SJR, which has a full-body tactile sensory area. The design of CARL-SJR is such that it encourages people to communicate with it through gentle touch. CARL-SJR provides feedback to users by displaying bright colors on its surface. In the present study, we show that CARL-SJR is capable of learning associations between conditioned stimuli (CS; a color pattern on its surface) and unconditioned stimuli (US; a preferred touch pattern) by applying a spiking neural network (SNN) with neurobiologically inspired plasticity. Specifically, we modeled the primary somatosensory cortex, prefrontal cortex, striatum, and the insular cortex, which is important for hedonic touch, to process noisy data generated directly from CARL-SJR's tactile sensory area. To facilitate learning, we applied dopamine-modulated Spike Timing Dependent Plasticity (STDP) to our simulated prefrontal cortex, striatum, and insular cortex. To cope with noisy, varying inputs, the SNN was tuned to produce traveling waves of activity that carried spatiotemporal information. Despite the noisy tactile sensors, spike trains, and variations in subject hand swipes, the learning was quite robust. Further, insular cortex activities in the incremental pathway of dopaminergic reward system allowed us to control CARL-SJR's preference for touch direction without heavily pre-processed inputs. The emerged behaviors we found in this model match animal's behaviors wherein they prefer touch in particular areas and directions. Thus, the results in this paper could serve as an explanation on the underlying neural mechanisms for developing tactile preferences and hedonic touch.

Neuronal Reward and Decision Signals: From Theories to Data

Rewards are crucial objects that induce learning, approach behavior, choices, and emotions. Whereas emotions are difficult to investigate in animals, the learning function is mediated by neuronal reward prediction error signals which implement basic constructs of reinforcement learning theory. These signals are found in dopamine neurons, which emit a global reward signal to striatum and frontal cortex, and in specific neurons in striatum, amygdala, and frontal cortex projecting to select neuronal populations. The approach and choice functions involve subjective value, which is objectively assessed by behavioral choices eliciting internal, subjective reward preferences. Utility is the formal mathematical characterization of subjective value and a prime decision variable in economic choice theory. It is coded as utility prediction error by phasic dopamine responses. Utility can incorporate various influences, including risk, delay, effort, and social interaction. Appropriate for formal decision mechanisms, rewards are coded as object value, action value, difference value, and chosen value by specific neurons. Although all reward, reinforcement, and decision variables are theoretical constructs, their neuronal signals constitute measurable physical implementations and as such confirm the validity of these concepts. The neuronal reward signals provide guidance for behavior while constraining the free will to act.

Why bother using non-human primate models of cognitive disorders in translational research?

Although everyone would agree that successful translation of therapeutic candidates for central nervous disorders should involve non-human primate (nhp) models of cognitive disorders, we are left with the paucity of publications reporting either the target validation or the actual preclinical testing in heuristic nhp models. In this review, we discuss the importance of nhps in translational research, highlighting the advances in technological/methodological approaches for 'bridging the gap' between preclinical and clinical experiments. In this process, we acknowledge that nhps remain a vital tool for the investigation of complex cognitive functions, given their resemblance to humans in aspects of behaviour, anatomy and physiology. The recent improvements made for a suitable nhp model in cognitive research, including new surrogates of disease and application of innovative methodological approaches, are continuous strides for reaching efficient translation for human benefit. This will ultimately aid the development of innovative treatments against the current and future threat of neurological and psychiatric disorders to the global population.

Lateral hypothalamus, nucleus accumbens, and ventral pallidum roles in eating and hunger: interactions between homeostatic and reward circuitry

The study of the neural bases of eating behavior, hunger, and reward has consistently implicated the lateral hypothalamus (LH) and its interactions with mesocorticolimbic circuitry, such as mesolimbic dopamine projections to nucleus accumbens (NAc) and ventral pallidum (VP), in controlling motivation to eat. The NAc and VP play special roles in mediating the hedonic impact (“liking”) and motivational incentive salience (“wanting”) of food rewards, and their interactions with LH help permit regulatory hunger/satiety modulation of food motivation and reward. Here, we review some progress that has been made regarding this circuitry and its functions: the identification of localized anatomical hedonic hotspots within NAc and VP for enhancing hedonic impact; interactions of NAc/VP hedonic hotspots with specific LH signals such as orexin; an anterior-posterior gradient of sites in NAc shell for producing intense appetitive eating vs. intense fearful reactions; and anatomically distributed appetitive functions of dopamine and mu opioid signals in NAc shell and related structures. Such findings help improve our understanding of NAc, VP, and LH interactions in mediating affective and motivation functions, including “liking” and “wanting” for food rewards.

Phasic dopamine as a prediction error of intrinsic and extrinsic reinforcements driving both action acquisition and reward maximization: a simulated robotic study

An important issue of recent neuroscientific research is to understand the functional role of the phasic release of dopamine in the striatum, and in particular its relation to reinforcement learning. The literature is split between two alternative hypotheses: one considers phasic dopamine as a reward prediction error similar to the computational TD-error, whose function is to guide an animal to maximize future rewards; the other holds that phasic dopamine is a sensory prediction error signal that lets the animal discover and acquire novel actions. In this paper we propose an original hypothesis that integrates these two contrasting positions: according to our view phasic dopamine represents a TD-like reinforcement prediction error learning signal determined by both unexpected changes in the environment (temporary, intrinsic reinforcements) and biological rewards (permanent, extrinsic reinforcements). Accordingly, dopamine plays the functional role of driving both the discovery and acquisition of novel actions and the maximization of future rewards. To validate our hypothesis we perform a series of experiments with a simulated robotic system that has to learn different skills in order to get rewards. We compare different versions of the system in which we vary the composition of the learning signal. The results show that only the system reinforced by both extrinsic and intrinsic reinforcements is able to reach high performance in sufficiently complex conditions.

A Spiking Neural Model for Stable Reinforcement of Synapses Based on Multiple Distal Rewards

In this letter, a novel critic-like algorithm was developed to extend the synaptic plasticity rule described in Florian ( 2007 ) and Izhikevich ( 2007 ) in order to solve the problem of learning multiple distal rewards simultaneously. The system is augmented with short-term plasticity (STP) to stabilize the learning dynamics, thereby increasing the system's learning capacity. A theoretical threshold is estimated for the number of distal rewards that this system can learn. The validity of the novel algorithm was verified by computer simulations.

Presynaptic dopamine modulation by stimulant self-administration

The mesolimbic dopamine system is an essential participant in the initiation and modulation of various forms of goal-directed behavior, including drug reinforcement and addiction processes. Dopamine neurotransmission is increased by acute administration of all drugs of abuse, including the stimulants cocaine and amphetamine. Chronic exposure to these drugs via voluntary self-administration provides a model of stimulant abuse that is useful in evaluating potential behavioral and neurochemical adaptations that occur during addiction. This review describes commonly used methodologies to measure dopamine and baseline parameters of presynaptic dopamine regulation, including exocytotic release and reuptake through the dopamine transporter in the nucleus accumbens core, as well as dramatic adaptations in dopamine neurotransmission and drug sensitivity that occur with acute non-contingent and chronic, contingent self-administration of cocaine and amphetamine.

Selective neural pathway targeting reveals key roles of thalamostriatal projection in the control of visual discrimination

The dorsal striatum receives converging excitatory inputs from diverse brain regions, including the cerebral cortex and the intralaminar/midline thalamic nuclei, and mediates learning processes contributing to instrumental motor actions. However, the roles of each striatal input pathway in these learning processes remain uncertain. We developed a novel strategy to target specific neural pathways and applied this strategy for studying behavioral roles of the pathway originating from the parafascicular nucleus (PF) and projecting to the dorsolateral striatum. A highly efficient retrograde gene transfer vector encoding the recombinant immunotoxin (IT) receptor was injected into the dorsolateral striatum in mice to express the receptor in neurons innervating the striatum. IT treatment into the PF of the vector-injected animals caused a selective elimination of neurons of the PF-derived thalamostriatal pathway. The elimination of this pathway impaired the response selection accuracy and delayed the motor response in the acquisition of a visual cue-dependent discrimination task. When the pathway elimination was induced after learning acquisition, it disturbed the response accuracy in the task performance with no apparent change in the response time. The elimination did not influence spontaneous locomotion, methamphetamine-induced hyperactivity, and motor skill learning that demand the function of the dorsal striatum. These results demonstrate that thalamostriatal projection derived from the PF plays essential roles in the acquisition and execution of discrimination learning in response to sensory stimulus. The temporal difference in the pathway requirement for visual discrimination suggests a stage-specific role of thalamostriatal pathway in the modulation of response time of learned motor actions.

Differential magnitude coding of gains and omitted rewards in the ventral striatum

Physiologic studies revealed that neurons in the dopaminergic midbrain of non-human primates encode reward prediction errors. It was furthermore shown that reward prediction errors are adaptively scaled with respect to the range of possible outcomes, enabling sensitive encoding for a large range of reward values. Congruently, neuroimaging studies in humans demonstrated that BOLD-responses in the ventral striatum encode reward prediction errors in similar fashion as dopaminergic midbrain neurons, suggesting that these BOLD-responses may be driven by dopaminergic midbrain activity. However, neuroimaging results are ambiguous with respect to the adaptive scaling of reward prediction errors, leading to the conjecture that under certain circumstances other than dopaminergic midbrain input may drive ventral striatal BOLD-responses. The goal of this study was to substantiate whether BOLD-responses in the ventral striatum rather respond to adaptively scaled reward prediction errors or absolute reward magnitude. In addition, we aimed to identify neuronal structures modulating activity in the ventral striatum. Sixteen healthy participants played a wheel of fortune game, where they could win three differently valued rewards while being scanned. BOLD-responses increased after gaining rewards; this gain was however independent of the absolute reward magnitude. In contrast BOLD-responses upon reward omission decreased with reward magnitude. A psychophysiological interaction analysis identified a cluster in the brainstem in proximity of the dorsal raphe nucleus, a cluster in the lateral orbitofrontal cortex, and a cluster in the rostral cingulate zone. These clusters changed their connectivity with the ventral striatum in relation to the absolute reward magnitude in reward omission trials.

Dopamine-signaled reward predictions generated by competitive excitation and inhibition in a spiking neural network model

Dopaminergic neurons in the mammalian substantia nigra display characteristic phasic responses to stimuli which reliably predict the receipt of primary rewards. These responses have been suggested to encode reward prediction-errors similar to those used in reinforcement learning. Here, we propose a model of dopaminergic activity in which prediction-error signals are generated by the joint action of short-latency excitation and long-latency inhibition, in a network undergoing dopaminergic neuromodulation of both spike-timing dependent synaptic plasticity and neuronal excitability. In contrast to previous models, sensitivity to recent events is maintained by the selective modification of specific striatal synapses, efferent to cortical neurons exhibiting stimulus-specific, temporally extended activity patterns. Our model shows, in the presence of significant background activity, (i) a shift in dopaminergic response from reward to reward-predicting stimuli, (ii) preservation of a response to unexpected rewards, and (iii) a precisely timed below-baseline dip in activity observed when expected rewards are omitted.

Serotonin and dopamine: unifying affective, activational, and decision functions

Serotonin, like dopamine (DA), has long been implicated in adaptive behavior, including decision making and reinforcement learning. However, although the two neuromodulators are tightly related and have a similar degree of functional importance, compared with DA, we have a much less specific understanding about the mechanisms by which serotonin affects behavior. Here, we draw on recent work on computational models of dopaminergic function to suggest a framework by which many of the seemingly diverse functions associated with both DA and serotonin-comprising both affective and activational ones, as well as a number of other functions not overtly related to either-can be seen as consequences of a single root mechanism.

Strong somatic stimulation differentially regulates the firing properties of prefrontal cortex neurons

Among the brain structures involved in processing affective stimuli, the roles of the prefrontal cortex (PFC) and the mesocorticolimbic dopaminergic (DA) innervation are well established. In contrast to our understanding of the reward stimuli, less is known about how strong somatic stimulation is processed within the PFC. Here, we examined the effects of a strong pinch delivered to the rat posterior paw on spontaneous and current-evoked activity of PFC neurons using intracellular recordings in anesthetized rats. Following the paw pinch, pyramidal cells exhibited a significant decrease in spontaneous activity along with a significant increase in the current-evoked firing. The increase in current-evoked firing elicited by the paw pinch was inversely correlated with the baseline firing rate. Systemic administration of a selective dopamine D2 receptor antagonist partially blocked the effects elicited by the paw pinch on cortical excitability, whereas systemic administration of a D1 antagonist seems to facilitate paw-mediated increases in evoked firing. These results suggest that strong somatic stimuli decrease spontaneous firing while increasing depolarization-evoked firing in a DA receptor dependent manner. These mechanisms may help in the control of the signal to noise ratio or the salience of information processing in the PFC following strong somatic stimulation.

Dopamine signals for reward value and risk: basic and recent data

Background

Previous lesion, electrical self-stimulation and drug addiction studies suggest that the midbrain dopamine systems are parts of the reward system of the brain. This review provides an updated overview about the basic signals of dopamine neurons to environmental stimuli.

Methods

The described experiments used standard behavioral and neurophysiological methods to record the activity of single dopamine neurons in awake monkeys during specific behavioral tasks.

Results

Dopamine neurons show phasic activations to external stimuli. The signal reflects reward, physical salience, risk and punishment, in descending order of fractions of responding neurons. Expected reward value is a key decision variable for economic choices. The reward response codes reward value, probability and their summed product, expected value. The neurons code reward value as it differs from prediction, thus fulfilling the basic requirement for a bidirectional prediction error teaching signal postulated by learning theory. This response is scaled in units of standard deviation. By contrast, relatively few dopamine neurons show the phasic activation following punishers and conditioned aversive stimuli, suggesting a lack of relationship of the reward response to general attention and arousal. Large proportions of dopamine neurons are also activated by intense, physically salient stimuli. This response is enhanced when the stimuli are novel; it appears to be distinct from the reward value signal. Dopamine neurons show also unspecific activations to non-rewarding stimuli that are possibly due to generalization by similar stimuli and pseudoconditioning by primary rewards. These activations are shorter than reward responses and are often followed by depression of activity. A separate, slower dopamine signal informs about risk, another important decision variable. The prediction error response occurs only with reward; it is scaled by the risk of predicted reward.

Conclusions

Neurophysiological studies reveal phasic dopamine signals that transmit information related predominantly but not exclusively to reward. Although not being entirely homogeneous, the dopamine signal is more restricted and stereotyped than neuronal activity in most other brain structures involved in goal directed behavior.

Dopamine signals for reward value and risk: basic and recent data

BACKGROUND

Previous lesion, electrical self-stimulation and drug addiction studies suggest that the midbrain dopamine systems are parts of the reward system of the brain. This review provides an updated overview about the basic signals of dopamine neurons to environmental stimuli.

METHODS

The described experiments used standard behavioral and neurophysiological methods to record the activity of single dopamine neurons in awake monkeys during specific behavioral tasks.

RESULTS

Dopamine neurons show phasic activations to external stimuli. The signal reflects reward, physical salience, risk and punishment, in descending order of fractions of responding neurons. Expected reward value is a key decision variable for economic choices. The reward response codes reward value, probability and their summed product, expected value. The neurons code reward value as it differs from prediction, thus fulfilling the basic requirement for a bidirectional prediction error teaching signal postulated by learning theory. This response is scaled in units of standard deviation. By contrast, relatively few dopamine neurons show the phasic activation following punishers and conditioned aversive stimuli, suggesting a lack of relationship of the reward response to general attention and arousal. Large proportions of dopamine neurons are also activated by intense, physically salient stimuli. This response is enhanced when the stimuli are novel; it appears to be distinct from the reward value signal. Dopamine neurons show also unspecific activations to non-rewarding stimuli that are possibly due to generalization by similar stimuli and pseudoconditioning by primary rewards. These activations are shorter than reward responses and are often followed by depression of activity. A separate, slower dopamine signal informs about risk, another important decision variable. The prediction error response occurs only with reward; it is scaled by the risk of predicted reward.

CONCLUSIONS

Neurophysiological studies reveal phasic dopamine signals that transmit information related predominantly but not exclusively to reward. Although not being entirely homogeneous, the dopamine signal is more restricted and stereotyped than neuronal activity in most other brain structures involved in goal directed behavior.

Personality traits are differentially associated with patterns of reward and novelty processing in the human substantia nigra/ventral tegmental area

BACKGROUND

The long-standing observation that the novelty-seeking personality trait is a predictor of drug use and other reinforcable risky behaviors raises the question as to how novelty and reward processing functionally interact in mesolimbic dopaminergic circuitry and how this interaction is modulated by the novelty-seeking personality trait.

METHODS

Functional magnetic resonance imaging (fMRI) hemodynamic responses to novelty and reward (monetary incentive) from the substantia nigra/ventral tegmental area (SN/VTA), the nucleus accumbens (NAcc), and the hippocampus of 29 subjects were correlated with novelty-seeking scores. These correlations were compared with those obtained for scores of reward-dependence. The fMRI data were taken from two experiments in which the interaction of novelty and reward was manipulated as a within-subject variable, and long-term memory for the critical stimuli was assessed after 24 hours.

RESULTS

Novelty-seeking was positively correlated with SN/VTA activation elicited by novel cues that did not predict reward, whereas reward-dependence was related to activations elicited by novel cues that predicted reward. The positive correlation between SN/VTA responses to novelty and novelty-seeking scores was accompanied by a negative correlation with reward-related SN/VTA activation and memory enhancement.

CONCLUSIONS

SN/VTA responses to novelty and reward are differentially affected by personality traits of novelty-seeking and reward-dependence. Importantly, novelty-seekers were more responsive to novel cues in the absence of reward and needed less reward to boost their memory for novel cues. These observations strongly suggest that for novelty-seekers, the motivational value of novelty is not necessarily based on actual reward-predicting stimulus properties.

Stimulus-response and response-outcome learning mechanisms in the striatum

While midbrain DA neurons show phasic activations in response to both reward-predicting and salient non-reward events, activation responses to primary and conditioned rewards are sustained for several hundreds of milliseconds beyond those elicited by salient non-reward-related stimuli. The longer-duration DA reward response and corresponding elevated DA release in striatal target sites may selectively strengthen currently-active corticostriatal synapses, i.e., those associated with the successful reward-procuring behavior. This paper describes how similar models of DA-mediated plasticity of corticostriatal synapses may describe both stimulus-response and response-outcome learning. DA-mediated strengthening of corticostriatal synapses in regions of the dorsolateral striatum receiving afferents from primary sensorimotor cortex is likely to bind corticostriatal inputs representing the previously-emitted movement to striatal outputs contributing to the selection of the next movement segment in a behavioral sequence. Within the striatum, more generally, inputs from distinct regions of the frontal cortex that code independently for movement direction and reward expectation send convergent projections to striatal output cells. DA-mediated strengthening of active corticostriatal synapses promotes the future output of the striatal cell under similar input conditions. This is postulated to promote persistence of neuronal activity in the very cortical cells that drive corticostriatal input, leading to the establishment of sustained reverberatory loops that permit cortical movement-related cells to maintain activity until the appropriate time of movement initiation.

Synaptic plasticity in the basal ganglia

Activity-dependent synaptic plasticity occurs in several parts of the basal ganglia. Increasing evidence supports the hypothesis that activity-dependent plasticity underlies the acquisition, maintenance, and extinction of certain types of learning in the basal ganglia. This review focuses on synaptic plasticity in the corticostriatal pathway. As in other systems, both long-term potentiation and long-term depression have been described, and intracellular calcium signalling plays an important role in the induction of plasticity. However, intracellular calcium levels do not appear to be the dominating control factor. Dopamine, via intracellular signalling cascades, also plays a crucial role in determining the magnitude and direction of plasticity, and in modulating the requirements for induction. Endocannabinoids also play an important role in mediating presynaptic expression of synaptic depression. Recent studies have highlighted spike-timing dependent plasticity phenomena, which also involve dopamine and endocannabinoid signalling. Despite significant progress in recent years, many important questions remain unanswered, especially in relation to long-term potentiation. Of particular interest is the question of how to link the molecular and cellular mechanisms of synaptic plasticity to learning operations at the systems level, which are expressed behaviourally as reinforcement-related learning.

Multiple memory stores and operant conditioning: a rationale for memory's complexity

Why does the brain contain more than one memory system? Genetic algorithms can play a role in elucidating this question. Here, model animals were constructed containing a dorsal striatal layer that controlled actions, and a ventral striatal layer that controlled a dopaminergic learning signal. Both layers could gain access to three modeled memory stores, but such access was penalized as energy expenditure. Model animals were then selected on their fitness in simulated operant conditioning tasks. Results suggest that having access to multiple memory stores and their representations is important in learning to regulate dopamine release, as well as in contextual discrimination. For simple operant conditioning, as well as stimulus discrimination, hippocampal compound representations turned out to suffice, a counterintuitive result given findings that hippocampal lesions tend not to affect performance in such tasks. We argue that there is in fact evidence to support a role for compound representations and the hippocampus in even the simplest conditioning tasks.

Influence of reward delays on responses of dopamine neurons

Psychological and microeconomic studies have shown that outcome values are discounted by imposed delays. The effect, called temporal discounting, is demonstrated typically by choice preferences for sooner smaller rewards over later larger rewards. However, it is unclear whether temporal discounting occurs during the decision process when differently delayed reward outcomes are compared or during predictions of reward delays by pavlovian conditioned stimuli without choice. To address this issue, we investigated the temporal discounting behavior in a choice situation and studied the effects of reward delay on the value signals of dopamine neurons. The choice behavior confirmed hyperbolic discounting of reward value by delays on the order of seconds. Reward delay reduced the responses of dopamine neurons to pavlovian conditioned stimuli according to a hyperbolic decay function similar to that observed in choice behavior. Moreover, the stimulus responses increased with larger reward magnitudes, suggesting that both delay and magnitude constituted viable components of dopamine value signals. In contrast, dopamine responses to the reward itself increased with longer delays, possibly reflecting temporal uncertainty and partial learning. These dopamine reward value signals might serve as useful inputs for brain mechanisms involved in economic choices between delayed rewards.

Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation

Dopamine in the nucleus accumbens (NAc) is an important neurotransmitter for reward-seeking behaviors such as intracranial self-stimulation (ICSS), although its precise role remains unclear. Here, dynamic fluctuations in extracellular dopamine were measured during ICSS in the rat NAc shell with fast-scan cyclic voltammetry at carbon-fiber microelectrodes. Rats were trained to press a lever to deliver electrical stimulation to the substantia nigra (SNc)/ventral tegmental area (VTA) after the random onset of a cue that predicted reward availability. Latency to respond after cue onset significantly declined across trials, indicative of learning. Dopamine release was evoked by the stimulation but also developed across trials in a time-locked fashion to the cue. Once established, the cue-evoked dopamine transients continued to grow in amplitude, although they were variable from trial to trial. The emergence of cue-evoked dopamine correlated with a decline in electrically evoked dopamine release. Extinction of ICSS resulted in a significant decline in goal-directed behavior coupled to a significant decrease in cue-evoked phasic dopamine across trials. Subsequent reinstatement of ICSS was correlated with a return to preextinction transient amplitudes in response to the cue and reestablishment of ICSS behavior. The results show the dynamic nature of chemical signaling in the NAc during ICSS and provide new insight into the role of NAc dopamine in reward-related behaviors.

Research review: dopamine transfer deficit: a neurobiological theory of altered reinforcement mechanisms in ADHD

This review considers the hypothesis that changes in dopamine signalling might account for altered sensitivity to positive reinforcement in children with ADHD. The existing evidence regarding dopamine cell activity in relation to positive reinforcement is reviewed. We focus on the anticipatory firing of dopamine cells brought about by a transfer of dopamine cell responses to cues that precede reinforcers. It is proposed that in children with ADHD there is diminished anticipatory dopamine cell firing, which we call the dopamine transfer deficit (DTD). The DTD theory leads to specific and testable predictions for human and animal research. The extent to which DTD explains symptoms of ADHD and effects of pharmacological interventions is discussed. We conclude by considering the neural changes underlying the etiology of DTD.

Neonatal tactile stimulation enhances spatial working memory, prefrontal long-term potentiation, and D1 receptor activation in adult rats

Environmental stimuli during neonatal periods play an important role in the development of cognitive function. In this study, we examined the long-term effects of neonatal tactile stimulation (TS) on spatial working memory (SWM) and related mechanisms. We also investigated whether TS-induced effects could be counteracted by repeated short periods of maternal separation (MS). Wistar rat pups submitted to TS were handled and marked transiently per day during postnatal days 2-9 or 10-17. TS/MS pups were stimulated in the same way as TS pups and then individually separated from their mother for 1h/day. Their nontactile stimulated (NTS) siblings served as controls. In adulthood, TS and TS/MS rats showed better performance in two versions of the delayed alternation task and superior in vivo long-term potentiation of the hippocampo-prefrontal cortical pathway when compared with controls. Furthermore, there were more doses of A77636 (a selective dopamine D1 agonist) to significantly improve SWM performance in TS and TS/MS rats than in NTS rats, suggesting that activation of prefrontal D1 receptors in TS and TS/MS rats is more optimal for SWM function than in NTS rats. MS did not counteract TS-induced effects because no significant difference was found between TS/MS and TS animals. These data indicate that in early life, external tactile stimulation leads to long-term facilitative effects in SWM-related neural function.

Striatal Contributions to Reward and Decision Making: Making Sense of Regional Variations in a Reiterated Processing Matrix

The striatum is the major input nucleus of the basal ganglia. It is thought to play a key role in learning on the basis of positive reinforcement and in action selection. One view of the striatum conceives it as comprising a reiterated matrix of processing units that perform common operations in different striatal regions, namely synaptic plasticity according to a three-factor rule, and lateral inhibition. These operations are required for reinforcement learning and selection of previously reinforced actions. Analysis of the behavioral effects of circumscribed lesions of the striatum, however, suggests regional specialization of learning and decision-making operations. We consider how a basic processing unit may be modified by regional variations in neurochemical parameters, for example, by the gradient in density of dopamine terminals from dorsal to ventral striatum. These variations suggest subtle differences between dorsolateral and ventromedial striatal regions in the temporal properties of dopamine signaling, which are superimposed on regional differences in connectivity. We propose that these variations make sense in relation to the temporal structure of activity in striatal inputs from different regions, and the requirements of different learning operations. Dorsolateral striatal (DLS) regions may be subject to brief, precisely timed pulses of dopamine, whereas ventromedial striatal regions integrate dopamine signals over a longer time course. These differences may be important for understanding regional variations in the contribution to reinforcement of habits, versus incentive processes that are sensitive to the value of expected rewards.

Tuning the engine of cognition: A focus on NMDA/D1 receptor interactions in prefrontal cortex

The prefrontal cortex of the primate frontal lobes provides the capacity for judgment which can constantly adapt behavior in order to optimize its outcome. Adjudicating between long-term memory programs and prepotent responses, this capacity reviews all incoming information and provides an interpretation dependent on the events that have just occurred, the events that are predicted to happen, and the alternative response strategies that are available in the given situation. It has been theorized that this function requires two essential integrated components, a central executive which guides selective attention based on mechanisms of associative memory, as well as the second component, working memory buffers, in which information is held online, abstracted, and translated on a mental sketchpad of work in progress. In this review, we critically outline the evidence that the integration of these processes and, in particular, the induction and maintenance of persistent activity in prefrontal cortex and related networks, is dependent upon the interaction of dopamine D1 and glutamate NMDA receptor signaling at critical nodes within local circuits and distributed networks. We argue that this interaction is not only essential for representational memory, but also core to mechanisms of neuroadaptation and learning. Understanding its functional significance promises to reveal major new insights into prefrontal dysfunction in schizophrenia and, hence, to target a new generation of drugs designed to ameliorate the debilitating working memory deficits in this disorder.

Differential contributions of dopaminergic D1- and D2-like receptors to cognitive function in rhesus monkeys

RATIONALE

Dopaminergic neurotransmission is critically involved in many aspects of complex behavior and cognition beyond reward/reinforcement and motor function. Mental and behavioral disorders associated with major disruptions of dopamine neurotransmission, including schizophrenia, attention deficit/hyperactivity disorder, Parkinson's disease, Huntington's disease, and substance abuse produce constellations of neuropsychological deficits in learning, memory, and attention in addition to other defining symptoms.

OBJECTIVE

To delineate the role dopaminergic D1- and D2-like receptor subtypes play in complex brain functions.

MATERIALS AND METHODS

Monkeys (N = 6) were trained on cognitive tests adapted from a human neuropsychological assessment battery (CAmbridge Neuropsychological Test Automated Battery). The battery included tests of spatial working memory (self-ordered spatial search task), visuo-spatial associative memory and learning (visuo-spatial paired associates learning task, vsPAL) and motivation (progressive ratio task, PR). Tests of motor function (bimanual motor skill task, BMS; rotating turntable task, RTT) were also included. The effects of the dopamine D2-like antagonist raclopride (10-56 microg/kg, i.m.) and the D1-like antagonist SCH23390 (SCH, 3.2-56 microg/kg, i.m.) on cognitive performance were then determined.

RESULTS

Deficits on PR, RTT, and BMS performance were observed after both raclopride and SCH23390. Spatial working memory accuracy was reduced to a greater extent by raclopride than by SCH, which was unexpected, given prior reports on the involvement of D1 signaling for spatial working memory in monkeys. Deficits were observed on vsPAL performance after raclopride, but not after SCH23390.

CONCLUSIONS

The intriguing results suggest a greater contribution of D2- over D1-like receptors to both spatial working memory and object-location associative memory.

Role of brain dopamine in food reward and reinforcement

The ability of food to establish and maintain response habits and conditioned preferences depends largely on the function of brain dopamine systems. While dopaminergic transmission in the nucleus accumbens appears sufficient for some forms of reward, the role of dopamine in food reward does not appear to be restricted to this region. Dopamine plays an important role in both the ability to energize feeding and to reinforce food-seeking behaviour; the role in energizing feeding is secondary to the prerequisite role in reinforcement. Dopaminergic activation is triggered by the auditory and visual as well as the tactile, olfactory, and gustatory stimuli of foods. While dopamine plays a central role in the feeding and food-seeking of normal animals, some food rewarded learning can be seen in genetically engineered dopamine-deficient mice.

Role of dopamine in the primate caudate nucleus in reward modulation of saccades

Expected reward impacts behavior and neuronal activity in brain areas involved in sensorimotor processes. However, where and how reward signals affect sensorimotor signals is unclear. Here, we show evidence that reward-dependent modulation of behavior depends on normal dopamine transmission in the striatum. Monkeys performed a visually guided saccade task in which expected reward gain was different depending on the position of the target. Saccadic reaction times were reliably shorter on large-reward trials than on small-reward trials. When position-reward contingency was switched, the reaction time difference changed rapidly. Injecting dopamine D1 antagonist into the caudate significantly attenuated the reward-dependent saccadic reaction time changes. Conversely, injecting D2 antagonist into the same region enhanced the reward-dependent changes. These results suggest that reward-dependent changes in saccadic eye movements depend partly on dopaminergic modulation of neuronal activity in the caudate nucleus.

A functional magnetic resonance imaging study of the effects of pergolide, a dopamine receptor agonist, on component processes of working memory

Working memory is an important cognitive process dependent on a network of prefrontal and posterior cortical regions. In this study we tested the effects of the mixed D1-D2 dopamine receptor agonist pergolide on component processes of human working memory using functional magnetic resonance imaging (fMRI). An event-related trial design allowed separation of the effects on encoding, maintenance, and retrieval processes. Subjects were tested with spatial and object memoranda to investigate modality-specific effects of dopaminergic stimulation. We also measured baseline working memory capacity as previous studies have shown that effects of dopamine agonists vary with working memory span. Pergolide improved reaction time for high-span subjects and impaired reaction time for low-span subjects. This span-dependent change in behavior was accompanied by span-dependent changes in delay-related activity in the premotor cortex. We also found evidence for modality-specific effects of pergolide only during the response period. Pergolide increased activity for spatial memoranda and decreased activity for object memoranda in task-related regions including the prefrontal and parietal cortices.

Neurons in dopamine-rich areas of the rat medial midbrain predominantly encode the outcome-related rather than behavioural switching properties of conditioned stimuli

Midbrain dopamine neurons are phasically activated by a variety of sensory stimuli. It has been hypothesized that these activations contribute to reward prediction or behavioural switching. To test the latter hypothesis we recorded from 131 single neurons in the ventral tegmental area and retrorubral field of thirsty rats responding during a modified go/no-go task. One-quarter (n = 33) of these neurons responded to conditioned stimuli in the task, which varied according to the outcome with which they were associated (saccharin or quinine solution) and according to whether they triggered a switch in the ongoing sequence of the animal's behaviour ('behavioural switching'). Almost half the neurons (45%) responded differentially to saccharin- vs. quinine-conditioned stimuli; the activity of a minority (15%) correlated with an aspect of behavioural switching (mostly exhibiting changes from baseline activity in the absence of a behavioural switch) and one-third (33%) encoded various outcome-switch combinations. The strongest response was excitation to the saccharin-conditioned stimulus. Additionally, a proportion (38%) of neurons responded during outcome delivery, typically exhibiting inhibition during saccharin consumption. The neurons sampled did not fall into distinct clusters on the basis of their electrophysiological characteristics. However, most neurons that responded to the outcome-related properties of conditioned stimuli had long action potentials (> 1.2 ms), a reported characteristic of dopamine neurons. Moreover, responses to saccharin-conditioned stimuli were functionally akin to dopamine responses found in the macaque and rat nucleus accumbens responses observed within the same task. In conclusion, our data are more consistent with the reward-prediction than the behavioural switching hypothesis.