Role of dopamine tone in the pursuit of brain stimulation reward

Dorsal raphe stimulation relays a reward signal to the ventral tegmental area via GluN2C NMDA receptors

BACKGROUND

Glutamate relays a reward signal from the dorsal raphe (DR) to the ventral tegmental area (VTA). However, the role of the different subtypes of N-methyl-D-aspartate (NMDA) receptors is complex and not clearly understood. Therefore, we measured NMDA receptors subunits expression in limbic brain areas. In addition, we studied the effects of VTA down-regulation of GluN2C NMDA receptor on the reward signal that arises from DR electrical stimulation.

METHODS

Using qPCR, we identified the relative composition of the different Grin2a-d subunits of the NMDA receptors in several brain areas. Then, we used fluorescent in situ hybridization (FISH) to evaluate the colocalization of Grin2c and tyrosine hydroxylase (TH) mRNA in VTA neurons. To assess the role of GluN2C in brain stimulation reward, we downregulated this receptor using small interfering RNA (siRNA) in rats self-stimulating for electrical pulses delivered to the DR. To delineate further the specific role of GluN2C in relaying the reward signal, we pharmacologically altered the function of VTA NMDA receptors by bilaterally microinjecting the NMDA receptor antagonist PPPA.

RESULTS

We identified GluN2C as the most abundant subunit of the NMDA receptor expressed in the VTA. FISH revealed that about 50% of TH-positive neurons colocalize with Grin2c transcript. siRNA manipulation produced a selective down-regulation of the GluN2C protein subunit and a significant reduction in brain stimulation reward. Interestingly, PPPA enhanced brain stimulation reward, but only in rats that received the nonactive RNA sequence.

CONCLUSION

The present results suggest that VTA glutamate neurotransmission relays a reward signal initiated by DR stimulation by acting on GluN2C NMDA receptors.

The priming effect of rewarding brain stimulation in rats depends on both the cost and strength of reward but survives blockade of D2-like dopamine receptors

Receipt of an intense reward boosts motivation to work for more of that reward. This phenomenon is called the priming effect of rewards. Using a novel measurement method, we show that the priming effect of rewarding electrical brain stimulation depends on the cost, as well as on the strength, of the anticipated reward. Previous research on the priming effect of electrical brain stimulation utilized a runway paradigm in which running speed serves as the measure of motivation. In the present study, the measure of motivation was the vigour with which rats executed a two-lever response chain, in a standard operant-conditioning chamber, to earn rewarding electrical stimulation of the lateral hypothalamus. In a second experiment, we introduced a modification that entails self-administered priming stimulation and alternating blocks of primed and unprimed trials. Reliable, consistent priming effects of substantial magnitude were obtained in the modified paradigm, which is closely analogous to the runway paradigm. In a third experiment, the modified paradigm served to assess the dependence of the priming effect on dopamine D2-like receptors. The priming effect proved resilient to the effect of eticlopride, a selective D2-like receptor antagonist. These results are discussed within the framework of a new model of brain reward circuitry in which non-dopaminergic medial forebrain bundle fibers and dopamine axons provide parallel inputs to the final common paths for reward and incentive motivation.

Comparison of progressive hold and progressive response schedules of reinforcement

Progressive ratio (PR) schedules have been widely used to study motivation to work for a reinforcer. After a post-reinforcer pause, subjects engage in pressing a lever until a reinforcer is obtained. However, the discrete nature of lever presses allows alternative behaviors during inter-response time and has led to several behavioral explanations of pauses and work time. A progressive hold-down (PH) is incompatible with alternative responses and may allow a precise estimation of work time. Performance of rats trained in both PR and PH that received sucrose or intracranial self-stimulation (ICSS) as reinforcer were compared. We observed that rats mastered the PR and PH schedules. Post-reinforcer pauses (PS^RP), work time and inter-reinforcer time increased as a function of the response or hold requirement. However, rats' performance suggested that the PH progression may be experienced by the rats as easier that the PR progression. Elimination of consummatory behavior with ICSS reduced post-reinforcer pause in accordance with predictions of explanatory models of fixed and variable schedules of reinforcement. In the case of PH performance, pauses showed little variation across intermediate requirements but increased rapidly on later requirements; since rats controlled their pause length and work time was close to the hold requirement, time allocation between PR and PH schedules diverged. Finally, the Mathematical Principles of Reinforcement model of Bradshaw and Killeen (Psychopharmacology 2012, 222: 549) rendered a good description of the performance in both PR and PH schedules.

Dopamine and Beyond: Implications of Psychophysical Studies of Intracranial Self-Stimulation for the Treatment of Depression

Major depressive disorder is a leading cause of disability and suicide worldwide. Consecutive rounds of conventional interventions are ineffective in a significant sub-group of patients whose disorder is classified as treatment-resistant depression. Significant progress in managing this severe form of depression has been achieved through the use of deep brain stimulation of the medial forebrain bundle (MFB). The beneficial effect of such stimulation appears strong, safe, and enduring. The proposed neural substrate for this promising clinical finding includes midbrain dopamine neurons and a subset of their cortical afferents. Here, we aim to broaden the discussion of the candidate circuitry by exploring potential implications of a new “convergence” model of brain reward circuitry in rodents. We chart the evolution of the new model from its predecessors, which held that midbrain dopamine neurons constituted an obligatory stage of the final common path for reward seeking. In contrast, the new model includes a directly activated, non-dopaminergic pathway whose output ultimately converges with that of the dopaminergic neurons. On the basis of the new model and the relative ineffectiveness of dopamine agonists in the treatment of depression, we ask whether non-dopaminergic circuitry may contribute to the clinical efficacy of deep brain stimulation of the MFB.

The Convergence Model of Brain Reward Circuitry: Implications for Relief of Treatment-Resistant Depression by Deep-Brain Stimulation of the Medial Forebrain Bundle

Deep-brain stimulation of the medial forebrain bundle (MFB) can provide effective, enduring relief of treatment-resistant depression. Panksepp provided an explanatory framework: the MFB constitutes the core of the neural circuitry subserving the anticipation and pursuit of rewards: the “SEEKING” system. On that view, the SEEKING system is hypoactive in depressed individuals; background electrical stimulation of the MFB alleviates symptoms by normalizing activity. Panksepp attributed intracranial self-stimulation to excitation of the SEEKING system in which the ascending projections of midbrain dopamine neurons are an essential component. In parallel with Panksepp’s qualitative work, intracranial self-stimulation has long been studied quantitatively by psychophysical means. That work argues that the predominant directly stimulated substrate for MFB self-stimulation are myelinated, non-dopaminergic fibers, more readily excited by brief electrical current pulses than the thin, unmyelinated axons of the midbrain dopamine neurons. The series-circuit hypothesis reconciles this view with the evidence implicating dopamine in MFB self-stimulation as follows: direct activation of myelinated MFB fibers is rewarding due to their trans-synaptic activation of midbrain dopamine neurons. A recent study in which rats worked for optogenetic stimulation of midbrain dopamine neurons challenges the series-circuit hypothesis and provides a new model of intracranial self-stimulation in which the myelinated non-dopaminergic neurons and the midbrain dopamine projections access the behavioral final common path for reward seeking via separate, converging routes. We explore the potential implications of this convergence model for the interpretation of the antidepressant effect of MFB stimulation. We also discuss the consistent finding that psychomotor stimulants, which boost dopaminergic neurotransmission, fail to provide a monotherapy for depression. We propose that non-dopaminergic MFB components may contribute to the therapeutic effect in parallel to, in synergy with, or even instead of, a dopaminergic component.

On the Similarity Between the Reinforcing and the Discriminative Properties of Intracranial Self-Stimulation

Rats work very hard for intracranial self-stimulation (ICSS) and tradeoff effort or time allocation for intensity and frequency parameters producing a sigmoidal function of the subjective reward magnitude of ICSS. Previous studies using electrical intracranial stimuli (ICS) as a discriminative cue focused on estimating detection thresholds or on the discrimination between intensities. To our knowledge, there is no direct comparison of the reinforcer tradeoff functions with the discriminative functions. Rats were trained to press and hold the lever for ICSS using the maximum reinforcing intensity below motor alterations or avoidance behavior. First, rats were trained to hold the lever for 1 s; after stability, they undergo trials where intensity or frequency was decreased on 0.1 log step. Thereafter, they undergo further training with a hold of 2 and later of 4 s to determine tradeoff with intensity or frequency. The same rats were trained on a discrimination task where the previously used ICSS signaled a lever where a 1 s hold response was followed by a reinforcing ICSS; on randomly alternating trials, a −0.6 log ICS signaled an alternate lever where a similar hold response led to a reinforcer. After mastering discrimination, generalization tests were carried out with varying intensity or frequency. Rats completed training with 2 and later 4 s hold response. After the completion of each task, the rats had different doses of a pimozide challenge while their intensity and hold-down requirement were varied. With regards to the rats’ tradeoff response time allocation as a function of intensity or frequency, sigmoid functions were displaced to the right when long responses were required. Rats that learned the discrimination task attained a discrimination index of 90–98%. Discrimination accuracy decreased slightly with the increase of hold requirement, but generalization gradients were not displaced to the right as a function of the response requirement. Pimozide induced a dose-dependent displacement of the time-allocation gradients, but it did not affect the generalization gradients. It is concluded that rats integrate response requirements as part of the reinforcement tradeoff function, but the response cost is not integrated into the discriminative function of ICSS.

Chronic nicotine increases midbrain dopamine neuron activity and biases individual strategies towards reduced exploration in mice

Long-term exposure to nicotine alters brain circuits and induces profound changes in decision-making strategies, affecting behaviors both related and unrelated to drug seeking and consumption. Using an intracranial self-stimulation reward-based foraging task, we investigated in mice the impact of chronic nicotine on midbrain dopamine neuron activity and its consequence on the trade-off between exploitation and exploration. Model-based and archetypal analysis revealed substantial inter-individual variability in decision-making strategies, with mice passively exposed to nicotine shifting toward a more exploitative profile compared to non-exposed animals. We then mimicked the effect of chronic nicotine on the tonic activity of dopamine neurons using optogenetics, and found that photo-stimulated mice adopted a behavioral phenotype similar to that of mice exposed to chronic nicotine. Our results reveal a key role of tonic midbrain dopamine in the exploration/exploitation trade-off and highlight a potential mechanism by which nicotine affects the exploration/exploitation balance and decision-making.

Optogenetic brain-stimulation reward: A new procedure to re-evaluate the rewarding versus aversive effects of cannabinoids in dopamine transporter-Cre mice

Despite extensive research, the rewarding effects of cannabinoids are still debated. Here, we used a newly established animal procedure called optogenetic intracranial self-stimulation (ICSS) (oICSS) to re-examine the abuse potential of cannabinoids in mice. A specific adeno-associated viral vector carrying a channelrhodopsin gene was microinjected into the ventral tegmental area (VTA) to express light-sensitive channelrhodopsin in dopamine (DA) neurons of transgenic dopamine transporter (DAT)-Cre mice. Optogenetic stimulation of VTA DA neurons was highly reinforcing and produced a classical "sigmoidal"-shaped stimulation-response curve dependent upon the laser pulse frequency. Systemic administration of cocaine dose-dependently enhanced oICSS and shifted stimulation-response curves upward, in a way similar to previously observed effects of cocaine on electrical ICSS. In contrast, Δ⁹ -tetrahydrocannabinol (Δ⁹ -THC), but not cannabidiol, dose-dependently decreased oICSS responding and shifted oICSS curves downward. WIN55,212-2 and ACEA, two synthetic cannabinoids often used in laboratory settings, also produced dose-dependent reductions in oICSS. We then examined several new synthetic cannabinoids, which are used recreationally. XLR-11 produced a cocaine-like increase, AM-2201 produced a Δ⁹ -THC-like reduction, while 5F-AMB had no effect on oICSS responding. Immunohistochemistry and RNAscope in situ hybridization assays indicated that CB₁ Rs are expressed mainly in VTA GABA and glutamate neurons, while CB₂ Rs are expressed mainly in VTA DA neurons. Together, these findings suggest that most cannabinoids are not reward enhancing, but rather reward attenuating or aversive in mice. Activation of CB₁ R and/or CB₂ R in different populations of neurons in the brain may underlie the observed actions.

Modulation of brain stimulation reward and locomotor activity by ionotropic glutamate receptors of the tail of the ventral tegmental area

The rostromedial tegmental nucleus also referred to as the tail of the ventral tegmental area (tVTA) contains a cluster of gamma-aminobutyric acid (GABA)ergic neurons that receive dense glutamatergic afferents from the lateral habenula (LHb), and project to dopamine (DA) neurons of the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc). In light of previous evidence implicating glutamate transmission in the regulation of midbrain DA neuronal activity, we first assessed the impact of intra-tVTA microinjection of NBQX (0.8 nmol/side) and PPPA (0.825 nmol/side), respectively AMPA and NMDA receptor antagonists, on reward induced by intracranial self-stimulation (ICSS) and on locomotor activity. Since the tVTA contains a large concentration of mu opioid receptors, additional measures were obtained following microinjection of endomorphin-1 (EM-1, 1 nmol/side) to confirm tVTA placements. Then, using small interfering RNAs (siRNAs), we tested the effect of tVTA downregulation of the GluN1 subunit of the NMDA receptor on reward and locomotor activity. Results show that NBQX, PPPA and EM-1 all enhance reward and locomotor activity, effects that were of different magnitude in rostral and intermediate parts of the tVTA. On the other hand, a reduction in GluN1 subunits used a marked decrease in operant responding for ICSS, but failed to alter ICSS reward and the reward-enhancing effect of PPPA. Our results support a role for the tVTA as a main inhibitory component of DA-dependent behavioral measures, and suggest that tVTA NMDA receptors that modulate reward are most likely expressed on tVTA afferent terminals.

Dopamine neurons do not constitute an obligatory stage in the final common path for the evaluation and pursuit of brain stimulation reward

The neurobiological study of reward was launched by the discovery of intracranial self-stimulation (ICSS). Subsequent investigation of this phenomenon provided the initial link between reward-seeking behavior and dopaminergic neurotransmission. We re-evaluated this relationship by psychophysical, pharmacological, optogenetic, and computational means. In rats working for direct, optical activation of midbrain dopamine neurons, we varied the strength and opportunity cost of the stimulation and measured time allocation, the proportion of trial time devoted to reward pursuit. We found that the dependence of time allocation on the strength and cost of stimulation was similar formally to that observed when electrical stimulation of the medial forebrain bundle served as the reward. When the stimulation is strong and cheap, the rats devote almost all their time to reward pursuit; time allocation falls off as stimulation strength is decreased and/or its opportunity cost is increased. A 3D plot of time allocation versus stimulation strength and cost produces a surface resembling the corner of a plateau (the “reward mountain”). We show that dopamine-transporter blockade shifts the mountain along both the strength and cost axes in rats working for optical activation of midbrain dopamine neurons. In contrast, the same drug shifted the mountain uniquely along the opportunity-cost axis when rats worked for electrical MFB stimulation in a prior study. Dopamine neurons are an obligatory stage in the dominant model of ICSS, which positions them at a key nexus in the final common path for reward seeking. This model fails to provide a cogent account for the differential effect of dopamine transporter blockade on the reward mountain. Instead, we propose that midbrain dopamine neurons and neurons with non-dopaminergic, MFB axons constitute parallel limbs of brain-reward circuitry that ultimately converge on the final-common path for the evaluation and pursuit of rewards.

Differential rewarding effects of electrical stimulation of the lateral hypothalamus and parabrachial complex: Functional characterization and the relevance of opioid systems and dopamine

BACKGROUND

Since the discovery of rewarding intracranial self-stimulation by Olds and Milner, extensive data have been published on the biological basis of reward. Although participation of the mesolimbic dopaminergic system is well documented, its precise role has not been fully elucidated, and some authors have proposed the involvement of other neural systems in processing specific aspects of reinforced behaviour.

AIMS AND METHODS

We reviewed published data, including our own findings, on the rewarding effects induced by electrical stimulation of the lateral hypothalamus (LH) and of the external lateral parabrachial area (LPBe) - a brainstem region involved in processing the rewarding properties of natural and artificial substances - and compared its functional characteristics as observed in operant and non-operant behavioural procedures.

RESULTS

Brain circuits involved in the induction of preferences for stimuli associated with electrical stimulation of the LBPe appear to functionally and neurochemically differ from those activated by electrical stimulation of the LH.

INTERPRETATION

We discuss the possible involvement of the LPBe in processing emotional-affective aspects of the brain reward system.

The priming effect of food persists following blockade of dopamine receptors

The priming effect of rewards is a boost in the vigor of reward seeking resulting from the previous receipt of a reward. Extensive work has been carried out on the priming effect of electrical brain stimulation, but much less research exists on the priming effect of natural rewards, such as food. While both reinforcement and motivation are linked with dopamine transmission in the brain, the priming effect of rewards does not appear to be dopamine-dependent. In the present study, an operant method was developed to measure the priming effect of food and then applied to investigate whether it is affected by dopamine receptor antagonism. Long-Evans rats were administered saline or one of the three doses (0.01, 0.05, 0.075 mg/kg) of the dopamine D1 receptor family antagonist, SCH23390, or the dopamine D2 receptor family antagonist, eticlopride. Although dopamine receptor antagonism affected pursuit of food, it did not eliminate the priming effect. These data suggest that despite the involvement of dopamine transmission in reinforcement and motivation, the priming effect of food does not depend on dopamine transmission.

The power of price compels you: Behavioral economic insights into dopamine-based valuation of rewarding and aversively motivated behavior

The mesocorticolimbic dopamine pathway is generally considered to be a reward pathway. While shortsighted, there is a strong basis for this view of dopamine function. Here, we first describe the role of phasic dopamine release events in reward seeking. We then explain why these release events are being reconsidered as value signals and how we applied behavioral economics to confirm they play a causal role in the valuation of reward. Just because dopamine release can function as a dopamine reward value signal however, does not imply that dopamine is solely a reward molecule. Rather, mesocorticolimbic dopamine appears to mediate many adaptive behaviors, including: reward seeking, avoidance, escape and fear-associated conditioned freezing. While more studies are needed before a consensus is reached on when, where and how dopamine mediates aversively-motivated behavior, its involvement is almost irrefutable. Thus, we next describe the role dopamine plays in these ethologically-relevant defensive behaviors. We conclude by describing our recent behavioral economics results that reveal a causal role for dopamine in the valuation of avoidance.

Optogenetic self-stimulation in the nucleus accumbens: D1 reward versus D2 ambivalence

The nucleus accumbens (NAc) contains multiple subpopulations of medium spiny neurons (MSNs). One subpopulation expresses D1-type dopamine receptors, another expresses D2-type receptors, and a third expresses both. The relative roles in NAc of D1 neurons versus D2 neurons in appetitive motivation were assessed here. Specifically, we asked whether D1-Cre mice would instrumentally seek optogenetic self-stimulation specifically targeted at D1 MSNs in NAc, and similarly if D2-Cre mice would self-stimulate D2 neurons in NAc. Mice were implanted with Cre-targeted channelrhodopsin (ChR2) virus and optic fibers in NAc. Subsequently, mice could earn brief NAc laser illuminations by actively touching a metal spout in one task, or by going to a particular location in a separate task. Results indicated that D1 neuronal excitation in NAc supported intense self-stimulation in both tasks. D1-Cre mice earned hundreds to thousands of spout-touches per half-hour session, and also sought out locations that delivered NAc laser to excite D1 MSNs. By comparison, D2 ChR2 mice showed lower but still positive levels of self-stimulation in the spout-touch task, earning dozens to hundreds of NAc laser illuminations. However, in the location task, D2 mice failed to show positive self-stimulation. If anything, a few D2 individuals gradually avoided the laser location. Brain-wide measures indicated that D1 and D2 stimulations in NAc recruited heavily overlapping patterns of Fos activation in distant limbic structures. These results confirm that excitation of D1 MSNs in NAc supports strong incentive motivation to self-stimulate. They also suggest that excitation of D2 neurons in NAc supports self-stimulation under some conditions, but fails under others and possibly may even shift to negative avoidance.

Dissociation of Appetitive Overexpectation and Extinction in the Infralimbic Cortex

Behavioral change is paramount to adaptive behavior. Two ways to achieve alterations in previously established behavior are extinction and overexpectation. The infralimbic (IL) portion of the medial prefrontal cortex controls the inhibition of previously established aversive behavioral responses in extinction. The role of the IL cortex in behavioral modification in appetitive Pavlovian associations remains poorly understood. Here, we seek to determine if the IL cortex modulates overexpectation and extinction of reward learning. Using overexpectation or extinction to achieve a reduction in behavior, the present findings uncover a dissociable role for the IL cortex in these paradigms. Pharmacologically inactivating the IL cortex left overexpectation intact. In contrast, pre-training manipulations in the IL cortex prior to extinction facilitated the reduction in conditioned responding but led to a disrupted extinction retrieval on test drug-free. Additional studies confirmed that this effect is restricted to the IL and not dependent on the dorsally-located prelimbic cortex. Together, these results show that the IL cortex underlies extinction but not overexpectation-driven reduction in behavior, which may be due to regulating the expression of conditioned responses influenced by stimulus–response associations rather than stimulus–stimulus associations.

Modeling human methamphetamine use patterns in mice: chronic and binge methamphetamine exposure, reward function and neurochemistry

Different methamphetamine use patterns in human subjects may contribute to inconsistent findings regarding the effects of methamphetamine abuse on brain and behavior. The present study investigated whether human-derived chronic and binge methamphetamine use patterns have differential effects on reward and neurochemistry in mice. Brain reward function in mice was evaluated during acute/prolonged withdrawal, and in response to methamphetamine challenge using the intracranial self-stimulation procedure. Brain dopaminergic, serotonergic and glutamatergic neurochemistry was determined with high-performance liquid chromatography. Chronic and binge regimens induced withdrawal-related decreases in reward function that were more severe during the binge regimen during cycles 1-2. Despite large differences in methamphetamine dose, both regimens induced similar reward deficits during cycles 3-4. Neither methamphetamine regimen led to persistent alterations in the sensitivity to the reward-enhancing effects of acute methamphetamine challenge. The binge regimen severely depleted striatal dopamine levels and increased brain glutamine levels. The chronic regimen had milder effects on striatal dopamine levels and altered cortical dopamine and serotonin levels. This work highlights that the magnitude of acute/prolonged withdrawal may not reflect amount or frequency of methamphetamine intake. In contrast, the array of underlying neurochemical alterations was methamphetamine regimen dependent. Thus, stratifying methamphetamine-dependent individuals based on use pattern may help to cater therapeutic interventions more appropriately by targeting use pattern-specific neurotransmitter systems.

Cross-Species Alterations in Synaptic Dopamine Regulation After Chronic Alcohol Exposure

Alcohol use disorders are a leading public health concern, engendering enormous costs in terms of both economic loss and human suffering. These disorders are characterized by compulsive and excessive alcohol use, as well as negative affect and alcohol craving during abstinence. Extensive research has implicated the dopamine system in both the acute pharmacological effects of alcohol and the symptomology of alcohol use disorders that develop after extended alcohol use. Preclinical research has shed light on many mechanisms by which chronic alcohol exposure dysregulates the dopamine system. However, many of the findings are inconsistent across experimental parameters such as alcohol exposure length, route of administration, and model organism. We propose that the dopaminergic alterations driving the core symptomology of alcohol use disorders are likely to be relatively stable across experimental settings. Recent work has been aimed at using multiple model organisms (mouse, rat, monkey) across various alcohol exposure procedures to search for commonalities. Here, we review recent advances in our understanding of the effects of chronic alcohol use on the dopamine system by highlighting findings that are consistent across experimental setting and species.

A transient dopamine signal encodes subjective value and causally influences demand in an economic context

The mesolimbic dopamine system is strongly implicated in motivational processes. Currently accepted theories suggest that transient mesolimbic dopamine release events energize reward seeking and encode reward value. During the pursuit of reward, critical associations are formed between the reward and cues that predict its availability. Conditioned by these experiences, dopamine neurons begin to fire upon the earliest presentation of a cue, and again at the receipt of reward. The resulting dopamine concentration scales proportionally to the value of the reward. In this study, we used a behavioral economics approach to quantify how transient dopamine release events scale with price and causally alter price sensitivity. We presented sucrose to rats across a range of prices and modeled the resulting demand curves to estimate price sensitivity. Using fast-scan cyclic voltammetry, we determined that the concentration of accumbal dopamine time-locked to cue presentation decreased with price. These data confirm and extend the notion that dopamine release events originating in the ventral tegmental area encode subjective value. Using optogenetics to augment dopamine concentration, we found that enhancing dopamine release at cue made demand more sensitive to price and decreased dopamine concentration at reward delivery. From these observations, we infer that value is decreased because of a negative reward prediction error (i.e., the animal receives less than expected). Conversely, enhancing dopamine at reward made demand less sensitive to price. We attribute this finding to a positive reward prediction error, whereby the animal perceives they received a better value than anticipated.

Valuation of opportunity costs by rats working for rewarding electrical brain stimulation

Pursuit of one goal typically precludes simultaneous pursuit of another. Thus, each exclusive activity entails an “opportunity cost:” the forgone benefits from the next-best activity eschewed. The present experiment estimates, in laboratory rats, the function that maps objective opportunity costs into subjective ones. In an operant chamber, rewarding electrical brain stimulation was delivered when the cumulative time a lever had been depressed reached a criterion duration. The value of the activities forgone during this duration is the opportunity cost of the electrical reward. We determined which of four functions best describes how objective opportunity costs, expressed as the required duration of lever depression, are translated into their subjective equivalents. The simplest account is the identity function, which equates subjective and objective opportunity costs. A variant of this function called the “sigmoidal-slope function,” converges on the identity function at longer durations but deviates from it at shorter durations. The sigmoidal-slope function has the form of a hockey stick. The flat “blade” denotes a range over which opportunity costs are subjectively equivalent; these durations are too short to allow substitution of more beneficial activities. The blade extends into an upward-curving portion over which costs become discriminable and finally into the straight “handle,” over which objective and subjective costs match. The two remaining functions are based on hyperbolic and exponential temporal discounting, respectively. The results are best described by the sigmoidal-slope function. That this is so suggests that different principles of intertemporal choice are involved in the evaluation of time spent working for a reward or waiting for its delivery. The subjective opportunity-cost function plays a key role in the evaluation and selection of goals. An accurate description of its form and parameters is essential to successful modeling and prediction of instrumental performance and reward-related decision making.

The effects of HIV-1 regulatory TAT protein expression on brain reward function, response to psychostimulants and delay-dependent memory in mice

Depression and psychostimulant abuse are common comorbidities among humans with immunodeficiency virus (HIV) disease. The HIV regulatory protein TAT is one of multiple HIV-related proteins associated with HIV-induced neurotoxicity. TAT-induced dysfunction of dopamine and serotonin systems in corticolimbic brain areas may result in impaired reward function, thus, contributing to depressive symptoms and psychostimulant abuse. Transgenic mice with doxycycline-induced TAT protein expression in the brain (TAT+, TAT- control) show neuropathology resembling brain abnormalities in HIV+ humans. We evaluated brain reward function in response to TAT expression, nicotine and methamphetamine administration in TAT+ and TAT- mice using the intracranial self-stimulation procedure. We evaluated the brain dopamine and serotonin systems with high-performance liquid chromatography. The effects of TAT expression on delay-dependent working memory in TAT+ and TAT- mice using the operant delayed nonmatch-to-position task were also assessed. During doxycycline administration, reward thresholds were elevated by 20% in TAT+ mice compared with TAT- mice. After the termination of doxycycline treatment, thresholds of TAT+ mice remained significantly higher than those of TAT- mice and this was associated with changes in mesolimbic serotonin and dopamine levels. TAT+ mice showed a greater methamphetamine-induced threshold lowering compared with TAT- mice. TAT expression did not alter delay-dependent working memory. These results indicate that TAT expression in mice leads to reward deficits, a core symptom of depression, and a greater sensitivity to methamphetamine-induced reward enhancement. Our findings suggest that the TAT protein may contribute to increased depressive-like symptoms and continued methamphetamine use in HIV-positive individuals.

Toward a systems-oriented approach to the role of the extended amygdala in adaptive responding

Research into the structure and function of the basal forebrain macrostructure called the extended amygdala (EA) has recently seen considerable growth. This paper reviews that work, with the objectives of identifying underlying themes and developing a common goal towards which investigators of EA function might work. The paper begins with a brief review of the structure and the ontological and phylogenetic origins of the EA. It continues with a review of research into the role of the EA in both aversive and appetitive states, noting that these two seemingly disparate avenues of research converge on the concept of reinforcement - either negative or positive - of adaptive responding. These reviews lead to a proposal as to where the EA may fit in the organization of the basal forebrain, and an invitation to investigators to place their findings in a unifying conceptual framework of the EA as a collection of neural ensembles that mediate adaptive responding.

To Act or Not to Act: Endocannabinoid/Dopamine Interactions in Decision-Making

Decision-making is an ethologically adaptive construct that is impaired in multiple psychiatric disorders. Activity within the mesocorticolimbic dopamine system has been traditionally associated with decision-making. The endocannabinoid system through its actions on inhibitory and excitatory synapses modulates dopamine activity and decision-making. The aim of this brief review is to present a synopsis of available data obtained when the endocannabinoid system is manipulated and dopamine activity recorded. To this end, we review research using different behavioral paradigms to provide further insight into how this ubiquitous signaling system biases dopamine-related behaviors to regulate decision-making.

Changes in D1 but not D2 dopamine or mu-opioid receptor expression in limbic and motor structures after lateral hypothalamus electrical self-stimulation: A quantitative autoradiographic study

Intracranial self-stimulation (ICSS) of the lateral hypothalamus (LH) is involved in the activation of neuroanatomical systems that are also associated with the processing of natural and other artificial rewarding stimuli. Specific components of this behavior (hedonic impact, learning, and motor behavior) may involve changes in different neurotransmitters, such as dopamine and opioids. In this study, quantitative autoradiography was used to examine changes in mu-opioid and D1/D2-dopamine receptor expression in various anatomical regions related to the motor and mesolimbic reward systems after intracranial self-stimulation of the LH. Results of the behavioral procedure and subsequent radiochemical assays show selective changes in D1 but not D2 or mu receptors in Accumbens-Shell, Ventral Pallidum, Caudate-Putamen, and Medial Globus Pallidus. These findings are discussed in relation to the different psychobiological components of the appetitive motivational system, identifying some dissociation among them, particularly with respect to the involvement of the D1-dopamine subsystem (but not D2 or mu receptors) in goal-directed behaviors.

The neural substrates for the rewarding and dopamine-releasing effects of medial forebrain bundle stimulation have partially discrepant frequency responses

Midbrain dopamine neurons have long been implicated in the rewarding effect produced by electrical brain stimulation of the medial forebrain bundle (MFB). These neurons are excited trans-synaptically, but their precise role in intracranial self-stimulation (ICSS) has yet to be determined. This study assessed the hypothesis that midbrain dopamine neurons are in series with the directly stimulated substrate for self-stimulation of the MFB and either perform spatio-temporal integration of synaptic input from directly activated MFB fibers or relay the results of such integration to efferent stages of the reward circuitry. Psychometric current-frequency trade-off functions were derived from ICSS performance, and chemometric trade-off functions were derived from stimulation-induced dopamine transients in the nucleus accumbens (NAc) shell, measured by means of fast-scan cyclic voltammetry. Whereas the psychometric functions decline monotonically over a broad range of pulse frequencies and level off only at high frequencies, the chemometric functions obtained with the same rats and electrodes are either U-shaped or level off at lower pulse frequencies. This discrepancy was observed when the dopamine transients were recorded in either anesthetized or awake subjects. The lack of correspondence between the psychometric and chemometric functions is inconsistent with the hypothesis that dopamine neurons projecting to the NAc shell constitute an entire series stage of the neural circuit subserving self-stimulation of the MFB.

Budget Constraints Affect Male Rats' Choices between Differently Priced Commodities

Demand theory can be applied to analyse how a human or animal consumer changes her selection of commodities within a certain budget in response to changes in price of those commodities. This change in consumption assessed over a range of prices is defined as demand elasticity. Previously, income-compensated and income-uncompensated price changes have been investigated using human and animal consumers, as demand theory predicts different elasticities for both conditions. However, in these studies, demand elasticity was only evaluated over the entirety of choices made from a budget. As compensating budgets changes the number of attainable commodities relative to uncompensated conditions, and thus the number of choices, it remained unclear whether budget compensation has a trivial effect on demand elasticity by simply sampling from a different total number of choices or has a direct effect on consumers’ sequential choice structure. If the budget context independently changes choices between commodities over and above price effects, this should become apparent when demand elasticity is assessed over choice sets of any reasonable size that are matched in choice opportunities between budget conditions. To gain more detailed insight in the sequential choice dynamics underlying differences in demand elasticity between budget conditions, we trained N=8 rat consumers to spend a daily budget by making a number of nosepokes to obtain two liquid commodities under different price regimes, in sessions with and without budget compensation. We confirmed that demand elasticity for both commodities differed between compensated and uncompensated budget conditions, also when the number of choices considered was matched, and showed that these elasticity differences emerge early in the sessions. These differences in demand elasticity were driven by a higher choice rate and an increased reselection bias for the preferred commodity in compensated compared to uncompensated budget conditions, suggesting a budget context effect on relative valuation.

Psychophysical inference of frequency-following fidelity in the neural substrate for brain stimulation reward

The rewarding effect of electrical brain stimulation has been studied extensively for 60 years, yet the identity of the underlying neural circuitry remains unknown. Previous experiments have characterized the directly stimulated ("first-stage") neurons implicated in self-stimulation of the medial forebrain bundle. Their properties are consistent with those of fine, myelinated axons, at least some of which project rostro-caudally. These properties do not match those of dopaminergic neurons. The present psychophysical experiment estimates an additional first-stage characteristic: maximum firing frequency. We test a frequency-following model that maps the experimenter-set pulse frequency into the frequency of firing induced in the directly stimulated neurons. As pulse frequency is increased, firing frequency initially increases at the same rate, then becomes probabilistic, and finally levels off. The frequency-following function is based on the counter model which holds that the rewarding effect of a pulse train is determined by the aggregate spike rate triggered in first-stage neurons during a given interval. In 7 self-stimulating rats, we measured current- vs. pulse-frequency trade-off functions. The trade-off data were well described by the frequency-following model, and its upper asymptote was approached at a median value of 360 Hz (IQR = 46 Hz). This value implies a highly excitable, non-dopaminergic population of first-stage neurons. Incorporating the frequency-following function and parameters in Shizgal's 3-dimensional reward-mountain model improves its accuracy and predictive power.

Adaptations of presynaptic dopamine terminals induced by psychostimulant self-administration

A great deal of research has focused on investigating neurobiological alterations induced by chronic psychostimulant use in an effort to describe, understand, and treat the pathology of psychostimulant addiction. It has been known for several decades that dopamine neurotransmission in the nucleus accumbens is integrally involved in the selection and execution of motivated and goal-directed behaviors, and that psychostimulants act on this system to exert many of their effects. As such, a large body of work has focused on defining the consequences of psychostimulant use on dopamine signaling in the striatum as it relates to addictive behaviors. Here, we review presynaptic dopamine terminal alterations observed following self-administration of cocaine and amphetamine, as well as possible mechanisms by which these alterations occur and their impact on the progression of addiction.

Some work and some play: microscopic and macroscopic approaches to labor and leisure

Given the option, humans and other animals elect to distribute their time between work and leisure, rather than choosing all of one and none of the other. Traditional accounts of partial allocation have characterised behavior on a macroscopic timescale, reporting and studying the mean times spent in work or leisure. However, averaging over the more microscopic processes that govern choices is known to pose tricky theoretical problems, and also eschews any possibility of direct contact with the neural computations involved. We develop a microscopic framework, formalized as a semi-Markov decision process with possibly stochastic choices, in which subjects approximately maximise their expected returns by making momentary commitments to one or other activity. We show macroscopic utilities that arise from microscopic ones, and demonstrate how facets such as imperfect substitutability can arise in a more straightforward microscopic manner.

Dividing limited time between work and leisure when both are attractive is a common everyday decision. Rather than doing one exclusively, humans and other animals distribute their time between both. Traditional explanations of this phenomenon have studied the macroscopic average times spent in both. By contrast, we develop a microscopic framework in which we can model the real-time decisions that underpin these averages. In the framework, subjects' choices are approximately optimal, according to a natural, microscopic, utility function. We show that the assumptions of previous theories are not necessary for partial allocation to be optimal, and show possibilities and limits to the integration of macroscopic and microscopic views. Our approach opens new vistas onto the real-time processes underlying cost-benefit decision-making.

The effect of probability discounting on reward seeking: a three-dimensional perspective

Rats will work for electrical stimulation of the medial forebrain bundle. The rewarding effect arises from the volleys of action potentials fired by the stimulation and subsequent spatio-temporal integration of their post-synpatic impact. The proportion of time allocated to self-stimulation depends on the intensity of the rewarding effect as well as on other key determinants of decision-making, such as subjective opportunity costs and reward probability. We have proposed that a 3D model relating time allocation to the intensity and cost of reward can distinguish manipulations acting prior to the output of the spatio-temporal integrator from those acting at or beyond it. Here, we test this proposition by varying reward probability, a variable that influences the computation of payoff in the 3D model downstream from the output of the integrator. On riskless trials, reward was delivered on every occasion that the rat held down the lever for a cumulative duration called the “price,” whereas on risky trials, reward was delivered with probability 0.75 or 0.50. According to the model, the 3D structure relating time allocation to reward intensity and price is shifted leftward along the price axis by reductions in reward probability; the magnitude of the shift estimates the change in subjective probability. The predictions were borne out: reducing reward probability shifted the 3D structure systematically along the price axis while producing only small, inconsistent displacements along the pulse-frequency axis. The results confirm that the model can accurately distinguish manipulations acting at or beyond the spatio-temporal integrator and strengthen the conclusions of previous studies showing similar shifts following dopaminergic manipulations. Subjective and objective reward probabilities appeared indistinguishable over the range of 0.5 ≤ p ≤ 1.0.

Impact of personal economic environment and personality factors on individual financial decision making

This study on healthy young male students aimed to enlighten the associations between an individual’s financial decision making and surrogate makers for environmental factors covering long-term financial socialization, the current financial security/responsibility, and the personal affinity to financial affairs as represented by parental income, funding situation, and field of study. A group of 150 male young healthy students underwent two versions of the Holt and Laury (2002) lottery paradigm (matrix and random sequential version). Their financial decision was mainly driven by the factor “source of funding”: students with strict performance control (grants, scholarships) had much higher rates of relative risk aversion (RRA) than subjects with support from family (ΔRRA = 0.22; p = 0.018). Personality scores only modestly affected the outcome. In an ANOVA, however, also the intelligence quotient significantly and relevantly contributed to the explanation of variance; the effects of parental income and the personality factors “agreeableness” and “openness” showed moderate to modest – but significant – effects. These findings suggest that environmental factors more than personality factors affect risk aversion.

Optimal indolence: a normative microscopic approach to work and leisure

Dividing limited time between work and leisure when both have their attractions is a common everyday decision. We provide a normative control-theoretic treatment of this decision that bridges economic and psychological accounts. We show how our framework applies to free-operant behavioural experiments in which subjects are required to work (depressing a lever) for sufficient total time (called the price) to receive a reward. When the microscopic benefit-of-leisure increases nonlinearly with duration, the model generates behaviour that qualitatively matches various microfeatures of subjects' choices, including the distribution of leisure bout durations as a function of the pay-off. We relate our model to traditional accounts by deriving macroscopic, molar, quantities from microscopic choices.

A new view of the effect of dopamine receptor antagonism on operant performance for rewarding brain stimulation in the rat

RATIONALE

Previous studies of neuroleptic challenges to intracranial self-stimulation (ICSS) employed two-dimensional (2D) measurements (curve shifts). Results so obtained are ambiguous with regard to the stage of neural processing at which the drug produces its performance-altering effect. We substituted a three-dimensional (3D) method that measures reward-seeking as a function of both the strength and cost of reward. This method reveals whether changes in reward seeking are due to drug action prior to the output of the circuitry that performs spatiotemporal integration of the stimulation-induced neural activity.

OBJECTIVES

The aim of this study was to obtain new information about the stage of neural processing at which pimozide acts to alter pursuit of brain stimulation reward (BSR).

METHODS

Following treatment with pimozide (0.1 mg/kg) or its vehicle, the proportion of trial time allocated to working for BSR was measured as a function of pulse frequency and opportunity cost. A surface defined by Shizgal's reward-mountain model was fitted to the drug and vehicle data.

RESULTS

Pimozide lowered the cost required to decrease performance for a maximal BSR to half its maximal level but did not alter the pulse-frequency required to produce a reward of half-maximal intensity.

CONCLUSIONS

Like indirect dopamine agonists, pimozide does not alter the sensitivity of brain reward circuity but changes reward-system gain, subjective effort costs, and/or the value of activities that compete with ICSS. The 3D method is more sensitive and informative than the 2D methods employed previously.

Validation and extension of the reward-mountain model

The reward-mountain model relates the vigor of reward seeking to the strength and cost of reward. Application of this model provides information about the stage of processing at which manipulations such as drug administration, lesions, deprivation states, and optogenetic interventions act to alter reward seeking. The model has been updated by incorporation of new information about frequency following in the directly stimulated neurons responsible for brain stimulation reward and about the function that maps objective opportunity costs into subjective ones. The behavioral methods for applying the model have been updated and improved as well. To assess the impact of these changes, two related predictions of the model that were supported by earlier work have been retested: (1) altering the duration of rewarding brain stimulation should change the pulse frequency required to produce a reward of half-maximal intensity, and (2) this manipulation should not change the opportunity cost at which half-maximal performance is directed at earning a maximally intense reward. Prediction 1 was supported in all six subjects, but prediction 2 was supported in only three. The latter finding is interpreted to reflect recruitment, at some stimulation sites, of a heterogeneous reward substrate comprising dual, parallel circuits that integrate the stimulation-induced neural signals.