A new view of the effect of dopamine receptor antagonism on operant performance for rewarding brain stimulation in the rat

The priming effect of rewarding brain stimulation in rats depends on both the cost and strength of reward but survives blockade of D2-like dopamine receptors

Receipt of an intense reward boosts motivation to work for more of that reward. This phenomenon is called the priming effect of rewards. Using a novel measurement method, we show that the priming effect of rewarding electrical brain stimulation depends on the cost, as well as on the strength, of the anticipated reward. Previous research on the priming effect of electrical brain stimulation utilized a runway paradigm in which running speed serves as the measure of motivation. In the present study, the measure of motivation was the vigour with which rats executed a two-lever response chain, in a standard operant-conditioning chamber, to earn rewarding electrical stimulation of the lateral hypothalamus. In a second experiment, we introduced a modification that entails self-administered priming stimulation and alternating blocks of primed and unprimed trials. Reliable, consistent priming effects of substantial magnitude were obtained in the modified paradigm, which is closely analogous to the runway paradigm. In a third experiment, the modified paradigm served to assess the dependence of the priming effect on dopamine D2-like receptors. The priming effect proved resilient to the effect of eticlopride, a selective D2-like receptor antagonist. These results are discussed within the framework of a new model of brain reward circuitry in which non-dopaminergic medial forebrain bundle fibers and dopamine axons provide parallel inputs to the final common paths for reward and incentive motivation.

Dopamine and Beyond: Implications of Psychophysical Studies of Intracranial Self-Stimulation for the Treatment of Depression

Major depressive disorder is a leading cause of disability and suicide worldwide. Consecutive rounds of conventional interventions are ineffective in a significant sub-group of patients whose disorder is classified as treatment-resistant depression. Significant progress in managing this severe form of depression has been achieved through the use of deep brain stimulation of the medial forebrain bundle (MFB). The beneficial effect of such stimulation appears strong, safe, and enduring. The proposed neural substrate for this promising clinical finding includes midbrain dopamine neurons and a subset of their cortical afferents. Here, we aim to broaden the discussion of the candidate circuitry by exploring potential implications of a new “convergence” model of brain reward circuitry in rodents. We chart the evolution of the new model from its predecessors, which held that midbrain dopamine neurons constituted an obligatory stage of the final common path for reward seeking. In contrast, the new model includes a directly activated, non-dopaminergic pathway whose output ultimately converges with that of the dopaminergic neurons. On the basis of the new model and the relative ineffectiveness of dopamine agonists in the treatment of depression, we ask whether non-dopaminergic circuitry may contribute to the clinical efficacy of deep brain stimulation of the MFB.

The Convergence Model of Brain Reward Circuitry: Implications for Relief of Treatment-Resistant Depression by Deep-Brain Stimulation of the Medial Forebrain Bundle

Deep-brain stimulation of the medial forebrain bundle (MFB) can provide effective, enduring relief of treatment-resistant depression. Panksepp provided an explanatory framework: the MFB constitutes the core of the neural circuitry subserving the anticipation and pursuit of rewards: the “SEEKING” system. On that view, the SEEKING system is hypoactive in depressed individuals; background electrical stimulation of the MFB alleviates symptoms by normalizing activity. Panksepp attributed intracranial self-stimulation to excitation of the SEEKING system in which the ascending projections of midbrain dopamine neurons are an essential component. In parallel with Panksepp’s qualitative work, intracranial self-stimulation has long been studied quantitatively by psychophysical means. That work argues that the predominant directly stimulated substrate for MFB self-stimulation are myelinated, non-dopaminergic fibers, more readily excited by brief electrical current pulses than the thin, unmyelinated axons of the midbrain dopamine neurons. The series-circuit hypothesis reconciles this view with the evidence implicating dopamine in MFB self-stimulation as follows: direct activation of myelinated MFB fibers is rewarding due to their trans-synaptic activation of midbrain dopamine neurons. A recent study in which rats worked for optogenetic stimulation of midbrain dopamine neurons challenges the series-circuit hypothesis and provides a new model of intracranial self-stimulation in which the myelinated non-dopaminergic neurons and the midbrain dopamine projections access the behavioral final common path for reward seeking via separate, converging routes. We explore the potential implications of this convergence model for the interpretation of the antidepressant effect of MFB stimulation. We also discuss the consistent finding that psychomotor stimulants, which boost dopaminergic neurotransmission, fail to provide a monotherapy for depression. We propose that non-dopaminergic MFB components may contribute to the therapeutic effect in parallel to, in synergy with, or even instead of, a dopaminergic component.

On the Similarity Between the Reinforcing and the Discriminative Properties of Intracranial Self-Stimulation

Rats work very hard for intracranial self-stimulation (ICSS) and tradeoff effort or time allocation for intensity and frequency parameters producing a sigmoidal function of the subjective reward magnitude of ICSS. Previous studies using electrical intracranial stimuli (ICS) as a discriminative cue focused on estimating detection thresholds or on the discrimination between intensities. To our knowledge, there is no direct comparison of the reinforcer tradeoff functions with the discriminative functions. Rats were trained to press and hold the lever for ICSS using the maximum reinforcing intensity below motor alterations or avoidance behavior. First, rats were trained to hold the lever for 1 s; after stability, they undergo trials where intensity or frequency was decreased on 0.1 log step. Thereafter, they undergo further training with a hold of 2 and later of 4 s to determine tradeoff with intensity or frequency. The same rats were trained on a discrimination task where the previously used ICSS signaled a lever where a 1 s hold response was followed by a reinforcing ICSS; on randomly alternating trials, a −0.6 log ICS signaled an alternate lever where a similar hold response led to a reinforcer. After mastering discrimination, generalization tests were carried out with varying intensity or frequency. Rats completed training with 2 and later 4 s hold response. After the completion of each task, the rats had different doses of a pimozide challenge while their intensity and hold-down requirement were varied. With regards to the rats’ tradeoff response time allocation as a function of intensity or frequency, sigmoid functions were displaced to the right when long responses were required. Rats that learned the discrimination task attained a discrimination index of 90–98%. Discrimination accuracy decreased slightly with the increase of hold requirement, but generalization gradients were not displaced to the right as a function of the response requirement. Pimozide induced a dose-dependent displacement of the time-allocation gradients, but it did not affect the generalization gradients. It is concluded that rats integrate response requirements as part of the reinforcement tradeoff function, but the response cost is not integrated into the discriminative function of ICSS.

Dopamine neurons do not constitute an obligatory stage in the final common path for the evaluation and pursuit of brain stimulation reward

The neurobiological study of reward was launched by the discovery of intracranial self-stimulation (ICSS). Subsequent investigation of this phenomenon provided the initial link between reward-seeking behavior and dopaminergic neurotransmission. We re-evaluated this relationship by psychophysical, pharmacological, optogenetic, and computational means. In rats working for direct, optical activation of midbrain dopamine neurons, we varied the strength and opportunity cost of the stimulation and measured time allocation, the proportion of trial time devoted to reward pursuit. We found that the dependence of time allocation on the strength and cost of stimulation was similar formally to that observed when electrical stimulation of the medial forebrain bundle served as the reward. When the stimulation is strong and cheap, the rats devote almost all their time to reward pursuit; time allocation falls off as stimulation strength is decreased and/or its opportunity cost is increased. A 3D plot of time allocation versus stimulation strength and cost produces a surface resembling the corner of a plateau (the “reward mountain”). We show that dopamine-transporter blockade shifts the mountain along both the strength and cost axes in rats working for optical activation of midbrain dopamine neurons. In contrast, the same drug shifted the mountain uniquely along the opportunity-cost axis when rats worked for electrical MFB stimulation in a prior study. Dopamine neurons are an obligatory stage in the dominant model of ICSS, which positions them at a key nexus in the final common path for reward seeking. This model fails to provide a cogent account for the differential effect of dopamine transporter blockade on the reward mountain. Instead, we propose that midbrain dopamine neurons and neurons with non-dopaminergic, MFB axons constitute parallel limbs of brain-reward circuitry that ultimately converge on the final-common path for the evaluation and pursuit of rewards.

The priming effect of food persists following blockade of dopamine receptors

The priming effect of rewards is a boost in the vigor of reward seeking resulting from the previous receipt of a reward. Extensive work has been carried out on the priming effect of electrical brain stimulation, but much less research exists on the priming effect of natural rewards, such as food. While both reinforcement and motivation are linked with dopamine transmission in the brain, the priming effect of rewards does not appear to be dopamine-dependent. In the present study, an operant method was developed to measure the priming effect of food and then applied to investigate whether it is affected by dopamine receptor antagonism. Long-Evans rats were administered saline or one of the three doses (0.01, 0.05, 0.075 mg/kg) of the dopamine D1 receptor family antagonist, SCH23390, or the dopamine D2 receptor family antagonist, eticlopride. Although dopamine receptor antagonism affected pursuit of food, it did not eliminate the priming effect. These data suggest that despite the involvement of dopamine transmission in reinforcement and motivation, the priming effect of food does not depend on dopamine transmission.

Valuation of opportunity costs by rats working for rewarding electrical brain stimulation

Pursuit of one goal typically precludes simultaneous pursuit of another. Thus, each exclusive activity entails an “opportunity cost:” the forgone benefits from the next-best activity eschewed. The present experiment estimates, in laboratory rats, the function that maps objective opportunity costs into subjective ones. In an operant chamber, rewarding electrical brain stimulation was delivered when the cumulative time a lever had been depressed reached a criterion duration. The value of the activities forgone during this duration is the opportunity cost of the electrical reward. We determined which of four functions best describes how objective opportunity costs, expressed as the required duration of lever depression, are translated into their subjective equivalents. The simplest account is the identity function, which equates subjective and objective opportunity costs. A variant of this function called the “sigmoidal-slope function,” converges on the identity function at longer durations but deviates from it at shorter durations. The sigmoidal-slope function has the form of a hockey stick. The flat “blade” denotes a range over which opportunity costs are subjectively equivalent; these durations are too short to allow substitution of more beneficial activities. The blade extends into an upward-curving portion over which costs become discriminable and finally into the straight “handle,” over which objective and subjective costs match. The two remaining functions are based on hyperbolic and exponential temporal discounting, respectively. The results are best described by the sigmoidal-slope function. That this is so suggests that different principles of intertemporal choice are involved in the evaluation of time spent working for a reward or waiting for its delivery. The subjective opportunity-cost function plays a key role in the evaluation and selection of goals. An accurate description of its form and parameters is essential to successful modeling and prediction of instrumental performance and reward-related decision making.

Toward a systems-oriented approach to the role of the extended amygdala in adaptive responding

Research into the structure and function of the basal forebrain macrostructure called the extended amygdala (EA) has recently seen considerable growth. This paper reviews that work, with the objectives of identifying underlying themes and developing a common goal towards which investigators of EA function might work. The paper begins with a brief review of the structure and the ontological and phylogenetic origins of the EA. It continues with a review of research into the role of the EA in both aversive and appetitive states, noting that these two seemingly disparate avenues of research converge on the concept of reinforcement - either negative or positive - of adaptive responding. These reviews lead to a proposal as to where the EA may fit in the organization of the basal forebrain, and an invitation to investigators to place their findings in a unifying conceptual framework of the EA as a collection of neural ensembles that mediate adaptive responding.

To Act or Not to Act: Endocannabinoid/Dopamine Interactions in Decision-Making

Decision-making is an ethologically adaptive construct that is impaired in multiple psychiatric disorders. Activity within the mesocorticolimbic dopamine system has been traditionally associated with decision-making. The endocannabinoid system through its actions on inhibitory and excitatory synapses modulates dopamine activity and decision-making. The aim of this brief review is to present a synopsis of available data obtained when the endocannabinoid system is manipulated and dopamine activity recorded. To this end, we review research using different behavioral paradigms to provide further insight into how this ubiquitous signaling system biases dopamine-related behaviors to regulate decision-making.

The neural substrates for the rewarding and dopamine-releasing effects of medial forebrain bundle stimulation have partially discrepant frequency responses

Midbrain dopamine neurons have long been implicated in the rewarding effect produced by electrical brain stimulation of the medial forebrain bundle (MFB). These neurons are excited trans-synaptically, but their precise role in intracranial self-stimulation (ICSS) has yet to be determined. This study assessed the hypothesis that midbrain dopamine neurons are in series with the directly stimulated substrate for self-stimulation of the MFB and either perform spatio-temporal integration of synaptic input from directly activated MFB fibers or relay the results of such integration to efferent stages of the reward circuitry. Psychometric current-frequency trade-off functions were derived from ICSS performance, and chemometric trade-off functions were derived from stimulation-induced dopamine transients in the nucleus accumbens (NAc) shell, measured by means of fast-scan cyclic voltammetry. Whereas the psychometric functions decline monotonically over a broad range of pulse frequencies and level off only at high frequencies, the chemometric functions obtained with the same rats and electrodes are either U-shaped or level off at lower pulse frequencies. This discrepancy was observed when the dopamine transients were recorded in either anesthetized or awake subjects. The lack of correspondence between the psychometric and chemometric functions is inconsistent with the hypothesis that dopamine neurons projecting to the NAc shell constitute an entire series stage of the neural circuit subserving self-stimulation of the MFB.

Budget Constraints Affect Male Rats' Choices between Differently Priced Commodities

Demand theory can be applied to analyse how a human or animal consumer changes her selection of commodities within a certain budget in response to changes in price of those commodities. This change in consumption assessed over a range of prices is defined as demand elasticity. Previously, income-compensated and income-uncompensated price changes have been investigated using human and animal consumers, as demand theory predicts different elasticities for both conditions. However, in these studies, demand elasticity was only evaluated over the entirety of choices made from a budget. As compensating budgets changes the number of attainable commodities relative to uncompensated conditions, and thus the number of choices, it remained unclear whether budget compensation has a trivial effect on demand elasticity by simply sampling from a different total number of choices or has a direct effect on consumers’ sequential choice structure. If the budget context independently changes choices between commodities over and above price effects, this should become apparent when demand elasticity is assessed over choice sets of any reasonable size that are matched in choice opportunities between budget conditions. To gain more detailed insight in the sequential choice dynamics underlying differences in demand elasticity between budget conditions, we trained N=8 rat consumers to spend a daily budget by making a number of nosepokes to obtain two liquid commodities under different price regimes, in sessions with and without budget compensation. We confirmed that demand elasticity for both commodities differed between compensated and uncompensated budget conditions, also when the number of choices considered was matched, and showed that these elasticity differences emerge early in the sessions. These differences in demand elasticity were driven by a higher choice rate and an increased reselection bias for the preferred commodity in compensated compared to uncompensated budget conditions, suggesting a budget context effect on relative valuation.

Psychophysical inference of frequency-following fidelity in the neural substrate for brain stimulation reward

The rewarding effect of electrical brain stimulation has been studied extensively for 60 years, yet the identity of the underlying neural circuitry remains unknown. Previous experiments have characterized the directly stimulated ("first-stage") neurons implicated in self-stimulation of the medial forebrain bundle. Their properties are consistent with those of fine, myelinated axons, at least some of which project rostro-caudally. These properties do not match those of dopaminergic neurons. The present psychophysical experiment estimates an additional first-stage characteristic: maximum firing frequency. We test a frequency-following model that maps the experimenter-set pulse frequency into the frequency of firing induced in the directly stimulated neurons. As pulse frequency is increased, firing frequency initially increases at the same rate, then becomes probabilistic, and finally levels off. The frequency-following function is based on the counter model which holds that the rewarding effect of a pulse train is determined by the aggregate spike rate triggered in first-stage neurons during a given interval. In 7 self-stimulating rats, we measured current- vs. pulse-frequency trade-off functions. The trade-off data were well described by the frequency-following model, and its upper asymptote was approached at a median value of 360 Hz (IQR = 46 Hz). This value implies a highly excitable, non-dopaminergic population of first-stage neurons. Incorporating the frequency-following function and parameters in Shizgal's 3-dimensional reward-mountain model improves its accuracy and predictive power.

A generalized matching law analysis of cocaine vs. food choice in rhesus monkeys: effects of candidate 'agonist-based' medications on sensitivity to reinforcement

BACKGROUND

We have previously demonstrated reductions in cocaine choice produced by either continuous 14-day phendimetrazine and d-amphetamine treatment or removing cocaine availability under a cocaine vs. food choice procedure in rhesus monkeys. The aim of the present investigation was to apply the concatenated generalized matching law (GML) to cocaine vs. food choice dose-effect functions incorporating sensitivity to both the relative magnitude and price of each reinforcer. Our goal was to determine potential behavioral mechanisms underlying pharmacological treatment efficacy to decrease cocaine choice.

METHODS

A multi-model comparison approach was used to characterize dose- and time-course effects of both pharmacological and environmental manipulations on sensitivity to reinforcement.

RESULTS

GML models provided an excellent fit of the cocaine choice dose-effect functions in individual monkeys. Reductions in cocaine choice by both pharmacological and environmental manipulations were principally produced by systematic decreases in sensitivity to reinforcer price and non-systematic changes in sensitivity to reinforcer magnitude.

CONCLUSIONS

The modeling approach used provides a theoretical link between the experimental analysis of choice and pharmacological treatments being evaluated as candidate 'agonist-based' medications for cocaine addiction. The analysis suggests that monoamine releaser treatment efficacy to decrease cocaine choice was mediated by selectively increasing the relative price of cocaine. Overall, the net behavioral effect of these pharmacological treatments was to increase substitutability of food pellets, a nondrug reinforcer, for cocaine.

The effect of probability discounting on reward seeking: a three-dimensional perspective

Rats will work for electrical stimulation of the medial forebrain bundle. The rewarding effect arises from the volleys of action potentials fired by the stimulation and subsequent spatio-temporal integration of their post-synpatic impact. The proportion of time allocated to self-stimulation depends on the intensity of the rewarding effect as well as on other key determinants of decision-making, such as subjective opportunity costs and reward probability. We have proposed that a 3D model relating time allocation to the intensity and cost of reward can distinguish manipulations acting prior to the output of the spatio-temporal integrator from those acting at or beyond it. Here, we test this proposition by varying reward probability, a variable that influences the computation of payoff in the 3D model downstream from the output of the integrator. On riskless trials, reward was delivered on every occasion that the rat held down the lever for a cumulative duration called the “price,” whereas on risky trials, reward was delivered with probability 0.75 or 0.50. According to the model, the 3D structure relating time allocation to reward intensity and price is shifted leftward along the price axis by reductions in reward probability; the magnitude of the shift estimates the change in subjective probability. The predictions were borne out: reducing reward probability shifted the 3D structure systematically along the price axis while producing only small, inconsistent displacements along the pulse-frequency axis. The results confirm that the model can accurately distinguish manipulations acting at or beyond the spatio-temporal integrator and strengthen the conclusions of previous studies showing similar shifts following dopaminergic manipulations. Subjective and objective reward probabilities appeared indistinguishable over the range of 0.5 ≤ p ≤ 1.0.

Intracranial self-stimulation to evaluate abuse potential of drugs

Intracranial self-stimulation (ICSS) is a behavioral procedure in which operant responding is maintained by pulses of electrical brain stimulation. In research to study abuse-related drug effects, ICSS relies on electrode placements that target the medial forebrain bundle at the level of the lateral hypothalamus, and experimental sessions manipulate frequency or amplitude of stimulation to engender a wide range of baseline response rates or response probabilities. Under these conditions, drug-induced increases in low rates/probabilities of responding maintained by low frequencies/amplitudes of stimulation are interpreted as an abuse-related effect. Conversely, drug-induced decreases in high rates/probabilities of responding maintained by high frequencies/amplitudes of stimulation can be interpreted as an abuse-limiting effect. Overall abuse potential can be inferred from the relative expression of abuse-related and abuse-limiting effects. The sensitivity and selectivity of ICSS to detect abuse potential of many classes of abused drugs is similar to the sensitivity and selectivity of drug self-administration procedures. Moreover, similar to progressive-ratio drug self-administration procedures, ICSS data can be used to rank the relative abuse potential of different drugs. Strengths of ICSS in comparison with drug self-administration include 1) potential for simultaneous evaluation of both abuse-related and abuse-limiting effects, 2) flexibility for use with various routes of drug administration or drug vehicles, 3) utility for studies in drug-naive subjects as well as in subjects with controlled levels of prior drug exposure, and 4) utility for studies of drug time course. Taken together, these considerations suggest that ICSS can make significant contributions to the practice of abuse potential testing.