Pharmacological characterization of performance on a concurrent lever pressing/feeding choice procedure: effects of dopamine antagonist, cholinomimetic, sedative and stimulant drugs

Nucleus accumbens dopamine release reflects the selective nature of pair bonds

In monogamous species, prosocial behaviors directed toward partners are dramatically different from those directed toward unknown individuals and potential threats. Dopamine release in the nucleus accumbens has a well-established role in social reward and motivation, but how this mechanism may be engaged to drive the highly divergent social behaviors directed at a partner or unfamiliar conspecific remains unknown. Using monogamous prairie voles, we first employed receptor pharmacology in partner preference and social operant tasks to show that dopamine is critical for the appetitive drive for social interaction but not for low-effort, unconditioned consummatory behaviors. We then leveraged the subsecond temporal resolution of the fluorescent biosensor, GRABDA, to ask whether differential dopamine release might distinguish between partner and novel social access and interaction. We found that partner seeking, anticipation, and interaction resulted in more accumbal dopamine release than the same events directed toward a novel vole. Further, partner-associated dopamine release decreased after prolonged partner separation. Our results are consistent with a model in which dopamine signaling plays a prominent role in the appetitive aspects of social interactions. Within this framework, differences in partner- and novel-associated dopamine release reflect the selective nature of pair bonds and may drive the partner- and novel-directed social behaviors that reinforce and cement bonds over time. This provides a potential mechanism by which highly conserved reward systems can enable selective, species-appropriate social behaviors.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Disentangling the role of NAc D1 and D2 cells in hedonic eating

Overeating is driven by both the hedonic component ('liking') of food, and the motivation ('wanting') to eat it. The nucleus accumbens (NAc) is a key brain center implicated in these processes, but how distinct NAc cell populations encode 'liking' and 'wanting' to shape overconsumption remains unclear. Here, we probed the roles of NAc D1 and D2 cells in these processes using cell-specific recording and optogenetic manipulation in diverse behavioral paradigms that disentangle reward traits of 'liking' and 'wanting' related to food choice and overeating in healthy mice. Medial NAc shell D2 cells encoded experience-dependent development of 'liking', while D1 cells encoded innate 'liking' during the first food taste. Optogenetic control confirmed causal links of D1 and D2 cells to these aspects of 'liking'. In relation to 'wanting', D1 and D2 cells encoded and promoted distinct aspects of food approach: D1 cells interpreted food cues while D2 cells also sustained food-visit-length that facilitates consumption. Finally, at the level of food choice, D1, but not D2, cell activity was sufficient to switch food preference, programming subsequent long-lasting overconsumption. By revealing complementary roles of D1 and D2 cells in consumption, these findings assign neural bases to 'liking' and 'wanting' in a unifying framework of D1 and D2 cell activity.

Prenatal dexamethasone exposure alters effort decision making and triggers nucleus accumbens and anterior cingulate cortex functional changes in male rats

Daily, individuals select actions based on cost-benefit to allocate resources into goal-directed actions. Different brain regions coordinate this complex decision, including the nucleus accumbens (NAc), anterior cingulate cortex (ACC), and ventral tegmental area (VTA). In utero exposure to synthetic glucocorticoids (iuGC), such as dexamethasone, triggers prominent motivation deficits but the impact of this exposure in the ACC-NAc and/or ACC-VTA circuits is unknown. Here, we show that iuGC exposure causes decreased motivation for natural rewards (food) and impaired effort-based decision-making. Importantly, reduced neuronal activation (number of c-fos⁺ neurons) was observed in the NAc core and ACC of iuGC rats in comparison to CTR rats after performing the effort-based decision-making task. In addition, iuGC treatment led to increased NAc and ACC basal neuronal activity. Electrophysiological recordings during optogenetic modulation of ACC terminals in the NAc revealed that the ACC-NAc circuit is dysfunctional in iuGC animals. These data suggest that iuGC animals present motivational and effort-based decision-making deficits that can be associated with the observed ACC-NAc dysfunction.

Systemic D1-R and D2-R antagonists in Non-Human Primates Differentially Impact Learning and Memory While Impairing Motivation and Motor Performance

Dopamine (DA) modulates cognition in part via differential activation of D1 and D2 receptors within the striatum and prefrontal cortex, yet evidence for cognitive impairments stemming from DA blockade or deficiency is inconsistent. Given the predominance of D1 over D2 receptors (R) in the prefrontal cortex of primates, D1-R blockade should more strongly influence frontal executive function (including working memory), while D2-R blockade should impair processes more strongly associated with the dorsal striatum (including cognitive flexibility, and learning). To test how systemic DA blockade disrupts cognition, we administered D1-R and D2-R like antagonists to healthy monkeys while they performed a series of cognitive tasks. Two selective DA receptor antagonist drugs (SCH-23390 hydrochloride: D1/D5-R antagonist; or Eticlopride hydrochloride: D2/D3-R antagonist) or placebo (0.9% saline) were systemically administered. Four tasks were used: (1) 'visually guided reaching', to test response time and accuracy, (2) 'reversal learning', to test association learning and attention, (3) 'self-ordered sequential search' to test spatial working memory, and (4) 'delayed match to sample' to test object working memory. Increased reach response times and decreased motivation to work for liquid reward was observed with both the D1/D5-R and D2/D3-R antagonists at the maximum dosages that still enabled task performance. The D2/D3-R antagonist impaired performance in the reversal learning task, while object and spatial working memory performance was not consistently affected in the tested tasks for either drug. These results are consistent with the theory that systemic D2/D3-R antagonists preferentially influence striatum processes (cognitive flexibility) while systemic D1/D5-R administration is less detrimental to frontal executive function.

The effects of early life stress on impulsivity

Elevated impulsivity is a symptom shared by various psychiatric disorders such as substance use disorder, bipolar disorder, and attention-deficit/hyperactivity disorder. However, impulsivity is not a unitary construct and impulsive behaviors fall into two subcategories: impulsive action and impulsive choice. Impulsive choice refers to the tendency to prefer immediate, small rewards over delayed, large rewards, whereas impulsive action involves difficulty inhibiting rash, premature, or mistimed behaviors. These behaviors are mediated by the mesocorticolimbic dopamine (DA) system, which consists of projections from the ventral tegmental area to the nucleus accumbens and prefrontal cortex. Early life stress (ELS) alters both impulsive choice and impulsive action in rodents. ELS also changes DA receptor expression, transmission, and activity within the mesocorticolimbic system. This review integrates the dopamine, impulsivity, and ELS literature to provide evidence that ELS alters impulsivity via inducing changes in the mesocorticolimbic DA system. Understanding how ELS affects brain circuits associated with impulsivity can help advance treatments aimed towards reducing impulsivity symptoms in a variety of psychiatric disorders.

Effects of the dopamine depleting agent tetrabenazine on detailed temporal parameters of effort-related choice responding

The dopamine-depleting agent tetrabenazine alters effort-based choice, suppressing food-reinforced behaviors with high response requirements, while increasing selection of low-cost options. In the present experiments, rats were tested on a concurrent fixed ratio 5/chow feeding choice task, in which high-carbohydrate Bio-serv pellets reinforced lever pressing and lab chow was concurrently available. Detailed timing of lever pressing was monitored with an event recording system, and the temporal characteristics of operant behavior seen after 1.0 mg/kg tetrabenazine or vehicle injections were analyzed. Tetrabenazine shifted choice, decreasing lever pressing but increasing chow intake. There was a small effect on the interresponse-time distribution within ratios, but marked increases in the total duration of pauses in responding. The postreinforcement-pause (PRP) distribution was bimodal, but tetrabenazine did not increase the duration of PRPs. Tetrabenazine increased time feeding and duration and number of feeding bouts, but did not affect feeding rate or total time spent lever pressing for pellets and consuming chow. Thus, TBZ appears to predominantly affect the relative allocation of lever pressing versus chow, with little alteration in consummatory motor acts involved in chow intake. Tetrabenazine is used to model motivational symptoms in psychopathology, and these effects in rats could have implications for psychiatric research.

Event-based control of autonomic and emotional states by the anterior cingulate cortex

Despite being an intensive area of research, the function of the anterior cingulate cortex (ACC) remains somewhat of a mystery. Human imaging studies implicate the ACC in various cognitive functions, yet surgical ACC lesions used to treat emotional disorders have minimal lasting effects on cognition. An alternative view is that ACC regulates autonomic states, consistent with its interconnectivity with autonomic control regions and that stimulation evokes changes in autonomic/emotional states. At the cellular level, ACC neurons are highly multi-modal and promiscuous, and can represent a staggering array of task events. These neurons nevertheless combine to produce highly event-specific ensemble patterns that likely alter activity in downstream regions controlling emotional and autonomic tone. Since neuromodulators regulate the strength of the ensemble activity patterns, they would regulate the impact these patterns have on downstream targets. Through these mechanisms, the ACC may determine how strongly to react to the very events its ensembles represent. Pathologies arise when specific event-related representations gain excessive control over autonomic/emotional states.

Dopaminergic modulation of reward discounting in healthy rats: a systematic review and meta-analysis

RATIONALE

Although numerous studies have suggested that pharmacological alteration of the dopamine (DA) system modulates reward discounting, these studies have produced inconsistent findings.

OBJECTIVES

Here, we conducted a systematic review and pre-registered meta-analysis to evaluate DA drug-mediated effects on reward discounting of time, probability, and effort costs in studies of healthy rats. This produced a total of 1343 articles to screen for inclusion/exclusion. From the literature, we identified 117 effects from approximately 1549 individual rats.

METHODS

Using random effects with maximum-likelihood estimation, we meta-analyzed placebo-controlled drug effects for (1) DA D1-like receptor agonists and (2) antagonists, (3) D2-like agonists and (4) antagonists, and (5) DA transporter-modulating drugs.

RESULTS

Meta-analytic effects showed that DAT-modulating drugs decreased reward discounting. While D1-like and D2-like antagonists both increased discounting, agonist drugs for those receptors had no significant effect on discounting behavior. A number of these effects appear contingent on study design features like cost type, rat strain, and microinfusion location.

CONCLUSIONS

These findings suggest a nuanced relationship between DA and discounting behavior and urge caution when drawing generalizations about the effects of pharmacologically manipulating dopamine on reward-based decision-making.

'Liking' and 'wanting' in eating and food reward: Brain mechanisms and clinical implications

It is becoming clearer how neurobiological mechanisms generate 'liking' and 'wanting' components of food reward. Mesocorticolimbic mechanisms that enhance 'liking' include brain hedonic hotspots, which are specialized subregions that are uniquely able to causally amplify the hedonic impact of palatable tastes. Hedonic hotspots are found in nucleus accumbens medial shell, ventral pallidum, orbitofrontal cortex, insula cortex, and brainstem. In turn, a much larger mesocorticolimbic circuitry generates 'wanting' or incentive motivation to obtain and consume food rewards. Hedonic and motivational circuitry interact together and with hypothalamic homeostatic circuitry, allowing relevant physiological hunger and satiety states to modulate 'liking' and 'wanting' for food rewards. In some conditions such as drug addiction, 'wanting' is known to dramatically detach from 'liking' for the same reward, and this may also occur in over-eating disorders. Via incentive sensitization, 'wanting' selectively becomes higher, especially when triggered by reward cues when encountered in vulnerable states of stress, etc. Emerging evidence suggests that some cases of obesity and binge eating disorders may reflect an incentive-sensitization brain signature of cue hyper-reactivity, causing excessive 'wanting' to eat. Future findings on the neurobiological bases of 'liking' and 'wanting' can continue to improve understanding of both normal food reward and causes of clinical eating disorders.

Inhibition of Catechol-O-methyltransferase Does Not Alter Effort-Related Choice Behavior in a Fixed Ratio/Concurrent Chow Task in Male Mice

Effort-related choice (ERC) tasks allow animals to choose between high-value reinforcers that require high effort to obtain and low-value/low-effort reinforcers. Dopaminergic neuromodulation regulates ERC behavior. The enzyme catechol-O-methyltransferase (COMT) metabolizes synaptically-released dopamine. COMT is the predominant regulator of dopamine turnover in regions of the brain with low levels of dopamine transporters (DATs), including the prefrontal cortex (PFC). Here, we evaluated the effects of the COMT inhibitor tolcapone on ERC performance in a touchscreen-based fixed-ratio/concurrent chow task in male mice. In this task, mice were given the choice between engaging in a fixed number of instrumental responses to acquire a strawberry milk reward and consuming standard lab chow concurrently available on the chamber floor. We found no significant effects of tolcapone treatment on either strawberry milk earned or chow consumed compared to vehicle treatment. In contrast, we found that haloperidol decreased instrumental responding for strawberry milk and increased chow consumption as seen in previously published studies. These data suggest that COMT inhibition does not significantly affect effort-related decision making in a fixed-ratio/concurrent chow task in male mice.

Investigating serotonergic contributions to cognitive effort allocation, attention, and impulsive action in female rats

BACKGROUND

Individuals must frequently evaluate whether it is worth allocating cognitive effort for desired outcomes. Motivational deficits are a common feature of psychiatric illness such as major depression. Selective serotonin reuptake inhibitors are commonly used to treat this disorder, yet some data suggest these compounds are ineffective at treating amotivation, and may even exacerbate it.

AIMS

Here we used the rodent Cognitive Effort Task (rCET) to assess serotonergic (5-hydroxytryptamine, 5-HT) contributions to decision-making with cognitive effort costs.

METHODS

The rCET is a modified version of the 5-choice serial reaction time task, a well-validated test of visuospatial attention and impulse control. At the start of each rCET trial, rats chose one of two levers, which set the difficulty of an attentional challenge, namely the localization of a visual stimulus illuminated for 0.2 or 1 s on hard versus easy trials. Successful completion of hard trials was rewarded with double the sugar pellets. Twenty-four female Long-Evans rats were trained on the rCET and systemically administered the 5-HT_1A agonist 8-OH-DPAT, the 5-HT_2A antagonist M100907, the 5-HT_2C agonist Ro-60-0175, as well as the 5-HT_2C antagonist SB 242, 084.

RESULTS

5-HT_2A antagonism dose-dependently reduced premature responding, while 5-HT_2C antagonism had the opposite effect. 8-OH-DPAT impaired accuracy of target detection at higher doses, while Ro-60-0175 dose-dependently improved accuracy on difficult trials. However, none of the drugs affected the rats' choice of the harder option.

CONCLUSION

When considered with existing work evaluating decision-making with physical effort costs, it appears that serotonergic signalling plays a minor role in guiding effort allocation.

Preference for Exercise vs. More Sedentary Reinforcers: Validation of an Animal Model of Tetrabenazine-Induced Anergia

Physical activities can have intrinsic motivational or reinforcing properties. The choice to engage in voluntary physical activity is undertaken in relation to the selection of other alternatives, such as sedentary behaviors, drugs, or food intake. The mesolimbic dopamine (DA) system plays a critical role in behavioral activation or exertion of effort, and DA antagonism or depletion induces anergia in effort-based decision-making tasks. However, little is known about the neural mechanisms underlying the decision-making processes that establish preferences for sedentary vs. activity-based reinforcers. In the present work with male CD1 mice, we evaluated the effect of tetrabenazine (TBZ), a DA-depleting agent, on a three-choice T-maze task developed to assess preference between reinforcers with different behavioral activation requirements and sensory properties [i.e., a running wheel (RW) vs. sweet pellets or a neutral nonsocial odor]. We also studied the effects of TBZ on the forced swim test (FST), which measures climbing and swimming in a stressful setting, and on anxiety tests [dark-light (DL) box and elevated plus maze (EPM)]. In the three-choice task, TBZ reduced time running in the wheel but increased time spent consuming sucrose, thus indicating reduced activation but relatively intact sucrose reinforcement. The effect of TBZ was not mimicked by motivational manipulations that change the value of the reinforcers, such as making the RW aversive or harder to move, food-restricting the animals, inducing a binge-like eating pattern, or introducing social odors. In the FST, TBZ decreased time climbing (most active behavior) and increased immobility but did not affect anxiety in the DL or EPM. These results indicate that the three-choice T-maze task could be useful for assessing DA modulation of preferences for exercise based on activation and effort requirements, differentiating those effects from changes in preference produced by altering physical requirements, food restriction state, and stress during testing.

Alterations in effort-related decision-making induced by stimulation of dopamine D1, D2, D3, and corticotropin-releasing factor receptors in nucleus accumbens subregions

RATIONALE

Nucleus accumbens (NAc) dopamine (DA) plays an integral role in overcoming effort costs, as blockade of D₁ and D₂ receptors reduces the choice of larger, more-costly rewards. Similarly, the stress neuropeptide corticotropin-releasing factor (CRF) modulates DA transmission and mediates stress-induced alterations in effort-related choice.

OBJECTIVES

The current study explored how excessive stimulation of different DA receptors within the NAc core and shell alters effort-related decision-making and compared these effects to those induced by CRF stimulation.

METHODS

Male Long Evans rats were well-trained on an effort-discounting task wherein they choose between a low-effort/low-reward and a high-effort/high-reward lever where the effort requirement increased over blocks (2-20 presses). Dopamine D₁ (SKF 81297, 0.2-2 μg), D_2/3 (quinpirole, 1-10 μg), or D₃ (PD 128,907, 1.5-3 μg) receptor agonists, or CRF (0.5 μg), were infused into the NAc core or shell prior to testing.

RESULTS

Stimulation of D_2/3 receptors with quinpirole in the NAc core or shell markedly reduced the choice of high-effort option and increase choice latencies, without altering preference for larger vs smaller rewards. Stimulation of D₁ or D₃ receptors did not alter choice, although SKF 81297 infusions into the shell reduced response vigor. In comparison, core infusions of CRF flattened the discounting curve, reducing effortful choice when costs were low and increasing it when costs were high.

CONCLUSIONS

Excessive stimulation of NAc D₂ receptors has detrimental effects on effort-related decision-making. Furthermore, CRF stimulation induces dissociable effects on decision-making compared with those induced the effects of stimulation of different DA receptors.

Alpha-melanocyte stimulating hormone increases the activity of melanocortin-3 receptor-expressing neurons in the ventral tegmental area

KEY POINTS

Alpha-melanocyte stimulating hormone (α-MSH) is an anorexigenic peptide. Injection of the α-MSH analog MTII into the ventral tegmental area (VTA) decreases food and sucrose intake and food reward. Melanocortin-3 receptors (MC3R) are highly expressed in the VTA, suggesting that the effects of intra-VTA α-MSH may be mediated by α-MSH changing the activity of MC3R-expressing VTA neurons. α-MSH increased the firing rate of MC3R VTA neurons in acute brain slices from mice, although it did not affect the firing rate of non-MC3R VTA neurons. The α-MSH induced increase in MC3R neuron firing rate is probably activity-dependent, and was independent of fast synaptic transmission and intracellular Ca²⁺ levels. These results help us to better understand how α-MSH acts in the VTA to affect feeding and other dopamine-dependent behaviours.

ABSTRACT

The mesocorticolimbic dopamine system, the brain's reward system, regulates multiple behaviours, including food intake and food reward. There is substantial evidence that the melanocortin system of the hypothalamus, an important neural circuit controlling feeding and body weight, interacts with the mesocorticolimbic dopamine system to affect feeding, food reward and body weight. For example, melanocortin-3 receptors (MC3Rs) are expressed in the ventral tegmental area (VTA) and our laboratory previously showed that intra-VTA injection of the MC3R agonist, MTII, decreases home-cage food intake and operant responding for sucrose pellets. However, the cellular mechanisms underlying the effects of intra-VTA alpha-melanocyte stimulating hormone (α-MSH) on feeding and food reward are unknown. To determine how α-MSH acts in the VTA to affect feeding, we performed electrophysiological recordings in acute brain slices from mice expressing enhanced yellow fluorescent protein in MC3R neurons to test how α-MSH affects the activity of VTA MC3R neurons. α-MSH significantly increased the firing rate of VTA MC3R neurons without altering the activity of non-MC3R expressing VTA neurons. In addition, the α-MSH-induced increase in MC3R neuron activity was independent of fast synaptic transmission and intracellular Ca²⁺ levels. Finally, we show that the effect of α-MSH on MC3R neuron firing rate is probably activity-dependent. Overall, these studies provide an important advancement in the understanding of how α-MSH acts in the VTA to affect feeding and food reward.

Dopamine depletion shifts behavior from activity based reinforcers to more sedentary ones and adenosine receptor antagonism reverses that shift: Relation to ventral striatum DARPP32 phosphorylation patterns

The mesolimbic dopamine (DA) system plays a critical role in behavioral activation and effort-based decision-making. DA depletion produces anergia (shifts to low effort options) in animals tested on effort-based decision-making tasks. Caffeine, the most consumed stimulant in the world, acts as an adenosine A₁/A_2A receptor antagonist, and in striatal areas DA D₁ and D₂ receptors are co-localized with adenosine A₁ and A_2A receptors respectively. In the present work, we evaluated the effect of caffeine on anergia induced by the VMAT-2 inhibitor tetrabenazine (TBZ), which depletes DA. Anergia was evaluated in a three-chamber T-maze task in which animals can chose between running on a wheel (RW) vs. sedentary activities such as consuming sucrose or sniffing a neutral odor. TBZ-caffeine interactions in ventral striatum were evaluated using DARPP-32 phosphorylation patterns as an intracellular marker of DA-adenosine receptor interaction. In the T-maze, control mice spent more time running and much less consuming sucrose or sniffing. TBZ (4.0 mg/kg) reduced ventral striatal DA tissue levels as measured by HPLC, and also shifted preferences in the T-maze, reducing selection of the reinforcer that involved vigorous activity (RW), but increasing consumption of a reinforcer that required little effort (sucrose), at doses that had no effect on independent measures of appetite or locomotion in a RW. Caffeine at doses that had no effect on their own reversed the effects of TBZ on T-maze performance, and also suppressed TBZ-induced pDARPP-32(Thr34) expression as measured by western blot, suggesting a role for D₂-A_2A interactions. These results support the idea that DA depletion produces anergia, but does not affect the primary motivational effects of sucrose. Caffeine, possibly by acting on A_2A receptors in ventral striatum, reversed the DA depletion effects. It is possible that caffeine, like selective adenosine A2A antagonists, could have some therapeutic benefit for treating effort-related symptoms.

Dopamine, Effort-Based Choice, and Behavioral Economics: Basic and Translational Research

Operant behavior is not only regulated by factors related to the quality or quantity of reinforcement, but also by the work requirements inherent in performing instrumental actions. Moreover, organisms often make effort-related decisions involving economic choices such as cost/benefit analyses. Effort-based decision making is studied using behavioral procedures that offer choices between high-effort options leading to relatively preferred reinforcers vs. low effort/low reward choices. Several neural systems, including the mesolimbic dopamine (DA) system and other brain circuits, are involved in regulating effort-related aspects of motivation. Considerable evidence indicates that mesolimbic DA transmission exerts a bi-directional control over exertion of effort on instrumental behavior tasks. Interference with DA transmission produces a low-effort bias in animals tested on effort-based choice tasks, while increasing DA transmission with drugs such as DA transport blockers tends to enhance selection of high-effort options. The results from these pharmacology studies are corroborated by the findings from recent articles using optogenetic, chemogenetic and physiological techniques. In addition to providing important information about the neural regulation of motivated behavior, effort-based choice tasks are useful for developing animal models of some of the motivational symptoms that are seen in people with various psychiatric and neurological disorders (e.g., depression, schizophrenia, Parkinson’s disease). Studies of effort-based decision making may ultimately contribute to the development of novel drug treatments for motivational dysfunction.

Dopamine Manipulation Affects Response Vigor Independently of Opportunity Cost

Dopamine is known to be involved in regulating effort investment in relation to reward, and the disruption of this mechanism is thought to be central in some pathological situations such as Parkinson's disease, addiction, and depression. According to an influential model, dopamine plays this role by encoding the opportunity cost, i.e., the average value of forfeited actions, which is an important parameter to take into account when making decisions about which action to undertake and how fast to execute it. We tested this hypothesis by asking healthy human participants to perform two effort-based decision-making tasks, following either placebo or levodopa intake in a double blind within-subject protocol. In the effort-constrained task, there was a trade-off between the amount of force exerted and the time spent in executing the task, such that investing more effort decreased the opportunity cost. In the time-constrained task, the effort duration was constant, but exerting more force allowed the subject to earn more substantial reward instead of saving time. Contrary to the model predictions, we found that levodopa caused an increase in the force exerted only in the time-constrained task, in which there was no trade-off between effort and opportunity cost. In addition, a computational model showed that dopamine manipulation left the opportunity cost factor unaffected but altered the ratio between the effort cost and reinforcement value. These findings suggest that dopamine does not represent the opportunity cost but rather modulates how much effort a given reward is worth.

SIGNIFICANCE STATEMENT

Dopamine has been proposed in a prevalent theory to signal the average reward rate, used to estimate the cost of investing time in an action, also referred to as opportunity cost. We contrasted the effect of dopamine manipulation in healthy participants in two tasks, in which increasing response vigor (i.e., the amount of effort invested in an action) allowed either to save time or to earn more reward. We found that levodopa-a synthetic precursor of dopamine-increases response vigor only in the latter situation, demonstrating that, rather than the opportunity cost, dopamine is involved in computing the expected value of effort.

Deciphering Decision Making: Variation in Animal Models of Effort- and Uncertainty-Based Choice Reveals Distinct Neural Circuitries Underlying Core Cognitive Processes

Maladaptive decision-making is increasingly recognized to play a significant role in numerous psychiatric disorders, such that therapeutics capable of ameliorating core impairments in judgment may be beneficial in a range of patient populations. The field of "decision neuroscience" is therefore in its ascendancy, with researchers from diverse fields bringing their expertise to bear on this complex and fascinating problem. In addition to the advances in neuroimaging and computational neuroscience that contribute enormously to this area, an increase in the complexity and sophistication of behavioral paradigms designed for nonhuman laboratory animals has also had a significant impact on researchers' ability to test the causal nature of hypotheses pertaining to the neural circuitry underlying the choice process. Multiple such decision-making assays have been developed to investigate the neural and neurochemical bases of different types of cost/benefit decisions. However, what may seem like relatively trivial variation in behavioral methodologies can actually result in recruitment of distinct cognitive mechanisms, and alter the neurobiological processes that regulate choice. Here we focus on two areas of particular interest, namely, decisions that involve an assessment of uncertainty or effort, and compare some of the most prominent behavioral paradigms that have been used to investigate these processes in laboratory rodents. We illustrate how an appreciation of the diversity in the nature of these tasks can lead to important insights into the circumstances under which different neural regions make critical contributions to decision making.

Neuropeptide-Y alters VTA dopamine neuron activity through both pre- and postsynaptic mechanisms

The mesocorticolimbic dopamine system, the brain's reward system, regulates many different behaviors including food intake, food reward, and feeding-related behaviors, and there is increasing evidence that hypothalamic feeding-related neuropeptides alter dopamine neuron activity to affect feeding. For example, neuropeptide-Y (NPY), a strong orexigenic hypothalamic neuropeptide, increases motivation for food when injected into the ventral tegmental area (VTA). How NPY affects the activity of VTA dopamine neurons to regulate feeding behavior is unknown, however. In these studies we have used whole cell patch-clamp electrophysiology in acute brain slices from mice to examine how NPY affects VTA dopamine neuron activity. NPY activated an outward current that exhibited characteristics of a G protein-coupled inwardly rectifying potassium channel current in ~60% of dopamine neurons tested. In addition to its direct effects on VTA dopamine neurons, NPY also decreased the amplitude and increased paired-pulse ratios of evoked excitatory postsynaptic currents in a subset of dopamine neurons, suggesting that NPY decreases glutamatergic transmission through a presynaptic mechanism. Interestingly, NPY also strongly inhibited evoked inhibitory postsynaptic currents onto dopamine neurons by a presynaptic mechanism. Overall these studies demonstrate that NPY utilizes multiple mechanisms to affect VTA dopamine neuron activity, and they provide an important advancement in our understanding of how NPY acts in the VTA to control feeding behavior.NEW & NOTEWORTHY Neuropeptide-Y (NPY) has been shown to act on mesolimbic dopamine circuits to increase motivated behaviors toward food, but it is unclear exactly how NPY causes these responses. Here, we demonstrate that NPY directly inhibited a subset of ventral tegmental area (VTA) dopamine neurons through the activation of G protein-coupled inwardly rectifying potassium currents, and it inhibited both excitatory postsynaptic currents and inhibitory postsynaptic currents onto subsets of dopamine neurons through a presynaptic mechanism. Thus NPY uses multiple mechanisms to dynamically control VTA dopamine neuron activity.

Assessment of a glycine uptake inhibitor in animal models of effort-related choice behavior: implications for motivational dysfunctions

RATIONALE

Motivated behavior can be characterized by a substantial exertion of effort, and organisms often make effort-related decisions based upon analyses of work-related response costs and reinforcement preference. Moreover, alterations in effort-based choice can be seen in people with major depression and schizophrenia. Effort-related decision making is studied using tasks offering choices between high effort options leading to highly valued reinforces vs low effort/low reward options. Interference with dopamine (DA) transmission by administration of the DA D2 family antagonist haloperidol biases behavior towards the lower effort option that can be obtained with minimal work, and previous research has shown that DA interacts with other transmitters, including adenosine and GABA, to regulate effort-based choice.

OBJECTIVES

The present studies focused upon the ability of the glycine transport inhibitor bitopertin to attenuate haloperidol-induced shifts in effort-related choice behavior.

METHODS

Effort-based choice in rats was assessed using the concurrent fixed ratio (FR) 5/chow feeding choice task and the T-maze barrier choice procedure.

RESULTS

Haloperidol shifted effort-based choice, biasing animals towards the low effort option in each task. Co-administration of bitopertin (1.0-10.0 mg/kg) significantly attenuated haloperidol-induced shifts in choice behavior, but the same doses of bitopertin had no effect when administered alone.

CONCLUSIONS

These results indicated that elevation of extracellular glycine via inhibition of glycine uptake was able to reverse the effects of D2 antagonism. Increases in extracellular glycine, possibly through actions on the glycine allosteric site on the NMDA receptor, may be a useful strategy for treating motivational dysfunctions in humans.

Effort-related motivational effects of the pro-inflammatory cytokine interleukin-6: pharmacological and neurochemical characterization

RATIONALE

Motivational dysfunctions such as anergia, fatigue, and reduced effort expenditure are common in patients with depression and other disorders. Pro-inflammatory cytokines are implicated in depression, and cytokine administration induces motivational deficits in humans.

OBJECTIVES

These studies focused on the effects of the cytokine interleukin-6 (IL-6) on effort-related decision-making.

METHODS

Rats were assessed using the concurrent fixed ratio 5-lever pressing/chow feeding choice procedure, which measures the tendency of rats to work for a preferred food (high carbohydrate pellets) in the presence of a concurrently available but less preferred substitute (lab chow).

RESULTS

IL-6 (2.0-8.0 μg/kg IP) shifted choice behavior, significantly decreasing lever pressing and increasing chow intake. Further experiments showed that the adenosine A2A antagonist MSX-3 and the stimulant methylphenidate attenuated the effort-related impairments produced by IL-6, increasing lever pressing and decreasing chow intake in IL-6 treated rats. The same doses of IL-6 did not alter food intake or preference in parallel free-feeding choice studies, demonstrating that these low doses were not altering preference for the high carbohydrate pellets or generally suppressing appetite. Also, IL-6 did not affect body temperature. Microdialysis studies showed that 8.0 μg/kg IL-6 significantly decreased extracellular dopamine in nucleus accumbens core.

CONCLUSIONS

In summary, IL-6 reduces the tendency to work for food, even at low doses that do not produce fever or loss of appetite. Dopaminergic mechanisms may be involved in these effort-related effects. This research has implications for the involvement of cytokines in motivational dysfunctions such as anergia and fatigue.

Behavioral representation of cost and benefit balance in rats

Decision making is dependent upon individual motivation. Previous studies showed that animals with higher levels of motivation are more likely to invest more time to acquire larger rewards rather than acquiring smaller rewards with less time to wait. However, little is known about how this motivation mediates the cognitive effort animals devote upon making said decisions in detail. In the present study, we investigated the behavioral response in a goal-directed action under a differential reward schedule by training rats to perform a "Do more, get more" (DM-GM) task using a nosepoke operandum when longer nosepoke durations resulted in correspondingly larger rewards. In general, the subjects learned this DM-GM rule and reached a steady behavioral state within 15days. During the training stage, the rats found the most cost-effective action choice and behaved according to that guideline more frequently than other possible actions. In addition, when the cost-benefit ratio changed, the rats again found a new most cost-effective choice to obtain maximum rewards. Our results demonstrate that there is a "balance point" of cost and benefit in rat valuation system and that this "balance point" not only guides the rats to make the appropriate decision, but that this point can be modified upon new situations to choose a newer optimum action plan.

The pharmacology of effort-related choice behavior: Dopamine, depression, and individual differences

This review paper is focused upon the involvement of mesolimbic dopamine (DA) and related brain systems in effort-based processes. Interference with DA transmission affects instrumental behavior in a manner that interacts with the response requirements of the task, such that rats with impaired DA transmission show a heightened sensitivity to ratio requirements. Impaired DA transmission also affects effort-related choice behavior, which is assessed by tasks that offer a choice between a preferred reinforcer that has a high work requirement vs. less preferred reinforcer that can be obtained with minimal effort. Rats and mice with impaired DA transmission reallocate instrumental behavior away from food-reinforced tasks with high response costs, and show increased selection of low reinforcement/low cost options. Tests of effort-related choice have been developed into models of pathological symptoms of motivation that are seen in disorders such as depression and schizophrenia. These models are being employed to explore the effects of conditions associated with various psychopathologies, and to assess drugs for their potential utility as treatments for effort-related symptoms. Studies of the pharmacology of effort-based choice may contribute to the development of treatments for symptoms such as psychomotor slowing, fatigue or anergia, which are seen in depression and other disorders.

Selection of sucrose concentration depends on the effort required to obtain it: studies using tetrabenazine, D1, D2, and D3 receptor antagonists

RATIONALE

Low doses of dopamine (DA) antagonists and accumbens DA depletions reduce food-reinforced instrumental behavior but do not impair primary food motivation, causing animals to reallocate behavior away from food-reinforced tasks with high response requirements and select less effortful alternatives. However, it is uncertain if this same pattern of effects would occur if sucrose was used as the reinforcer.

OBJECTIVES

These experiments studied the impact of DA depletion and antagonism on performance of an effort-related choice task using sucrose as the reinforcer, as well as sucrose consumption, preference, and taste reactivity tests.

METHODS

The effects of DA manipulations were assessed using a task in which rats chose between lever pressing on a fixed ratio 7 schedule for 5.0 % sucrose versus freely consuming a less concentrated solution (0.3 %).

RESULTS

The DA depleting agent tetrabenazine shifted effort-related choice, decreasing lever pressing for 5.0 % sucrose but increasing intake of the concurrently available 0.3 % sucrose. Tetrabenazine did not affect sucrose appetitive taste reactivity, or sucrose consumption or preference, in free consumption tests. The D1 antagonist ecopipam and the D2 antagonist haloperidol also shifted choice behavior at doses that did not alter sucrose consumption or preference. In contrast, sucrose pre-exposure reduced consumption across all conditions. D3 antagonism had no effects.

CONCLUSIONS

D1 and D2 receptor blockade and DA depletion reduce the tendency to work for sucrose under conditions that leave fundamental aspects of sucrose motivation (intake, preference, hedonic reactivity) intact. These findings have implications for studies employing sucrose intake or preference in animal models of depression.

Perinatal malnutrition stimulates motivation through reward and enhances drd(1a) receptor expression in the ventral striatum of adult mice

AIM

The aim of this study was to analyze the effects of protein perinatal malnutrition on the function of dopamine DRD1 and DRD2 receptors in regards to motivation and food consumption in adult mice. The study also analyzed the effect of protein perinatal malnutrition on the gene expression of these receptors in the ventral striatum.

METHODS

Wistar lineage mice were divided into two groups according to maternal diet: control (17% casein), n=30 and low protein (8% casein), n=30. Between 30 and 120days of life, the following factors were measured: body weight; the effect of dopamine D1 and D2 agonists on the ingestion of palatable food; the motivational aspect under the action of the D1 (SKF 38393) and D2 Quinpirole dopaminergic agonists; and the gene expression of DRD1 and DRD2 receptors in the ventral striatum.

RESULTS

The body weights of the malnourished animals remained significantly lower than those of the control group from 30 to 120days of life. Malnourished animals ingested a greater quantity of palatable food. There was a decrease in palatable diet consumption in both the control and malnourished groups after the application of D1 and D2 agonists; however, the anorexic effect of the D1 agonist was understated in malnourished animals. Perinatal malnutrition increases the motivational behavior of the animal when food reward is used. There was an increase in gene expression of the DRD1a receptor in the ventral striatum of malnourished animals, and there were no significant changes concerning the DRD2 receptor.

CONCLUSIONS

Perinatal protein malnutrition stimulates hedonic control of eating behavior by promoting increased intake of palatable foods, possibly due to increased expression of dopamine receptor DRD1a in the ventral striatum.

The dopamine reuptake inhibitor MRZ-9547 increases progressive ratio responding in rats

Drugs that are able to shift effort-related decision making in intact rats towards high-effort response options are largely unknown. Here, we examined the effects of two candidate drugs, MRZ-9547 and its l-enantiomer MRZ-9546 on progressive ratio (PR) responding using two different tasks, a standard PR task that involves increasing ratio requirements and a PR/chow feeding choice task in which animals can lever press for preferred food pellets under a PR schedule or approach freely available less preferred lab chow. Furthermore, we assessed the mechanisms of action of both drugs using in vitro-assay methods and in vivo-microdialysis. Results reveal that MRZ-9547 is a selective dopamine transporter (DAT) inhibitor that moderately stimulated striatal dopamine release. MRZ-9546 was a much less potent DAT inhibitor. Furthermore, MRZ-9547 dose dependently increased the tendency to work for food reinforcement both in the standard PR task and the PR/chow feeding choice task, MRZ-9546 was considerably less active. Relative to MRZ-9547, other DAT-interfering drugs had only moderate (methylphenidate) or marginal (modafinil, d-amphetamine) stimulant effects on PR responding in either task. Collectively, our data demonstrate that the DAT inhibitor MRZ-9547 can markedly stimulate PR responding and shift effort-related decision making in intact rats towards high-effort response options. An analysis of effort-related decision making in rodents could provide an animal model for motivational dysfunctions related to effort expenditure such as fatigue, e.g. in Parkinson's disease or major depression. Our findings suggest that DAT inhibitors such as MRZ-9547 could be potentially useful for treating energy-related symptoms in neurological or neuropsychiatric disorders.

Lipopolysaccharide reduces incentive motivation while boosting preference for high reward in mice

Inflammation has been implicated in the development of various psychiatric disorders, including depression. However, the neurobehavioral mechanism involved in this relationship remains elusive. This gap in knowledge may best be filled by evaluating elementary neurobehavioral units affected by inflammation rather than behavioral changes in conventional animal tests of depression. To this end, the current study used a concurrent choice paradigm to evaluate inflammation-induced motivational changes. Male C57BL/6J mice (n=27) were food restricted to between 85 and 90% of their free-feeding weight and were trained to perform a concurrent choice task where they nose-poked for grain rewards on a fixed ratio (FR) 1 schedule (low effort/low reward) and chocolate-flavored rewards on a FR-10 schedule (high effort/high reward). A counterbalanced-within subjects design was used. A single intraperitoneal injection of 0.33 mg/kg lipopolysaccharide (LPS) was used to induce peripheral inflammation. Twenty-four hours after LPS administration, mice showed a reduction in the total number of nose pokes. A proportionally greater reduction in nose pokes was observed for grain, resulting in an increase in percent chocolate pellets earned. These behavioral changes cannot be explained by reduced appetite as feeding before the test led to a similar increase in percent chocolate pellets earned but without any decrease in responding. These results indicate that inflammation modulates incentive motivation by affecting willingness to exert effort for reward and not by reducing sensitivity to reward.

Nicotine increases impulsivity and decreases willingness to exert cognitive effort despite improving attention in "slacker" rats: insights into cholinergic regulation of cost/benefit decision making

Successful decision making in our daily lives requires weighing an option’s costs against its associated benefits. The neuromodulator acetylcholine underlies both the etiology and treatment of a number of illnesses in which decision making is perturbed, including Alzheimer’s disease, attention-deficit/hyperactivity disorder, and schizophrenia. Nicotine acts on the cholinergic system and has been touted as a cognitive enhancer by both smokers and some researchers for its attention-boosting effects; however, it is unclear whether treatments that have a beneficial effect on attention would also have a beneficial effect on decision making. Here we utilize the rodent Cognitive Effort Task (rCET), wherein animals can choose to allocate greater visuospatial attention for a greater reward, to examine cholinergic contributions to both attentional performance and choice based on attentional demand. Following the establishment of baseline behavior, four drug challenges were administered: nicotine, mecamylamine, scopolamine, and oxotremorine (saline plus three doses for each). As per previous rCET studies, animals were divided by their baseline preferences, with “worker” rats choosing high-effort/high-reward options more than their “slacker” counterparts. Nicotine caused slackers to choose even fewer high-effort trials than at baseline, but had no effect on workers’ choice. Despite slackers’ decreased willingness to expend effort, nicotine improved their attentional performance on the task. Nicotine also increased measures of motor impulsivity in all animals. In contrast, scopolamine decreased animals’ choice of high-effort trials, especially for workers, while oxotremorine decreased motor impulsivity for all animals. In sum, the cholinergic system appears to contribute to decision making, and in part these contributions can be understood as a function of individual differences. While nicotine has been considered as a cognitive enhancer, these data suggest that its modest benefits to attention may be coupled with impulsiveness and decreased willingness to work hard, especially in individuals who are particularly sensitive to effort costs (i.e. slackers).

A role for the lateral dorsal tegmentum in memory and decision neural circuitry

A role for the hippocampus in memory is clear, although the mechanism for its contribution remains a matter of debate. Converging evidence suggests that hippocampus evaluates the extent to which context-defining features of events occur as expected. The consequence of mismatches, or prediction error, signals from hippocampus is discussed in terms of its impact on neural circuitry that evaluates the significance of prediction errors: Ventral tegmental area (VTA) dopamine cells burst fire to rewards or cues that predict rewards (Schultz, Dayan, & Montague, 1997). Although the lateral dorsal tegmentum (LDTg) importantly controls dopamine cell burst firing (Lodge & Grace, 2006) the behavioral significance of the LDTg control is not known. Therefore, we evaluated LDTg functional activity as rats performed a spatial memory task that generates task-dependent reward codes in VTA (Jo, Lee, & Mizumori, 2013; Puryear, Kim, & Mizumori, 2010) and another VTA afferent, the pedunculopontine nucleus (PPTg, Norton, Jo, Clark, Taylor, & Mizumori, 2011). Reversible inactivation of the LDTg significantly impaired choice accuracy. LDTg neurons coded primarily egocentric information in the form of movement velocity, turning behaviors, and behaviors leading up to expected reward locations. A subset of the velocity-tuned LDTg cells also showed high frequency bursts shortly before or after reward encounters, after which they showed tonic elevated firing during consumption of small, but not large, rewards. Cells that fired before reward encounters showed stronger correlations with velocity as rats moved toward, rather than away from, rewarded sites. LDTg neural activity was more strongly regulated by egocentric behaviors than that observed for PPTg or VTA cells that were recorded by Puryear et al. and Norton et al. While PPTg activity was uniquely sensitive to ongoing sensory input, all three regions encoded reward magnitude (although in different ways), reward expectation, and reward encounters. Only VTA encoded reward prediction errors. LDTg may inform VTA about learned goal-directed movement that reflects the current motivational state, and this in turn may guide VTA determination of expected subjective goal values. When combined it is clear the LDTg and PPTg provide only a portion of the information that dopamine cells need to assess the value of prediction errors, a process that is essential to future adaptive decisions and switches of cognitive (i.e. memorial) strategies and behavioral responses.

Dopamine signaling in the amygdala, increased by food ingestion and GLP-1, regulates feeding behavior

Mesolimbic dopamine plays a critical role in food-related reward processing and learning. The literature focuses primarily on the nucleus accumbens as the key dopaminergic target in which enhanced dopamine signaling is associated with reward. Here, we demonstrate a novel neurobiological mechanism by which dopamine transmission in the amygdala regulates food intake and reward. We show that food intake was associated with increased dopamine turnover in the amygdala. Next, we assess the impact of direct intra-amygdala D1 and D2 receptor activation on food intake and sucrose-driven progressive ratio operant conditioning in rats. Amygdala D2 receptor activation reduced food intake and operant behavior for sucrose, whereas D2 receptor blockade increased food intake but surprisingly reduced operant behavior. In contrast, D1 receptor stimulation or blockade did not alter feeding or operant conditioning for food. The glucagon-like peptide 1 (GLP-1) system, a target for type 2 diabetes treatment, in addition to regulating glucose homeostasis, also reduces food intake. We found that central GLP-1R receptor activation is associated with elevated dopamine turnover in the amygdala, and that part of the anorexic effect of GLP-1 is mediated by D2 receptor signaling in the amygdala. Our findings indicate that amygdala dopamine signaling is activated by both food intake and the anorexic brain-gut peptide GLP-1 and that amygdala D2 receptor activation is necessary and sufficient to change feeding behavior. Collectively these studies indicate a novel mechanism by which the dopamine system affects feeding-oriented behavior at the level of the amygdala.

Effort-related motivational effects of the pro-inflammatory cytokine interleukin 1-beta: studies with the concurrent fixed ratio 5/ chow feeding choice task

RATIONALE

Effort-related motivational symptoms such as anergia and fatigue are common in patients with depression and other disorders. Research implicates pro-inflammatory cytokines in depression, and administration of cytokines can induce effort-related motivational symptoms in humans.

OBJECTIVES

The present experiments focused on the effects of the pro-inflammatory cytokine interleukin 1-beta (IL-1β) on effort-related choice behavior.

METHODS

Rats were tested on a concurrent fixed ratio 5 lever pressing/chow feeding choice procedure, which assesses the tendency of rats to work for a preferred food (high carbohydrate pellets) in the presence of a concurrently available but less preferred substitute (laboratory chow).

RESULTS

IL-1β (1.0-4.0 μg/kg IP) shifted choice behavior, significantly decreasing lever pressing and increasing intake of the freely available chow. The second experiment assessed the ability of the adenosine A2A antagonist (E)-phosphoric acid mono-[3-[8-[2-(3-methoxyphenyl)vinyl]-7-methyl-2,6-dioxo-1-prop-2-ynyl-1,2,6,7-tetrahydropurin-3-yl] propyl] ester disodium salt (MSX-3) to reverse the behavioral effects of IL-1β. MSX-3 attenuated the effort-related impairments produced by IL-1β, increasing lever pressing and also decreasing chow intake. In the same dose range that shifted effort-related choice behavior, IL-1β did not alter food intake or preference in parallel free-feeding choice studies, indicating that these low doses were not generally suppressing appetite or altering preference for the high carbohydrate pellets. In addition, IL-1β did not affect core body temperature.

CONCLUSIONS

These results indicate that IL-1β can reduce the tendency to work for food, even at low doses that do not produce a general sickness, malaise, or loss of appetite. This research has implications for the involvement of cytokines in motivational symptoms such as anergia and fatigue.

Effort-related motivational effects of the VMAT-2 inhibitor tetrabenazine: implications for animal models of the motivational symptoms of depression

Motivated behaviors are often characterized by a high degree of behavioral activation, and work output and organisms frequently make effort-related decisions based upon cost/benefit analyses. Moreover, people with major depression and other disorders often show effort-related motivational symptoms such as anergia, psychomotor retardation, and fatigue. It has been suggested that tasks measuring effort-related choice behavior could be used as animal models of the motivational symptoms of depression, and the present studies characterized the effort-related effects of the vesicular monoamine transport (VMAT) inhibitor tetrabenazine. Tetrabenazine produces depressive symptoms in humans and, because of its selective inhibition of VMAT-2, it preferentially depletes dopamine (DA). Rats were assessed using a concurrent fixed-ratio 5/chow feeding choice task that is known to be sensitive to dopaminergic manipulations. Tetrabenazine shifted response choice in rats, producing a dose-related decrease in lever pressing and a concomitant increase in chow intake. However, it did not alter food intake or preference in parallel free-feeding choice studies. The effects of tetrabenazine on effort-related choice were reversed by the adenosine A2A antagonist MSX-3 and the antidepressant bupropion. A behaviorally active dose of tetrabenazine decreased extracellular DA in nucleus accumbens and increased expression of DARPP-32 in accumbens medium spiny neurons in a pattern indicative of reduced transmission at both D1 and D2 DA receptors. These experiments demonstrate that tetrabenazine, which is used in animal models to produce depression-like effects, can alter effort-related choice behavior. These studies have implications for the development of animal models of the motivational symptoms of depression and related disorders.

Measuring reinforcement learning and motivation constructs in experimental animals: relevance to the negative symptoms of schizophrenia

The present review article summarizes and expands upon the discussions that were initiated during a meeting of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS; http://cntrics.ucdavis.edu) meeting. A major goal of the CNTRICS meeting was to identify experimental procedures and measures that can be used in laboratory animals to assess psychological constructs that are related to the psychopathology of schizophrenia. The issues discussed in this review reflect the deliberations of the Motivation Working Group of the CNTRICS meeting, which included most of the authors of this article as well as additional participants. After receiving task nominations from the general research community, this working group was asked to identify experimental procedures in laboratory animals that can assess aspects of reinforcement learning and motivation that may be relevant for research on the negative symptoms of schizophrenia, as well as other disorders characterized by deficits in reinforcement learning and motivation. The tasks described here that assess reinforcement learning are the Autoshaping Task, Probabilistic Reward Learning Tasks, and the Response Bias Probabilistic Reward Task. The tasks described here that assess motivation are Outcome Devaluation and Contingency Degradation Tasks and Effort-Based Tasks. In addition to describing such methods and procedures, the present article provides a working vocabulary for research and theory in this field, as well as an industry perspective about how such tasks may be used in drug discovery. It is hoped that this review can aid investigators who are conducting research in this complex area, promote translational studies by highlighting shared research goals and fostering a common vocabulary across basic and clinical fields, and facilitate the development of medications for the treatment of symptoms mediated by reinforcement learning and motivational deficits.

Nucleus accumbens neurotransmission and effort-related choice behavior in food motivation: effects of drugs acting on dopamine, adenosine, and muscarinic acetylcholine receptors

Mesolimbic dopamine (DA) is a critical component of the brain circuitry regulating behavioral activation and effort-related processes. Although nucleus accumbens (NAc) DA depletions or antagonism leave aspects of appetite and primary food motivation intact, rats with impaired DA transmission reallocate their instrumental behavior away from food-reinforced tasks with high response requirements, and instead select less effortful food-seeking behaviors. Previous work showed that adenosine A2A antagonists can reverse the effects of DA D2 antagonists on effort-related choice, and that stimulation of adenosine A2A receptors produces behavioral effects that are similar to those induced by DA antagonism. The present review summarizes the literature on the role of NAc DA and adenosine in effort-related processes, and also presents original data on the effects of local stimulation of muscarinic acetylcholine receptors in NAc core. Local injections of the muscarinic agonist pilocarpine directly into NAc core produces shifts in effort-related choice behavior similar to those induced by DA antagonism or A2A receptor stimulation, decreasing lever pressing but increasing chow intake in rats responding on a concurrent fixed ratio/chow feeding choice task. In contrast, injections into a neostriatal control site dorsal to the NAc were ineffective. The actions of pilocarpine on this task were attenuated by co-administration of the muscarinic antagonist scopolamine. Thus, drugs that act on DA, adenosine A2A, and muscarinic receptors regulate effort-related choice behavior, which may have implications for the treatment of psychiatric symptoms such as psychomotor slowing, fatigue or anergia that can be observed in depression and other disorders.

Evaluation of chemically diverse 5-HT₂c receptor agonists on behaviours motivated by food and nicotine and on side effect profiles

RATIONALE

Selective 5-HT2C receptor agonists, such as lorcaserin, are being developed for the treatment of obesity. Studies suggest that they may also have therapeutic potential for addictive behaviours including nicotine dependence, although few drugs of this class have been evaluated.

OBJECTIVES

The primary aim was to evaluate the highly selective 5-HT2C agonist, CP-809101, against food-motivated (operant FR5 and progressive ratio schedules, palatability-induced feeding) and nicotine-motivated (intravenous self-administration, drug discrimination) behaviours in rats and to compare with equivalent findings for the structurally distinct 5-HT2C receptor agonists lorcaserin and Ro 60-0175. The secondary aims were to evaluate the side effect profiles of lorcaserin and CP-809101 and to determine the plasma levels of lorcaserin at a dose (1 mg/kg) that reduces both food and nicotine reinforcement for comparison to plasma concentrations reported in human trials.

RESULTS

CP-809101 (0.3-3 mg/kg SC) reduced responding for both nicotine and food and blocked the discriminative stimulus properties of nicotine in a similar manner to lorcaserin and Ro 60-0175. Behaviours such as hypolocomotion, chewing and ptosis became evident following both CP-809101 and lorcaserin administration at higher doses. Plasma levels of lorcaserin were of similar range to those reported in obesity trials.

CONCLUSIONS

These studies support the utility of 5-HT2C agonists as a therapeutic approach to treat nicotine dependence. Plasma exposure levels after acute lorcaserin treatment suggest that equivalent dosages could be used to evaluate these drugs in obesity and smoking cessation trials. Finally, there may be differences in the side effect profiles between lorcaserin and CP-809101, raising the possibility for tolerability differences amongst 5-HT2C agonists.

Receptor-specific modulation of risk-based decision making by nucleus accumbens dopamine

The nucleus accumbens (NAc) serves as an integral node within cortico-limbic circuitry that regulates various forms of cost-benefit decision making. The dopamine (DA) system has also been implicated in enabling organisms to overcome a variety of costs to obtain more valuable rewards. However, it remains unclear how DA activity within the NAc may regulate decision making involving reward uncertainty. This study investigated the contribution of different DA receptor subtypes in the NAc to risk-based decision making, assessed with a probabilistic discounting task. In well-trained rats, D1 receptor blockade with SCH 23,390 decreased preference for larger, uncertain rewards, which was associated with enhanced negative-feedback sensitivity (ie, an increased tendency to select a smaller/certain option after an unrewarded risky choice). Treatment with a D1 agonist (SKF 81,297) optimized decision making, increasing choice of the risky option when reward probability was high, and decreasing preference under low probability conditions. In stark contrast, neither blockade of NAc D2 receptors with eticlopride, nor stimulation of these receptors with quinpirole or bromocriptine influenced risky choice. In comparison, infusion of the D3-preferring agonist PD 128,907 decreased reward sensitivity and risky choice. Collectively, these results show that mesoaccumbens DA refines risk-reward decision biases via dissociable mechanisms recruiting D1 and D3, but not D2 receptors. D1 receptor activity mitigates the effect of reward omissions on subsequent choices to promote selection of reward options that may have greater long-term utility, whereas excessive D3 receptor activity blunts the impact that larger/uncertain rewards have in promoting riskier choices.

Principles of motivation revealed by the diverse functions of neuropharmacological and neuroanatomical substrates underlying feeding behavior

Circuits that participate in specific subcomponents of feeding (e.g., gustatory perception, peripheral feedback relevant to satiety and energy balance, reward coding, etc.) are found at all levels of the neural axis. Further complexity is conferred by the wide variety of feeding-modulatory neurotransmitters and neuropeptides that act within these circuits. An ongoing challenge has been to refine the understanding of the functional specificity of these neurotransmitters and circuits, and there have been exciting advances in recent years. We focus here on foundational work of Dr. Ann Kelley that identified distinguishable actions of striatal opioid peptide modulation and dopamine transmission in subcomponents of reward processing. We also discuss her work in overlaying these neuropharmacological effects upon anatomical pathways that link the telencephalon (cortex and basal ganglia) with feeding-control circuits in the hypothalamus. Using these seminal contributions as a starting point, we will discuss new findings that expand our understanding of (1) the specific, differentiable motivational processes that are governed by central dopamine and opioid transmission, (2) the manner in which other striatal neuromodulators, specifically acetylcholine, endocannabinoids and adenosine, modulate these motivational processes (including via interactions with opioid systems), and (3) the organization of the cortical-subcortical network that subserves opioid-driven feeding. The findings discussed here strengthen the view that incentive-motivational properties of food are coded by substrates and neural circuits that are distinguishable from those that mediate the acute hedonic experience of food reward. Striatal opioid transmission modulates reward processing by engaging frontotemporal circuits, possibly via a hypothalamic-thalamic axis, that ultimately impinges upon hypothalamic modules dedicated to autonomic function and motor pattern control. We will conclude by discussing implications for understanding disorders of "non-homeostatic" feeding.

The behavioral pharmacology of effort-related choice behavior: dopamine, adenosine and beyond

For many years, it has been suggested that drugs that interfere with dopamine (DA) transmission alter the "rewarding" impact of primary reinforcers such as food. Research and theory related to the functions of mesolimbic DA are undergoing a substantial conceptual restructuring, with the traditional emphasis on hedonia and primary reward yielding to other concepts and lines of inquiry. The present review is focused upon the involvement of nucleus accumbens DA in effort-related choice behavior. Viewed from the framework of behavioral economics, the effects of accumbens DA depletions and antagonism on food-reinforced behavior are highly dependent upon the work requirements of the instrumental task, and DA-depleted rats show a heightened sensitivity to response costs, especially ratio requirements. Moreover, interference with accumbens DA transmission exerts a powerful influence over effort-related choice behavior. Rats with accumbens DA depletions or antagonism reallocate their instrumental behavior away from food-reinforced tasks that have high response requirements, and show increased selection of low reinforcement/low cost options. Nucleus accumbens DA and adenosine interact in the regulation of effort-related functions, and other brain structures (anterior cingulate cortex, amygdala, ventral pallidum) also are involved. Studies of the brain systems regulating effort-based processes may have implications for understanding drug abuse, as well as symptoms such as psychomotor slowing, fatigue or anergia in depression and other neurological disorders.

Sensitivity to cognitive effort mediates psychostimulant effects on a novel rodent cost/benefit decision-making task

Amotivational states and insufficient recruitment of mental effort have been observed in a variety of clinical populations, including depression, traumatic brain injury, post-traumatic stress disorder, and attention deficit hyperactivity disorder. Previous rodent models of effort-based decision making have utilized physical costs whereas human studies of effort are primarily cognitive in nature, and it is unclear whether the two types of effortful decision making are underpinned by the same neurobiological processes. We therefore designed a novel rat cognitive effort task (rCET) based on the 5-choice serial reaction time task, a well-validated measure of attention and impulsivity. Within each trial of the rCET, rats are given the choice between an easy or hard visuospatial discrimination, and successful hard trials are rewarded with double the number of sugar pellets. Similar to previous human studies, stable individual variation in choice behavior was observed, with 'workers' choosing hard trials significantly more than their 'slacker' counterparts. Whereas workers 'slacked off' in response to administration of amphetamine and caffeine, slackers 'worked harder' under amphetamine, but not caffeine. Conversely, these stimulants increased motor impulsivity in all animals. Ethanol did not affect animals' choice but invigorated behavior. In sum, we have shown for the first time that rats are differentially sensitive to cognitive effort when making decisions, independent of other processes such as impulsivity, and these baseline differences can influence the cognitive response to psychostimulants. Such findings could inform our understanding of impairments in effort-based decision making and contribute to treatment development.

The novel adenosine A(2A) antagonist prodrug MSX-4 is effective in animal models related to motivational and motor functions

Adenosine A(2A) and dopamine D2 receptors interact to regulate diverse aspects of ventral and dorsal striatal functions related to motivational and motor processes, and it has been suggested that adenosine A(2A) antagonists could be useful for the treatment of depression, parkinsonism and other disorders. The present experiments were performed to characterize the effects of MSX-4, which is an amino acid ester prodrug of the potent and selective adenosine A(2A) receptor antagonist MSX-2, by assessing its ability to reverse pharmacologically induced motivational and motor impairments. In the first group of studies, MSX-4 reversed the effects of the D2 antagonist eticlopride on a concurrent lever pressing/chow feeding task that is used as a measure of effort-related choice behavior. MSX-4 was less potent after intraperitoneal administration than the comparison compound, MSX-3, though both were equally efficacious. With this task, MSX-4 was orally active in the same dose range as MSX-3. MSX-4 also reversed the locomotor suppression induced by eticlopride in the open field, but did not induce anxiogenic effects as measured by the relative amount of interior activity. Behaviorally active doses of MSX-4 also attenuated the increase in c-Fos and pDARPP-32(Thr34) expression in nucleus accumbens core that was induced by injections of eticlopride. In addition, MSX-4 suppressed the oral tremor induced by the anticholinesterase galantamine, which is consistent with an antiparkinsonian profile. These actions of MSX-4 indicate that this compound could have potential utility as a treatment for parkinsonism, as well as some of the motivational symptoms of depression and other disorders.

Adenosine A2A receptor antagonism and genetic deletion attenuate the effects of dopamine D2 antagonism on effort-based decision making in mice

Brain dopamine (DA) and adenosine interact in the regulation of behavioral activation and effort-related processes. In the present studies, a T-maze task was developed in mice for the assessment of effort-related decision making. With this task, the two arms of the maze have different reinforcement densities, and a vertical barrier is positioned in the arm with the higher density (HD), presenting the animal with an effort-related challenge. Under control conditions mice prefer the HD arm, and climb the barrier to obtain the larger amount of food. The DA D(2) receptor antagonist haloperidol decreased selection of the HD arm and increased selection of the arm with the low density of reinforcement. However, the HD arm was still the preferred choice in haloperidol-treated mice trained with barriers in both arms. Pre-feeding the mice to reduce food motivation dramatically increased omissions, an effect that was distinct from the actions of haloperidol. Co-administration of theophylline, a nonselective adenosine receptor antagonist, partially reversed the effects of haloperidol. This effect seems to be mediated by the A(2A) receptor but not the A(1) receptor, since the A(2A) antagonist MSX-3, but not the A(1) antagonist CPT, dose dependently reversed the effects of haloperidol on effort-related choice and on c-Fos expression in the dorsal striatum and nucleus accumbens. In addition, adenosine A(2A) receptor knockout mice were resistant to the effects of haloperidol on effort-related choice in the maze. These results indicate that DA D(2) and adenosine A(2A) receptors interact to regulate effort-related decision making and effort expenditure in mice.

Estradiol modulates effort-based decision making in female rats

Disorders of the dopamine system, such as schizophrenia or stimulant addiction, are associated with impairments in different forms of cost/benefit decision making. The neural circuitry (ie amygdala, prefrontal cortex, nucleus accumbens) underlying these functions receives dopamine input, which is thought to have a central role in mediating cost/benefit decisions. Estradiol modulates dopamine activity, and estrogen receptors (ERs) are found within this neurocircuitry, suggesting that decision making may be influenced by estradiol. The present study examined the contribution of estradiol and selective ERα and β agonists on cost/benefit decision making in adult female Long-Evans rats. An effort-discounting task was utilized, where rats could either emit a single response on a low-reward lever to receive two pellets, or make 2, 5, 10, or 20 responses on a high-reward lever to obtain four pellets. Ovariectomy increased the choice on the high-reward lever, whereas replacement with high (10 μg), but not low (0.3 μg), levels of estradiol benzoate reduced the choice on the high-reward lever. Interestingly, both an ERα agonist (propyl-pyrazole triol (PPT)) and an ERβ agonist (diarylpropionitrile (DPN)) increased choice on the high-reward lever when administered independently, but when these two agonists were combined, a decrease in choice for the high-reward lever was observed. The effects of estradiol, PPT, and DPN were more pronounced 24 h post-administration, suggesting that these effects may be genomic in nature. Together, these results demonstrate that estradiol modulates cost/benefit decision making in females, whereby concomitant activation of ERα and β receptors shifts the decision criteria and reduces preference for larger, yet more costly rewards.