Effect of reinforcer magnitude on performance maintained by progressive-ratio schedules

Control of motivation for sucrose in the paraventricular hypothalamic nucleus by dynorphin peptides and the kappa opioid receptor

The dynorphin peptides are the endogenous ligands for the kappa opioid receptor (KOR) and regulate food intake. Administration of dynorphin-A1-13 (DYN) in the paraventricular hypothalamic nucleus (PVN) increases palatable food intake, and this effect is blocked by co-administration of the orexin-A neuropeptide, which is co-released with DYN in PVN from neurons located in the lateral hypothalamus. While PVN administration of DYN increases palatable food intake, whether it increases food-seeking behaviors has yet to be examined. We tested the effects of DYN and norBNI (a KOR antagonist) on the seeking and consumption of sucrose using a progressive ratio (PR) and demand curve (DC) tasks. In PVN, DYN did not alter the sucrose breaking point (BP) in the PR task nor the elasticity or intensity of demand for sucrose in the DC task. Still, DYN reduced the delay in obtaining sucrose and increased licks during sucrose intake in the PR task, irrespective of the co-administration of orexin-A. In PVN, norBNI increased the delay in obtaining sucrose and reduced licks during sucrose intake in the PR task while increasing elasticity without altering intensity of demand in the DC task. However, subcutaneous norBNI reduced the BP for sucrose and increased the delay in obtaining sucrose in the PR task while reducing the elasticity of demand. Together, these data show different effects of systemic and PVN blockade of KOR on food-seeking, consummatory behaviors, and incentive motivation for sucrose and suggest that KOR activity in PVN is necessary but not sufficient to drive seeking behaviors for palatable food.

Further observations on the reinforcing value of sucrose solutions: Interaction between quantity and concentration

According to the Multiplicative Hyperbolic Model of reinforcer value (MHM), the value of a reinforcer is an increasing hyperbolic function of its size. In addition it is generally accepted that in the case of soluble reinforcers such as sucrose, value is an increasing function of molar concentration. The present experiment examined the interaction between size and concentration on the effectiveness of sucrose reinforcers in an operant choice paradigm. Rats were trained under an adjusting-magnitude schedule in which a response on lever B delivered a fixed volume of a sucrose solution, while a response on lever A delivered a 0.8-M sucrose solution, the volume of which, qA, was adjusted according to the rats' choices. When B was preferred in a given block of trials, qA was increased in the following block; when A was preferred, qA was reduced in the following block. The concentration and volume of B (cB, qB) were varied across six phases of the experiment and the corresponding indifference magnitudes of A (qA(50)) were measured. Log qA(50) increased and qB/qA(50) declined as approximately linear functions of log cB, consistent with the supposition that reinforcer value was determined by multiplicative combination of hyperbolic expressions representing concentration and volume.

Theory of reinforcement schedules

The three principles of reinforcement are (1) events such as incentives and reinforcers increase the activity of an organism; (2) that activity is bounded by competition from other responses; and (3) animals approach incentives and their signs, guided by their temporal and physical conditions, together called the "contingencies of reinforcement." Mathematical models of each of these principles comprised mathematical principles of reinforcement (MPR; Killeen, 1994). Over the ensuing decades, MPR was extended to new experimental contexts. This article reviews the basic theory and its extensions to satiation, warm-up, extinction, sign tracking, pausing, and sequential control in progressive-ratio and multiple schedules. In the latter cases, a single equation balancing target and competing responses governs behavioral contrast and behavioral momentum. Momentum is intrinsic in the fundamental equations, as behavior unspools more slowly from highly aroused responses conditioned by higher rates of incitement than it does from responses from leaner contexts. Habits are responses that have accrued substantial behavioral momentum. Operant responses, being predictors of reinforcement, are approached by making them: The sight and feel of a paw on a lever is approached by placing paw on lever, as attempted for any sign of reinforcement. Behavior in concurrent schedules is governed by approach to momentarily richer patches (melioration). Applications of MPR in behavioral pharmacology and delay discounting are noted.

Comparison of progressive hold and progressive response schedules of reinforcement

Progressive ratio (PR) schedules have been widely used to study motivation to work for a reinforcer. After a post-reinforcer pause, subjects engage in pressing a lever until a reinforcer is obtained. However, the discrete nature of lever presses allows alternative behaviors during inter-response time and has led to several behavioral explanations of pauses and work time. A progressive hold-down (PH) is incompatible with alternative responses and may allow a precise estimation of work time. Performance of rats trained in both PR and PH that received sucrose or intracranial self-stimulation (ICSS) as reinforcer were compared. We observed that rats mastered the PR and PH schedules. Post-reinforcer pauses (PS^RP), work time and inter-reinforcer time increased as a function of the response or hold requirement. However, rats' performance suggested that the PH progression may be experienced by the rats as easier that the PR progression. Elimination of consummatory behavior with ICSS reduced post-reinforcer pause in accordance with predictions of explanatory models of fixed and variable schedules of reinforcement. In the case of PH performance, pauses showed little variation across intermediate requirements but increased rapidly on later requirements; since rats controlled their pause length and work time was close to the hold requirement, time allocation between PR and PH schedules diverged. Finally, the Mathematical Principles of Reinforcement model of Bradshaw and Killeen (Psychopharmacology 2012, 222: 549) rendered a good description of the performance in both PR and PH schedules.

Activity of a direct VTA to ventral pallidum GABA pathway encodes unconditioned reward value and sustains motivation for reward

Dopamine signaling from the ventral tegmental area (VTA) plays critical roles in reward-related behaviors, but less is known about the functions of neighboring VTA GABAergic neurons. We show here that a primary target of VTA GABA projection neurons is the ventral pallidum (VP). Activity of VTA-to-VP-projecting GABA neurons correlates consistently with size and palatability of the reward and does not change following cue learning, providing a direct measure of reward value. Chemogenetic stimulation of this GABA projection increased activity of a subset of VP neurons that were active while mice were seeking reward. Optogenetic stimulation of this pathway improved performance in a cue-reward task and maintained motivation to work for reward over days. This VTA GABA projection provides information about reward value directly to the VP, likely distinct from the prediction error signal carried by VTA dopamine neurons.

Reliability and validity of a transdiagnostic measure of reward valuation effort

To identify biobehavioral mechanisms underlying excessive reward consumption, reward valuation-effort (RV-E) assessments should (a) parallel measures in basic science to permit translation from preclinical to clinical studies; (b) quantify constructs dimensionally from healthy to disease states; and (c) hold relevance across different diagnostic categories. To address these aims, we developed a progressive ratio (PR) task whereby RV-E is measured as breakpoint when participants worked for access to playing a game. We evaluated test-retest reliability of breakpoint and convergent and discriminant validity of interpretations of this score against an established PR task for food. In Study 1, female undergraduates (N = 71; 33% racial minority; 28% ethnic minority) completed the game and food tasks in fasted and fed states. In Study 2, women (N = 189; 29% racial minority; 27% ethnic minority) with eating disorders (n = 158) were compared to controls (n = 31) on tasks. Game task breakpoint demonstrated excellent test-retest reliability, intraclass correlation coefficient (ICC) = .91, 95% CI [.80, -.96], over 2 weeks and convergent validity with the fasted food task (r = .51, p < .001). Consistent with animal models, breakpoint was lower in fed compared to fasted states across tasks, B (SE) = 321.01 (552.40), p < .001. Finally, the game task demonstrated discriminant validity from measurement of satiation. In Study 2, women with eating disorders demonstrated higher breakpoint on both tasks compared to controls, and game PR task breakpoint decreased from a fasted to fed state. The game PR task offers a novel approach for translating results from animal models of RV-E into testable hypotheses in nonclinical and clinical samples. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Rest Is Required to Learn an Appetitively-Reinforced Operant Task in Drosophila

Maladaptive operant conditioning contributes to development of neuropsychiatric disorders. Candidate genes have been identified that contribute to this maladaptive plasticity, but the neural basis of operant conditioning in genetic model organisms remains poorly understood. The fruit fly Drosophila melanogaster is a versatile genetic model organism that readily forms operant associations with punishment stimuli. However, operant conditioning with a food reward has not been demonstrated in flies, limiting the types of neural circuits that can be studied. Here we present the first sucrose-reinforced operant conditioning paradigm for flies. In the paradigm, flies walk along a Y-shaped track with reward locations at the terminus of each hallway. When flies turn in the reinforced direction at the center of the track, they receive a sucrose reward at the end of the hallway. Only flies that rest early in training learn the reward contingency normally. Flies rewarded independently of their behavior do not form a learned association but have the same amount of rest as trained flies, showing that rest is not driven by learning. Optogenetically-induced sleep does not promote learning, indicating that sleep itself is not sufficient for learning the operant task. We validated the sensitivity of this assay to detect the effect of genetic manipulations by testing the classic learning mutant dunce. Dunce flies are learning-impaired in the Y-Track task, indicating a likely role for cAMP in the operant coincidence detector. This novel training paradigm will provide valuable insight into the molecular mechanisms of disease and the link between sleep and learning.

Dorsomedial prefrontal neural ensembles reflect changes in task utility that culminate in task quitting

When performing a physically demanding behavior, sometimes the optimal choice is to quit the behavior rather than persist to minimize energy expenditure for the benefits gained. The dorsomedial prefrontal cortex (dmPFC), consisting of the anterior cingulate cortex and secondary motor area, likely contributes toward such utility assessments. Here, we examined how male rat dmPFC single unit and ensemble-level activity corresponded to changes in task utility and quitting in an effortful weight lifting task. Rats carried out two task paradigms: one that became progressively more physically demanding over time and a second fixed effort version. Rats could quit the task at any time. Dorsomedial PFC neurons were highly responsive to each behavioral stage of the task, consisting of rope pulling, reward retrieval, and reward area leaving. Activity was highest early in sessions, commensurate with the highest relative task utility, then decreased until the point of quitting. Neural ensembles consistently represented the sequential behavioral phases of the task. However, these representations were modified over time and became more distinct over the course of the session. These results suggest that dmPFC neurons represent behavioral states that are dynamically modified as behaviors lose their utility, culminating in task quitting.NEW & NOTEWORTHY When carrying out a physically demanding task, animals must continually assess whether to persist or quit. In this study, we recorded neurons in the dorsomedial prefrontal cortex (dmPFC) of rats as they carried out a challenging weightlifting task, up to the point of quitting. We demonstrate that dmPFC neurons form a representation of the task that is modified, via a decrease in firing rate, by the decreasing the utility of the task that may signal quitting.

The Relevance of Operant Behavior in Conceptualizing the Psychological Well-Being of Captive Animals

The term "psychological well-being" is used in reference to husbandry with animals in human care settings such as research, agriculture, and zoos. This article seeks to clarify and conceptualize the term based upon two approaches that draw from several bodies of literature: the experimental analysis of behavior, experimental psychology, animal welfare and husbandry, farm animal behavior, zoo husbandry, and ethology. One approach focuses on the presence of problem behavior such as stereotypies, depressive-like behavior, and aggression, and emphasizes the conditions under which aberrant behavior in animals under human care occurs. The second approach examines what might be considered wellness by emphasizing opportunities to engage with its environment, or the absence of such opportunities, even if problematic behavior is not exhibited. Here, access to an interactive environment is relatively limited so opportunities for operant (voluntary) behavior could be considered. Designing for operant behavior provides opportunities for variability in both behavior and outcomes. Operant behavior also provides control over the environment, a characteristic that has been a core assumption of well-being. The importance of interactions with one's environment is especially evident in observations that animals prefer opportunities to work for items necessary for sustenance, such as food, over having them delivered freely. These considerations raise the importance of operant behavior to psychological well-being, especially as benefits to animals under human care.

Effects of reward magnitude frames on measures of delay discounting in a hypothetical money scenario

The current study analyzed the effects of three frames of reward magnitude-quantity, volume, and duration-on the rate at which college students discounted hypothetical, delayed monetary rewards. Hypothetical scenarios were presented using the fill-in-the-blank discounting questionnaire and participants made choices between immediate and delayed hypothetical monetary rewards. Scenarios framed the monetary choices as (a) quantity of dollar bills, (b) height (inches) of a stack of dollar bills, and (c) duration of time spent in a hypothetical cash machine to collect dollar bills. For each scenario, participants' subjective values were used to calculate the area under the curve (AuC). Framing resulted in a moderate effect size: The duration frame yielded significantly smaller AuC values compared to the quantity and volume frames. Thus, the framing of reward magnitude was a significant variable in controlling discounting rates for hypothetical, delayed monetary rewards. Subsequent investigations should be aware of the independent effects of the reward magnitude frames on delay discounting rates.

An experimental analysis of jackpot reinforcers

Animal trainers and others often recommend the use of jackpot reinforcers, which are disproportionally large and come as a "surprise" to the animals. Because the actual behavioral effects of these jackpots remain uninvestigated, many basic questions about their use and even definition are unanswered. This series of experiments explored the definitions of jackpots using several different behavioral tests with both rats and pigeons. Because the original description of a jackpot resembled the reinstatement of previously reinforced and extinguished responding with response-independent deliveries of reinforcer, reinstatement effects of a jackpot, defined by its quality, were examined with rats in Experiment 1. In Experiment 2, response-potentiating effects of response-independent and -dependent deliveries of a jackpot, defined by its quantity, were assessed with pigeons when responding had nearly ceased. The response-potentiating effects of the frequency of jackpot-reinforcer delivery were investigated when responding of pigeons was maintained in single (Experiment 3) or concurrent (Experiment 4) schedules of reinforcement. Effects of jackpots on resistance to change were assessed with rats in Experiment 5. The effects of jackpots in each experiment were either absent or unsystematic across the subjects, casting doubt on their utility in animal training. Possible factors contributing to the negative results are discussed.

A Novel Weight Lifting Task for Investigating Effort and Persistence in Rats

Here we present a novel effort-based task for laboratory rats: the weight lifting task (WLT). Studies of effort expenditure in rodents have typically involved climbing barriers within T-mazes or operant lever pressing paradigms. These task designs have been successful for neuropharmacological and neurophysiological investigations, but both tasks involve simple action patterns. High climbing barriers may also present risk of injury to animals and/or issues with tethered recording equipment. In the WLT, a rat is placed in a large rectangular arena and tasked with pulling a rope 30 cm to trigger food delivery at a nearby spout; weights can be added to the rope in 45 g increments to increase the intensity of effort. As compared to lever pressing and barrier jumping, 30 cm of rope pulling is a multi-step action sequence requiring sustained effort. The actions are carried out on the single plane of the arena floor, making it safer for the animal and more suitable for tethered equipment and video tracking. A microcontroller and associated sensors enable precise timestamping of specific behaviors to synchronize with electrophysiological recordings. The rope and reward spout are spatially segregated to allow for spatial discrimination of the effort zone and the reward zone. We validated the task across five cohorts of rats (total n = 35) and report consistent behavioral metrics. The WLT is well-suited for neuropharmacological and/or in vivo neurophysiological investigations surrounding effortful behaviors, particularly when wanting to probe different aspects of effort expenditure (intensity vs. duration).

Evidence that neuropsychological deficits following early life adversity may underlie vulnerability to depression

Early life adversity (ELA) is a risk factor for major depressive disorder (MDD), however the underlying mechanisms are not well understood. Clinical studies suggest that negative affective biases (the process, whereby cognitive processes such as learning and memory and decision-making are modified by emotional state) represent a vulnerability factor for MDD. In this study we investigate the impact of ELA on affective biases and reward-associated behaviours in rats. Sprague Dawley rat pups underwent 14 days of postnatal maternal separation (180 min/day from postnatal day 1: MS180) whilst control pups remained unhandled. In adulthood, affective biases associated with reward learning and decision-making were assessed using the affective bias test (ABT), or judgement bias task (JBT) respectively. Changes in motivation and reward sensitivity were tested in a progressive ratio (PR) schedule of operant responding and the sucrose preference test (SPT) respectively. We observed that MS180 animals expressed enhanced negative biases in response to acute corticosterone treatment but without effects on antidepressant-induced positive biases. ELA animals were impaired in their ability to develop appropriate biases in response to changes in reward value in a modified ABT but in the absence of any changes in reward sensitivity or motivation. No effects on decision-making were observed in the JBT but MS180 animals failed to develop the same more optimistic behavioural profile as controls in response to an increase in reward value. These findings suggest that ELA in rats increases vulnerability to negative affective biases and impairs animals' ability to appropriately learn reward value, independent of a reward sensitivity or changes in motivation. These data provide important evidence linking ELA with relevant neuropsychological impairments that may explain increased risk of developing MDD.

Reinforcer quality matters: A test of the Mathematical Principles of Reinforcement with domestic hens (Gallus gallus domesticus)

This study evaluated the ability of Killeen's (1994) Mathematical Principles of Reinforcement to account for the effects of changes in reinforcer quality on hens' rates of responding on fixed-ratio schedules. Hens were trained to peck a key on a fixed-ratio schedule of reinforcement and then experienced an ascending series of ratio values in two separate conditions. In different conditions, the food reinforcer was either wheat or puffed wheat. Response rates initially increased with increases in ratio requirement before eventually decreasing at larger ratios. Quantitative fits of the model accounted for the data well. The fits revealed that different foods were systematically associated with changes in the specific activation parameter, a, and these were consistent with previous reports of preference for those food items.

Effects of sucrose concentration on choice in an adjusting-magnitude schedule

Rats were trained under a discrete-trials adjusting-magnitude schedule in which a response on lever A delivered either a larger or a smaller sucrose reinforcer (q_A1 = 8 μl, q_A2 = 64 μl) with equal probability, while a response on B delivered a reinforcer whose size q_B was adjusted according to the rats' choices. When A was preferred in a given block of trials, q_B was increased in the following block; when B was preferred, q_B was reduced in the following block. The oscillating changes in q_B, analysed by the Fourier transform, could be described by a power spectrum whose dominant frequency corresponded to a period of 40-50 trial blocks. The equilibrium value of q_B (q_B(50)) was inversely related to sucrose concentration; it significantly exceeded the arithmetic mean of q_A1 and q_A2 when the concentration was 0.2 or 0.4 M, but not when it was 0.8 or 1.6 M. Analysis by mixed-effects modelling revealed a trend for the power of oscillation of q_B to increase monotonically with sucrose concentration; the period of oscillation was not systematically related to sucrose concentration. These results are consistent with predictions derived from a revised version of the multiplicative hyperbolic model of intertemporal choice.

Validation and optimisation of a touchscreen progressive ratio test of motivation in male rats

RATIONALE

Across species, effort-related motivation can be assessed by testing behaviour under a progressive ratio (PR) schedule of reinforcement. However, to date, PR tasks for rodents have been available using traditional operant response systems only.

OBJECTIVES

Touchscreen operant response systems allow the assessment of behaviour in laboratory rodents, using tasks that share high face validity with the computerised assessments used in humans. Here, we sought to optimise a rat touchscreen variant of PR and validate it by assessing the effects of a number of manipulations known to affect PR performance in non-touchscreen paradigms.

METHODS

Separate groups of male Sprague-Dawley rats were trained on PR schedules with either linear (PR4) or exponential (PREXP) schedules of reinforcement. PR performance was assessed in response to manipulations in reward outcome. Animals were tested under conditions of increased reward magnitude and following reward devaluation through a prefeeding procedure. Subsequently, the effects of systemic administration of the dopamine D2/D3 receptor antagonist raclopride and the psychostimulant d-amphetamine were examined as traditional pharmacological methods for manipulating motivation.

RESULTS

Rats reinforced under PR4 and PREXP schedules consistently showed differential patterns of response rates within sessions. Furthermore, both PR4 and PREXP schedules were sensitive to suppression by prefeeding or raclopride administration. Performance under both schedules was facilitated by increasing reward magnitude or d-amphetamine administration.

CONCLUSIONS

Taken together, these findings mirror those observed in lever-based PR paradigms in rats. This study therefore demonstrates the successful validation of the rat touchscreen PR task. This will allow for the assessment of motivation in rats, within the same touchscreen apparatus used for the assessment of complex cognitive processes in this species.

Essential values of cocaine and non-drug alternatives predict the choice between them

This study investigated the relationship between reinforcer value and choice between cocaine and two non-drug alternative reinforcers in rats. The essential value (EV, a behavioral economic measure based on elasticity of demand) of intravenous cocaine and food (Experiment 1) or saccharin (Experiment 2) was determined in the first phase of each experiment. Food had higher EV than cocaine, whereas the EVs of cocaine and saccharin did not differ. In the second phase of each experiment, rats were allowed to make mutually exclusive choices between cocaine and the non-drug alternative reinforcer. The main findings were that the EV of cocaine was a positive predictor of cocaine preference and the EV of food or saccharin was a negative predictor of cocaine preference. An analysis of within-session patterns of choice behavior revealed sequential dependencies, whereby rats were more likely to choose cocaine on a particular trial after having chosen the non-drug alternative on previous trials. When the time between choices was increased, these sequential dependencies disappeared. The results of these experiments are consistent with the suggestion that addiction-like behavior involves both overvaluation of drug reinforcers and undervaluation of non-drug reinforcers.

Chemogenetic activation of the lateral hypothalamus reverses early life stress-induced deficits in motivational drive

Altered motivated behaviour is a cardinal feature of several neuropsychiatric conditions including mood disorders. One well-characterized antecedent to the development of mood disorders is exposure to early life stress (ELS). A key brain substrate controlling motivated behaviour is the lateral hypothalamus (LH). Here, we examined the effect of ELS on LH activation and the motivation to self-administer sucrose. We tested whether chemogenetic activation of LH circuits could modify sucrose responding in ELS rats and examined the impact on LH cell populations. Male rat pups were maternally separated for 0 or 3 h on postnatal days 2-14. During adolescence, rats received bilateral injections of hM3D(Gq), the excitatory designer receptor exclusively activated by designer drugs, into LH. In adulthood, rats were trained to self-administer sucrose and tested under a progressive ratio schedule to determine their motivation for reward following injection with either vehicle or 5 mg/kg clozapine-N-oxide. Brains were processed for Fos-protein immunohistochemistry. ELS significantly suppressed lever responding for sucrose, indicating a long-lasting impact of ELS on motivation circuits. hM3D(Gq) activation of LH increased responding, normalizing deficits in ELS rats, and increased Fos-positive orexin and MCH cell numbers within LH. Our findings indicate that despite being susceptible to environmental stressors, LH circuits retain the capacity to overcome ELS-induced deficits in motivated behaviour.

A novel operant task to assess social reward and motivation in rodents

BACKGROUND

Social reward plays a critical role in the development of beneficial social relationships, and disorders of the mechanisms controlling social reward are involved in the etiology of many psychiatric diseases.

NEW METHOD

We present a novel operant social preference task to quantify social reward in rodents using an apparatus with three chambers separated by one-way vertical-swing doors. The experimental animal is placed in the larger chamber while the two smaller chambers either remain empty or contain a stimulus animal or other potential reward stimulus. Adding weights to the door can alter effort required for rewards.

RESULTS

Hamsters (Mesocricetus auratus) entered the chamber containing a stimulus hamster significantly more frequently than an empty chamber. When the reinforcing effects of social interactions were compared to food reward under progressive cost requirements, the reinforcing effects of social interaction and sunflower seeds were similar. Progressively increasing the door weight decreased number of entries, but increased time spent attempting to open the doors.

COMPARISON WITH EXISTING METHODS

The quantification of the rewarding properties of social interactions has almost exclusively used the conditioned place preference (CPP) paradigm. Although robust and reliable, CPP includes a memory component, because it relies on the association of place with the social interaction while the operant task presented here does not.

CONCLUSIONS

This task allows for detailed and direct assessment of social and non-social rewards that may serve as effective behavioral reinforcers in this operant conditioning model, and it can be used to investigate the neural mechanisms regulating motivation.

A comparison of resetting and nonresetting contingencies in progressive-duration schedules

We compared two progressive schedules of reinforcement in which rats received access to sweetened condensed milk for depressing and holding down a response lever. Duration requirements increased after each reinforcer delivery in a manner similar to progressive-ratio schedules. Under one schedule, any response duration less than that required for reinforcement had no programmed consequences. Under the second schedule, the cumulative duration of all responses could meet the reinforcement criteria. Breaking points were consistently higher when all lever presses, regardless of duration, contributed to meeting the reinforcer requirements. Breaking points under both schedules increased when food deprivation was long enough to result in body-weight reductions, but the sensitivity of the schedules to brief periods of food deprivation was inconsistent. Under both schedules, food deprivation produced an increase in shorter durations, thus reducing the efficiency of responding.

Some properties of an adjusting-magnitude schedule of reinforcement: Implications for models of choice

Rats were trained under a discrete-trials adjusting-magnitude schedule in which a response on lever A delivered either a larger or a smaller reinforcer (q_A1 and q_A2) with equal probability, while a response on B delivered a reinforcer whose size q_B was adjusted according to the rats' choices. When A was preferred in a given block of trials, q_B was increased in the following block; when B was preferred, q_B was reduced in the following block. The oscillating changes in q_B, analysed by the Fourier transform, could be described by a power spectrum with a dominant period of about 50 trial blocks. With q_A1 held constant, the equilibrium value of q_B (q_B(50)) was monotonically related to q_A2, and exceeded the arithmetic mean of q_A1 and q_A2 when q_A1 was substantially greater than q_A2. A model derived from the multiplicative model of intertemporal choice provided a post hoc description of the data. Simulation of block-by-block changes in q_B derived from the model were generally consistent with the experimental data. Implications of the results for models of risky choice and for future use of the schedule in neurobehavioural experiments are discussed.

Neural substrates underlying effort, time, and risk-based decision making in motivated behavior

All mobile organisms rely on adaptive motivated behavior to overcome the challenges of living in an environment in which essential resources may be limited. A variety of influences ranging from an organism's environment, experiential history, and physiological state all influence a cost-benefit analysis which allows motivation to energize behavior and direct it toward specific goals. Here we review the substantial amount of research aimed at discovering the interconnected neural circuits which allow organisms to carry-out the cost-benefit computations which allow them to behave in adaptive ways. We specifically focus on how the brain deals with different types of costs, including effort requirements, delays to reward and payoff riskiness. An examination of this broad literature highlights the importance of the extended neural circuits which enable organisms to make decisions about these different types of costs. This involves Cortical Structures, including the Anterior Cingulate Cortex (ACC), the Orbital Frontal Cortex (OFC), the Infralimbic Cortex (IL), and prelimbic Cortex (PL), as well as the Baso-Lateral Amygdala (BLA), the Nucleus Accumbens (NAcc), the Ventral Pallidal (VP), the Sub Thalamic Nucleus (STN) among others. Some regions are involved in multiple aspects of cost-benefit computations while the involvement of other regions is restricted to information relating to specific types of costs.

Concurrent progressive ratio schedules: Effects of reinforcer probability on breakpoint and response allocation

Although progressive ratio (PR) schedules have been used to explore effects of a range of reinforcer parameters (e.g., magnitude, delay), effects of reinforcer probability remain underexplored. The present project used independently progressing concurrent PR PR schedules to examine effects of reinforcer probability on PR breakpoint (highest completed ratio prior to a session terminating 300s pause) and response allocation. The probability of reinforcement on one lever remained at 100% across all conditions while the probability of reinforcement on the other lever was systematically manipulated (i.e., 100%, 50%, 25%, 12.5%, and a replication of 25%). Breakpoints systematically decreased with decreasing reinforcer probabilities while breakpoints on the control lever remained unchanged. Patterns of switching between the two levers were well described by a choice-by-choice unit price model that accounted for the hyperbolic discounting of the value of probabilistic reinforcers.

Touchscreen assays of learning, response inhibition, and motivation in the marmoset (Callithrix jacchus)

Recent developments in precision gene editing have led to the emergence of the marmoset as an experimental subject of considerable interest and translational value. A better understanding of behavioral phenotypes of the common marmoset will inform the extent to which forthcoming transgenic mutants are cognitively intact. Therefore, additional information regarding their learning, inhibitory control, and motivational abilities is needed. The present studies used touchscreen-based repeated acquisition and discrimination reversal tasks to examine basic dimensions of learning and response inhibition. Marmosets were trained daily to respond to one of the two simultaneously presented novel stimuli. Subjects learned to discriminate the two stimuli (acquisition) and, subsequently, with the contingencies switched (reversal). In addition, progressive ratio performance was used to measure the effort expended to obtain a highly palatable reinforcer varying in magnitude and, thereby, provide an index of relative motivational value. Results indicate that rates of both acquisition and reversal of novel discriminations increased across successive sessions, but that rate of reversal learning remained slower than acquisition learning, i.e., more trials were needed for mastery. A positive correlation was observed between progressive ratio break point and reinforcement magnitude. These results closely replicate previous findings with squirrel monkeys, thus providing evidence of similarity in learning processes across nonhuman primate species. Moreover, these data provide key information about the normative phenotype of wild-type marmosets using three relevant behavioral endpoints.

A progressive-duration schedule of reinforcement

We describe a schedule of reinforcement involving systematic, within-session increases in response-duration requirements. Rats received access to appetitive reinforcers for depressing and holding down a response lever. Duration requirements increased after each reinforcer delivery. Sessions ended when reinforcement criteria were unmet for a period of ten minutes. Breaking points, defined as the terminal duration requirement in effect prior to the end of the session, stabilized when environmental conditions were held constant. Breaking points were sensitive to manipulations of both food deprivation and reinforcer quality. Analogous to progressive-ratio schedules, progressive-duration schedules may provide an assay for measuring the amount of behavior an organism will emit for a given reinforcer under current motivational conditions.

Evidence for a role of 5-HT2C receptors in the motor aspects of performance, but not the efficacy of food reinforcers, in a progressive ratio schedule

RATIONALE

5-Hydroxytryptamine2C (5-HT2C) receptor agonists reduce the breakpoint in progressive ratio schedules of reinforcement, an effect that has been attributed to a decrease of the efficacy of positive reinforcers. However, a reduction of the breakpoint may also reflect motor impairment. Mathematical models can help to differentiate between these processes.

OBJECTIVE

The effects of the 5-HT2C receptor agonist Ro-600175 ((αS)-6-chloro-5-fluoro-α-methyl-1H-indole-1-ethanamine) and the non-selective 5-HT receptor agonist 1-(m-chlorophenyl)piperazine (mCPP) on rats' performance on a progressive ratio schedule maintained by food pellet reinforcers were assessed using a model derived from Killeen's Behav Brain Sci 17:105-172, 1994 general theory of schedule-controlled behaviour, 'mathematical principles of reinforcement'.

METHOD

Rats were trained under the progressive ratio schedule, and running and overall response rates in successive ratios were analysed using the model. The effects of the agonists on estimates of the model's parameters, and the sensitivity of these effects to selective antagonists, were examined.

RESULTS

Ro-600175 and mCPP reduced the breakpoint. Neither agonist significantly affected a (the parameter expressing incentive value), but both agonists increased δ (the parameter expressing minimum response time). The effects of both agonists could be attenuated by the selective 5-HT2C receptor antagonist SB-242084 (6-chloro-5-methyl-N-{6-[(2-methylpyridin-3-yl)oxy]pyridin-3-yl}indoline-1-carboxamide). The effect of mCPP was not altered by isamoltane, a selective 5-HT1B receptor antagonist, or MDL-100907 ((±)2,3-dimethoxyphenyl-1-(2-(4-piperidine)methanol)), a selective 5-HT2A receptor antagonist.

CONCLUSIONS

The results are consistent with the hypothesis that the effect of the 5-HT2C receptor agonists on progressive ratio schedule performance is mediated by an impairment of motor capacity rather than by a reduction of the incentive value of the food reinforcer.

Quantitative analysis of performance on a progressive-ratio schedule: effects of reinforcer type, food deprivation and acute treatment with Δ⁹-tetrahydrocannabinol (THC)

Rats' performance on a progressive-ratio schedule maintained by sucrose (0.6M, 50 μl) and corn oil (100%, 25 μl) reinforcers was assessed using a model derived from Killeen's (1994) theory of schedule-controlled behaviour, 'Mathematical Principles of Reinforcement'. When the rats were maintained at 80% of their free-feeding body weights, the parameter expressing incentive value, a, was greater for the corn oil than for the sucrose reinforcer; the response-time parameter, δ, did not differ between the reinforcer types, but a parameter derived from the linear waiting principle (T0), indicated that the minimum post-reinforcement pause was longer for corn oil than for sucrose. When the rats were maintained under free-feeding conditions, a was reduced, indicating a reduction of incentive value, but δ was unaltered. Under the food-deprived condition, the CB1 cannabinoid receptor agonist Δ(9)-tetrahydrocannabinol (THC: 0.3, 1 and 3 mg kg(-1)) increased the value of a for sucrose but not for corn oil, suggesting a selective enhancement of the incentive value of sucrose; none of the other parameters was affected by THC. The results provide new information about the sensitivity of the model's parameters to deprivation and reinforcer quality, and suggest that THC selectively enhances the incentive value of sucrose.

Effect of streptozotocin-induced diabetes on performance on a progressive ratio schedule

RATIONALE

It has been suggested that streptozotocin (STZ)-induced diabetes causes a motivational deficit in rodents. However, some of the evidence adduced in support of this suggestion may be interpreted in terms of a motor impairment rather than a motivational deficit.

OBJECTIVE

This experiment examined the effect of STZ-induced diabetes on performance on a progressive ratio schedule. The data were analysed using a new model derived from Killeen's (Behav Brain Sci 17:105-172, 1994) Mathematical Principles of Reinforcement model which enables the effects of interventions on motivation or incentive value to be separated from effects on motor function.

METHOD

Animals were trained under a progressive ratio schedule using food-pellet reinforcement. Then they received a single intraperitoneal injection of 50 mg/kg of STZ or the vehicle. Training continued for 30 sessions after treatment. Running and overall response rates in successive ratios were analysed using the new model, and estimates of the model's parameters were compared between groups.

RESULTS

The parameter expressing incentive value was reduced in the group treated with STZ, whereas the parameters expressing motor capacity and post-reinforcement pausing were not affected by the treatment. Blood glucose concentration was significantly elevated in the STZ-treated group compared to the vehicle-treated group.

CONCLUSIONS

The results are consistent with the suggestion that STZ-induced diabetes is associated with a reduction of the incentive value of food.

A quantitative analysis of the effects of qualitatively different reinforcers on fixed ratio responding in inbred strains of mice

Previous studies of inbred mouse strains have shown reinforcer-strain interactions that may potentially mask differences among strains in memory performance. The present research examined the effects of two qualitatively different reinforcers (heterogeneous mix of flavored pellets and sweetened-condensed milk) on responding maintained by fixed-ratio schedules of reinforcement in three inbred strains of mice (BALB/c, C57BL/6, and DBA/2). Responses rates for all strains were a bitonic (inverted U) function of the size of the fixed-ratio schedule and were generally higher when responding was maintained by milk. For the DBA/2 and C57BL/6 and to a lesser extent the BALB/c, milk primarily increased response rates at moderate fixed ratios, but not at the largest fixed ratios tested. A formal model of ratio-schedule performance, Mathematical Principles of Reinforcement (MPR), was applied to the response rate functions of individual mice. According to MPR, the differences in response rates maintained by pellets and milk were mostly due to changes in motoric processes as indicated by changes in the minimum response time (δ) produced by each reinforcer type and not specific activation (a), a model term that represents value and is correlated with reinforcer magnitude and the break point obtained under progressive ratio schedules. MPR also revealed that, although affected by reinforcer type, a parameter interpreted as the rate of saturation of working memory (λ), differed among the strains.

Effects of SKF-83566 and haloperidol on performance on progressive ratio schedules maintained by sucrose and corn oil reinforcement: quantitative analysis using a new model derived from the Mathematical Principles of Reinforcement (MPR)

RATIONALE

Mathematical models can assist the interpretation of the effects of interventions on schedule-controlled behaviour and help to differentiate between processes that may be confounded in traditional performance measures such as response rate and the breakpoint in progressive ratio (PR) schedules.

OBJECTIVE

The effects of a D1-like dopamine receptor antagonist, 8-bromo-2,3,4,5-tetrahydro-3-methyl-5-phenyl-1H-3-benzazepin-7-ol hydrobromide (SKF-83566), and a D2-like receptor antagonist, haloperidol, on rats' performance on PR schedules maintained by sucrose and corn oil reinforcers were assessed using a new model derived from Killeen's (Behav Brain Sci 17:105-172, 1994) Mathematical Principles of Reinforcement.

METHOD

Separate groups of rats were trained under a PR schedule using sucrose or corn oil reinforcers. SKF-83566 (0.015 and 0.03 mg kg(-1)) and haloperidol (0.05 and 0.1 mg kg(-1)) were administered intraperitoneally (five administrations of each treatment). Running and overall response rates in successive ratios were analysed using the new model, and estimates of the model's parameters were compared between treatments.

RESULTS

Haloperidol reduced a (the parameter expressing incentive value) in the case of both reinforcers, but did not affect the parameters related to response time and post-reinforcement pausing. SKF-83566 reduced a and k (the parameter expressing sensitivity of post-reinforcement pausing to the prior inter-reinforcement interval) in the case of sucrose, but did not affect any of the parameters in the case of corn oil.

CONCLUSIONS

The results are consistent with the hypothesis that blockade of both D1-like and D2-like receptors reduces the incentive value of sucrose, whereas the incentive value of corn oil is more sensitive to blockade of D2-like than D1-like receptors.

Effects of amisulpride and aripiprazole on progressive-ratio schedule performance: comparison with clozapine and haloperidol

Clozapine and some other atypical antipsychotics (e.g. quetiapine, olanzapine) have been found to exert a characteristic profile of action on operant behaviour maintained by progressive-ratio schedules, as revealed by Killeen's Mathematical Principles of Reinforcement model of schedule-controlled behaviour. These drugs increase the value of a parameter that expresses the 'incentive value' of the reinforcer (a) and a parameter that is inversely related to the organism's 'motor capacity' (δ). This experiment examined the effects of two further atypical antipsychotics, aripiprazole and amisulpride, on progressive-ratio schedule performance in rats; the effects of clozapine and a conventional antipsychotic, haloperidol, were also examined. In agreement with previous findings, clozapine (4, 8 mg kg⁻¹) increased a and δ, whereas haloperidol (0.05, 0.1 mg kg⁻¹) reduced a and increased δ. Aripiprazole (3,30 mg kg⁻¹) increased δ but did not affect a. Amisulpride (5, 50 mg kg⁻¹) had a delayed and protracted effect: δ was increased 3-6 hours after treatment; a was increased 1.5 hours, and reduced 12-24 hours after treatment. Interpretation based on Killeen's model suggests that aripiprazole does not share clozapine's ability to enhance reinforcer value. Amisulpride produced a short-lived enhancement, followed by a long-lasting reduction, of reinforcer value. Both drugs impaired motor performance.

A theory of behaviour on progressive ratio schedules, with applications in behavioural pharmacology

RATIONALE

Mathematical principles of reinforcement (MPR) provide the theoretical basis for a family of models of schedule-controlled behaviour. A model of fixed-ratio schedule performance that was applied to behaviour on progressive ratio (PR) schedules showed systematic departures from the data.

OBJECTIVE

This study aims to derive a new model from MPR that will account for overall and running response rates in the component ratios of PR schedules, and their decline toward 0, the breakpoint.

METHOD

The role of pausing is represented in a real-time model containing four parameters: T (0) and k are the intercept and slope of the linear relation between post-reinforcement pause duration and the prior inter-reinforcer interval; a (specific activation) measures the incentive value of the reinforcer; δ (response time) sets biomechanical limits on response rate. Running rate is predicted to decrease with negative acceleration as ratio size increments, overall rate to increase and then decrease. Differences due to type of progression are explained as hysteresis in the control by reinforcement rates. Re-analysis of extant data focuses on the effects of acute treatment with antipsychotic drugs, lesions of the nucleus accumbens core, and destruction of orexinergic neurones of the lateral hypothalamus.

RESULTS

The new model resolves some anomalies evident in earlier analyses, and provides new insights to the results of these interventions.

CONCLUSIONS

Because they can render biologically relevant parameters, mathematical models can provide greater power in interpreting the effects of interventions on the processes underlying schedule-controlled behaviour than is possible for first-order data such as the breakpoint.

A clozapine-like effect of cyproheptadine on progressive ratio schedule performance

The atypical antipsychotic drug clozapine has multiple pharmacological actions, some of which, including 5-hydroxytryptamine (5-HT₂) and histamine (H₁) receptor antagonist effects, are shared by the non-selective 5-HT receptor antagonist cyproheptadine. Atypical antipsychotics have a characteristic profile of action on operant behaviour maintained by progressive ratio schedules, as revealed by Killeen's (1994) mathematical model of schedule controlled behaviour. These drugs increase the values of a parameter that expresses the 'incentive value' of the reinforcer (a) and a parameter that is inversely related to the 'motor capacity' of the organism (δ). This experiment examined the effects of acute treatment with cyproheptadine and clozapine on performance on a progressive ratio schedule of food reinforcement in rats; the effects of a conventional antipsychotic, haloperidol, and two drugs with food intake-enhancing effects, chlordiazepoxide and Δ⁹-tetrahydrocannabinol (THC), were also examined. Cyproheptadine (1, 5 mg kg⁻¹) and clozapine (3.75, 7.5 mg kg⁻¹) increased a and δ. Haloperidol (0.05, 0.1 mg kg⁻¹) reduced a and increased δ. Chlordiazepoxide (3, 10 mg kg⁻¹) increased a but reduced δ. THC (1, 3 mg kg⁻¹) had no effect. Interpretation based on Killeen's (1994) model suggests that cyproheptadine and clozapine enhanced the incentive value of the reinforcer and impaired motor performance. Motor impairment may be due to sedation (possibly reflecting H₁ receptor blockade). Enhancement of incentive value may reflect simultaneous blockade of H₁ and 5-HT₂ receptors, which has been proposed as the mechanism underlying the food intake-enhancing effect of cyproheptadine. In agreement with previous findings, haloperidol impaired motor performance and reduced the incentive value of the reinforcer. Chlordiazepoxide's effect on a is consistent with its food intake-enhancing effect.

Effect of orexin-B-saporin-induced lesions of the lateral hypothalamus on performance on a progressive ratio schedule

It has been suggested that a sub-population of orexinergic neurones whose somata lie in the lateral hypothalamic area (LHA) play an important role in regulating the reinforcing value of both food and drugs. This experiment examined the effect of disruption of orexinergic mechanisms in the LHA on performance on the progressive ratio schedule of reinforcement, in which the response requirement increases progressively for successive reinforcers. The data were analysed using a mathematical model which yields a quantitative index of reinforcer value and dissociates effects of interventions on motor and motivational processes. Rats were trained under a progressive ratio schedule using food-pellet reinforcement. They received bilateral injections of conjugated orexin-B-saporin (OxSap) into the LHA or sham lesions. Training continued for a further 40 sessions after surgery. Equations were fitted to the response rate data from each rat, and the parameters of the model were derived for successive blocks of 10 sessions. The OxSap lesion reduced the number of orexin-containing neurones in the LHA by approximately 50% compared with the sham-lesioned group. The parameter expressing the incentive value of the reinforcer was not significantly altered by the lesion. However, the parameter related to the maximum response rate was significantly affected, suggesting that motor capacity was diminished in the OxSap-lesioned group. The results indicate that OxSap lesions of the LHA disrupted food-reinforced responding on the progressive ratio schedule. It is suggested that this disruption was brought about by a change in non-motivational (motor) processes.

Diet-induced obesity blunts the behavioural effects of ghrelin: studies in a mouse-progressive ratio task

RATIONAL

The ghrelinergic system is implicated in the development of obesity and in modulating central reward systems. It has been reported that diet-induced obesity causes blunted responding on food intake to ghrelin administration, associated with central ghrelin resistance. Here we investigate whether the stimulatory effects of ghrelin on the reward system are altered in diet-induced obese mice.

METHODS

Obesity was induced in C57BL/6J mice by feeding high-fat diet for 13 weeks. Mice were trained in an operant fixed and exponential progressive ratio task to respond for sucrose rewards. In an ad libitum fed state, ghrelin and a ghrelin receptor antagonist were administered in the progressive ratio. Alterations in the central ghrelin system in diet-induced obese mice were assessed.

RESULTS

Obese mice showed attenuated acquisition and performance in the fixed and progressive ratio paradigm. Most importantly, diet-induced obesity inhibited the stimulatory effects of ghrelin (2 nmol, 3 nmol/10 g) on progressive ratio responding whereas lean animals presented with increased responding. Administration of the ghrelin-receptor antagonist (D-Lys(3))-GHRP-6 (66.6 nmol/10 g) decreased performance in lean but not obese mice. This insensitivity to ghrelin receptor ligands in mice on high-fat diet was further supported by decreased mRNA expression of the ghrelin receptor in the hypothalamus and the nucleus accumbens in obese mice.

CONCLUSIONS

This study demonstrates that the modulatory effects of ghrelin receptor ligands are blunted in a mouse model of diet-induced obesity in a progressive ratio task. Thereby, our data extend the previously described ghrelin resistance in these mice from food intake to reward-associated behaviours.

Nucleus accumbens and delay discounting in rats: evidence from a new quantitative protocol for analysing inter-temporal choice

RATIONALE

There is evidence that the core of the nucleus accumbens (AcbC) is involved in inter-temporal choice behaviour.

OBJECTIVE

A new behavioural protocol was used to examine the effect of destruction of the AcbC on delay discounting in inter-temporal choice schedules in rats.

METHOD

Rats with excitotoxic lesions of the AcbC or sham lesions made repeated choices on an adjusting-delay schedule between a smaller reinforcer (A) that was delivered immediately and a larger reinforcer (B) that was delivered after a delay which increased or decreased depending on the subject's choices. In two phases of the experiment, reinforcer sizes were selected which enabled theoretical parameters expressing delay discounting and sensitivity to reinforcer size to be estimated from the ratio of the indifference delays (i.e. the quasi-stable values of the adjusting delay seen after extended training) obtained in the two phases.

RESULTS

In both groups, indifference delays were shorter when the sizes of A and B were 14 and 25 μl than when they were 25 and 100 μl of a 0.6 M sucrose solution. Indifference delays were shorter in AcbC-lesioned than in sham-lesioned rats. Estimates of delay discounting rate based on the ratio of the indifference delays were lower in the AcbC-lesioned than in the sham-lesioned rats. The size sensitivity parameter did not differ between the groups. Adjusting delays in successive blocks of trials were analysed using Fourier transform. The period corresponding to the dominant frequency of the power spectrum and power within the dominant frequency band did not differ between the groups.

CONCLUSIONS

Destruction of the AcbC increased the rate of delay discounting.

Seeking-taking chain schedules of cocaine and sucrose self-administration: effects of reward size, reward omission, and α-flupenthixol

RATIONALE

In heterogeneous seeking-taking (ST) chain schedules of self-administration, seeking rewards and taking rewards are distinct actions, giving animals explicit control over their intake of the reward. However, the neurobehavioral characteristics of ST chain schedules are relatively unexplored.

OBJECTIVES

This study was made to evaluate two variants of ST chain schedules of self-administration to measure seeking and taking of sucrose and cocaine in rats.

METHODS

Rats had to respond on one lever (seeking lever) under a random interval (RI) or under a progressive ratio (PR) schedule, to gain access to a second lever (taking lever), responding on which under a fixed-ratio 1 (FR-1) schedule of reinforcement delivered the reward. We assessed the effects of reward size, reward omission, and administration of the dopamine receptor antagonist α-flupenthixol. The effects of α-flupenthixol on responding for cocaine or sucrose under an FR-1 schedule of reinforcement were also assessed.

RESULTS

Cocaine seeking under both schedules was reduced by decreasing reward size, reward omission, and α-flupenthixol treatment. Cocaine taking was decreased by α-flupenthixol treatment and reward omission, but not by altering reward size. Sucrose seeking was not affected by reward size, but was reduced by α-flupenthixol and reward omission. Sucrose taking was diminished by reward omission only. α-Flupenthixol increased cocaine but not sucrose intake under an FR-1 schedule of reinforcement.

CONCLUSIONS

Both ST(PR) and ST(RI) schedules can be used to assess seeking and taking of sucrose and cocaine. Dopaminergic neurotransmission mediates the positive subjective properties of cocaine but not sucrose and the motivational properties of both sucrose and cocaine.

Transitional and steady-state choice behavior under an adjusting-delay schedule

Twelve rats made repeated choices on an adjusting-delay schedule between a smaller reinforcer (A) that was delivered immediately after a response and a larger reinforcer (B) that was delivered after a delay which increased or decreased by 20% depending on the subject's choices in successive blocks of trials. In two phases of the experiment (100 sessions and 40 sessions), reinforcer sizes were selected which enabled theoretical parameters expressing the rate of delay discounting and sensitivity to reinforcer size to be estimated from the ratio of the indifference delays obtained in the two phases. Indifference delays, calculated from adjusting delays in the last 10 sessions of each phase, were shorter when the sizes of A and B were 14 and 25 µl of a 0.6 M sucrose solution than when they were 25 and 100 µl of the same solution. The ratio of the indifference delays was significantly smaller than that predicted on the basis of an assumed linear relation between reinforcer size and instantaneous reinforcer value, consistent with a previous proposal that this relation may be hyperbolic in form. Estimates of the rate of delay discounting based on the ratio of the two indifference delays (mean, 0.08 s(-1)) were similar to values obtained previously using different intertemporal choice protocols. Estimates of the size-sensitivity parameter (mean 113 µl) were similar to estimates recently derived from performance on progressive-ratio schedules. In both phases of the experiment, adjusting delays in successive blocks of trials were analyzed using the Fourier transform. The power spectrum obtained from individual rats had a dominant frequency that corresponded to a period of oscillation of the adjusting delay between 30 and 100 trial blocks (mean, 78). Power in the dominant frequency band was highest in the early sessions of the first phase and declined with extended training. It is suggested that this experimental protocol may have utility in neurobehavioral studies of intertemporal choice.

Choice between reinforcer delays versus choice between reinforcer magnitudes: differential Fos expression in the orbital prefrontal cortex and nucleus accumbens core

Lesions of the orbital prefrontal cortex (OPFC) and the nucleus accumbens core (AcbC) can disrupt performance in inter-temporal choice tasks, possibly by increasing the organism's sensitivity to delay and/or magnitude of reinforcement. This experiment examined whether exposure to an inter-temporal choice would induce neuronal activation in these areas, as indicated by enhanced expression of the Fos protein. Twelve rats were trained to press levers A and B under an adjusting-delay schedule in which a response on A delivered 50 microl of a sucrose reinforcer after 2 or 18s, whereas a response on B delivered the same reinforcer after a delay that was adjusted in accordance with the rat's choices. Another 12 rats were trained under a similar schedule in which a response on A delivered an immediate reinforcer of size 20 or 180 microl, whereas a response on B delivered an immediate reinforcer whose size was adjusted in accordance with the rat's choices. A third group received training under a schedule that did not entail variation of reinforcer size or delay, or choice between reinforcers, and a control group underwent food restriction without behavioural training. Exposure to the adjusting-delay schedule was associated with enhanced Fos expression in both the OPFC and AcbC, whereas exposure to the adjusting-magnitude schedule was associated with enhanced Fos expression in the OPFC but not the AcbC, compared to the control group. The results are consistent with previous findings that implicated the AcbC and OPFC in delay discounting, and the OPFC in sensitivity to reinforcer size.

Evidence for motivational effects elicited by activation of GABA-A or dopamine receptors in the nucleus accumbens shell

Microinjections of the inhibitory GABA-A receptor agonist muscimol into the shell region of the nucleus accumbens (AcbSh) have been reported to induce large increases in food intake, but the effect of these injections on motivational processes is less clear. In the current study, bilateral injections of saline, muscimol (50ng/side) or d-amphetamine (10mug/side) were made into the AcbSh of rats trained to lever press on a progressive ratio schedule for food reward. Injections of both muscimol and amphetamine were found to produce a large increase in the breaking point relative to saline injections. This result suggests that inactivation of the AcbSh does not simply drive ingestive behavior, but also affects motivational processes assessed by the progressive ratio schedule. Breaking points were also increased by injections of amphetamine into the AcbSh.