Unit price and choice in a token-reinforcement context

Substitution effects in a generalized token economy with pigeons

Pigeons made repeated choices between earning and exchanging reinforcer-specific tokens (green tokens exchangeable for food, red tokens exchangeable for water) and reinforcer-general tokens (white tokens exchangeable for food or water) in a closed token economy. Food and green food tokens could be earned on one panel; water and red water tokens could be earned on a second panel; white generalized tokens could be earned on either panel. Responses on one key produced tokens according to a fixed-ratio schedule, whereas responses on a second key produced exchange periods, during which all previously earned tokens could be exchanged for the appropriate commodity. Most conditions were conducted in a closed economy, and pigeons distributed their token allocation in ways that permitted food and water consumption. When the price of all tokens was equal and low, most pigeons preferred the generalized tokens. When token-production prices were manipulated, pigeons reduced production of the tokens that increased in price while increasing production of the generalized tokens that remained at a fixed price. The latter is consistent with a substitution effect: Generalized tokens increased and were exchanged for the more expensive reinforcer. When food and water were made freely available outside the session, token production and exchange was sharply reduced but was not eliminated, even in conditions when it no longer produced tokens. The results join with other recent data in showing sustained generalized functions of token reinforcers, and demonstrate the utility of token-economic methods for assessing demand for and substitution among multiple commodities in a laboratory context.

Characterizing the relationship between increases in the cost of nicotine and decreases in nicotine content in adult male rats: implications for tobacco regulation

BACKGROUND

A large reduction in the nicotine content of cigarettes may benefit public health by reducing the rate and the prevalence of smoking. A behavioral economics framework suggests that a decrease in nicotine content may be considered an increase in the unit price of nicotine (unit price = reinforcer cost/reinforcer magnitude). Increasing the price of cigarettes (i.e., increasing reinforcer cost) would be considered an equivalent change in unit price to reducing nicotine content (i.e., reducing reinforcer magnitude).

OBJECTIVES

The goal of the present experiments was to characterize the relationship between increases in nicotine cost and decreases in nicotine dose.

MATERIALS AND METHODS

A rat self-administration model was used to assess this relationship across three experiments, with an emphasis on very low nicotine doses to model a potential nicotine reduction policy. Cost was manipulated via changes in the number of responses required to earn an infusion.

RESULTS

Results show that increases in the cost of nicotine and decreases in nicotine content were not equivalent manipulations. Nicotine consumption was more sensitive to nicotine dose than to nicotine cost. Nicotine consumption was also not equivalent across a variety of cost and dose combinations forming a single unit price.

CONCLUSIONS

Results of the present studies suggest that nicotine reduction is likely to have a large impact on nicotine exposure from cigarettes.

The several roles of stimuli in token reinforcement

Three experiments were conducted with pigeons to identify the stimulus functions of tokens in second-order token-reinforcement schedules. All experiments employed two-component multiple schedules with a token-reinforcement schedule in one component and a schedule with equivalent response requirements and/or reinforcer density in the other. In Experiment 1, response rates were lower under a token-reinforcement schedule than under a tandem schedule with the same response requirements, suggesting a discriminative role for the tokens. In Experiment 2, response rates varied systematically with signaling functions of the tokens in a series of conditions designed to explore other aspects of the temporal-correlative relations between tokens and food. In Experiment 3, response rates were reduced but not eliminated by presenting tokens independent of responding, yoked to their temporal occurrence in a preceding token component, suggesting both a reinforcing function and eliciting/evocative functions based on stimulus-food relations. Only when tokens were removed entirely was responding eliminated. On the whole, the results suggest that tokens, as stimuli temporally correlated with food, may serve multiple stimulus functions in token-reinforcement procedures--reinforcing, discriminative, or eliciting--depending on the precise arrangement of the contingencies in which they are embedded.

Rats' choices with token stimuli in concurrent variable-interval schedules

Four rats responded on concurrent variable-interval schedules that delivered token stimuli (stimulus lights arranged vertically above each of two side levers). During exchange periods, each token could be exchanged for one food pellet by responding on a center lever, with one response required for each pellet delivery. In different conditions, the exchange requirements (number of tokens that had to be earned before they could be exchanged for food) varied between one and four for the two response levers. The experiments were closely patterned after research with pigeons by Mazur and Biondi (2013), and the results from the rats in the present experiment were similar. Response percentages on the two levers changed as each additional token was earned, and these patterns indicated that choice was controlled by both the time to the exchange periods and the number of food pellets that were delivered in the exchange period. In some conditions, the exchange requirement was three tokens for each lever, but the token lights were not turned on as they were earned for one of the two keys. The rats showed a slight preference for the lever without the token lights, which may indicate that the token lights were not serving as conditioned reinforcers (a result also found by Mazur and Biondi with pigeons). Overall, these results suggest that, in this choice procedure, the token stimuli served primarily as discriminative stimuli that signaled the temporal proximity and quantity of the primary reinforcer, food.

Generalized conditioned reinforcement with pigeons in a token economy

Six pigeons were studied in a token economy in which tokens could be produced and exchanged for food on one side of an experimental chamber and for water on the opposite side of the chamber. Responses on one key produced tokens according to a token-production fixed ratio (FR) schedule. Responses on a second key produced an exchange period during which tokens were exchanged for water or food. In Experiment 1a, food tokens could be earned and exchanged under restricted food budgets, and water tokens could be earned and exchanged under water restricted budgets. In Experiment 1b, a third (generalized) token type could be earned and exchanged for either food or water under water restricted budgets. Across Experiments 1a and 1b, the number of tokens accumulated prior to exchange increased as the exchange-production schedule was increased. In Experiment 1b, pigeons produced more generalized than specific tokens, suggesting enhanced reinforcing efficacy of generalized tokens. In Experiment 2, the FR token-production price was manipulated under water restriction and then under food restriction. Production of each token type generally declined as a function of its own price and increased as a function of the price of the alternate type, demonstrating own-price and cross-price elasticity. Production of food and water tokens often changed together, indicating complementarity. Production of specific and generalized tokens changed in opposite directions, indicating substitutability. This is the first demonstration of sustained generalized functions of tokens in nonhumans, and illustrates a promising method for exploring economic contingencies in a controlled environment.

Risky choice in pigeons: preference for amount variability using a token-reinforcement system

Pigeons were given repeated choices between variable and fixed numbers of token reinforcers (stimulus lamps arrayed above the response keys), with each earned token exchangeable for food. The number of tokens provided by the fixed-amount option remained constant within blocks of sessions, but varied parametrically across phases, assuming values of 2, 4, 6, or 8 tokens per choice. The number of tokens provided by the variable-amount option varied between 0 and 12 tokens per choice, arranged according to an exponential or rectangular distribution. In general, the pigeons strongly preferred the variable option when the fixed option provided equal or greater numbers of tokens than the variable amount. Preference for the variable amount decreased only when the alternatives provided widely disparate amounts favoring the fixed amount. When tokens were removed from the experimental context, preference for the variable option was reduced or eliminated, suggesting that the token presentation played a key role in maintaining risk-prone choice patterns. Choice latencies varied inversely with preferences, suggesting that local analyses may provide useful ancillary measures of reinforcer value. Overall, the results indicate that systematic risk sensitivity can be attained with respect to reinforcer amount, and that tokens may be critical in the development of such preferences.

Pigeons' choices with token stimuli in concurrent variable-interval schedules

Twelve pigeons responded on concurrent variable-interval schedules that delivered token stimuli (stimulus lights for some pigeons, and white circles on the response keys for others). During exchange periods, each token could be exchanged for food on a fixed-ratio 1 schedule. Across conditions, the exchange requirements (number of tokens that had to be earned before they could be exchanged for food) varied between one and four for the two response keys. The main findings were that the pigeons' response percentages varied as a function of the number of tokens earned at any given moment, and they were determined by both the delays to food and by the number of food deliveries in the exchange periods. In some conditions, tokens had to be earned but were not visible during the variable-interval schedules for one or both keys. When one key had visible tokens and the other did not, the pigeons showed a preference for the key without visible tokens. A model based on the matching law and a hyperbolic delay-discounting equation could account for the main patterns of choice responding, and for how response percentages changed as successive tokens were earned. The results are consistent with the view that the token stimuli served as discriminative stimuli that signaled the current delays to food.

Behavioural correlate of choice confidence in a discrete trial paradigm

How animals make choices in a changing and often uncertain environment is a central theme in the behavioural sciences. There is a substantial literature on how animals make choices in various experimental paradigms but less is known about the way they assess a choice after it has been made in terms of the expected outcome. Here, we used a discrete trial paradigm to characterise how the reward history shaped the behaviour on a trial by trial basis. Rats initiated each trial which consisted of a choice between two drinking spouts that differed in their probability of delivering a sucrose solution. Critically, sucrose was delivered after a delay from the first lick at the spouts--this allowed us to characterise the behavioural profile during the window between the time of choice and its outcome. Rats' behaviour converged to optimum choice, both during the acquisition phase and after the reversal of contingencies. We monitored the post-choice behaviour at a temporal precision of 1 millisecond; lick-response profiles revealed that rats spent more time at the spout with the higher reward probability and exhibited a sparser lick pattern. This was the case when we exclusively examined the unrewarded trials, where the outcome was identical. The differential licking profiles preceded the differential choice ratios and could thus predict the changes in choice behaviour.

Assessing the value of choice in a token system

Responding of 4 children was assessed under conditions in which (a) no programmed contingencies were arranged for target behavior, (b) responding produced tokens that could be exchanged for a single highly preferred edible item, and (c) responding produced a token that could be exchanged for a variety of preferred edible items. After assessing the effects of these contingencies, the preferences of 3 participants were assessed using a concurrent-chains schedule. Preference for the opportunity to choose from the same or qualitatively different edible items varied across participants, and findings were generally consistent with those of Tiger, Hanley, and Hernandez (2006).

Distributed versus exclusive preference in discrete-trial choice

Two experiments on discrete-trial choice examined the conditions under which pigeons would exhibit exclusive preference for the better of two alternatives as opposed to distributed preference (making some choices for each alternative). In Experiment 1, pigeons chose between red and green response keys that delivered food after delays of different durations, and in Experiment 2 they chose between red and green keys that delivered food with different probabilities. Some conditions of Experiment 1 had fixed delays to food and other conditions had variable delays. In both experiments, exclusive or nearly exclusive preference for the better alternative was found in some conditions, but distributed preference was found in other conditions, especially in Experiment 2 when key location varied randomly over trials. The results were used to evaluate several different theories about discrete-trial choice. The results suggest that exclusive preference for one alternative is a frequent outcome in discrete-trial choice. When distributed preference does occur, it is not the result of inherent tendencies to sample alternatives or to match response percentages to the values of the alternatives. Rather, distributed preference may occur when two factors (such as reinforcer delay and position bias) compete for the control of choice, or when the consequences for the two alternatives are similar and difficult to discriminate.

Consumption and response output as a function of unit price: manipulation of cost and benefit components

We conducted preference assessments with 4 typically developing children to identify potential reinforcers and assessed the reinforcing efficacy of those stimuli. Next, we tested two predictions of economic theory: that overall consumption (reinforcers obtained) would decrease as the unit price (response requirement per reinforcer) increased and that the cost and benefit components that defined unit price would not influence overall consumption considerably when unit price values were equal. We tested these predictions by arranging unit price such that the denominator was one (e.g., two responses produced one reinforcer) or two (e.g., four responses produced two reinforcers). Results showed that consumption decreased as unit price increased and that unit price values with different components produced similar consumption.

Token reinforcement: a review and analysis

Token reinforcement procedures and concepts are reviewed and discussed in relation to general principles of behavior. The paper is divided into four main parts. Part I reviews and discusses previous research on token systems in relation to common behavioral functions--reinforcement, temporal organization, antecedent stimulus functions, and aversive control--emphasizing both the continuities with other contingencies and the distinctive features of token systems. Part II describes the role of token procedures in the symmetrical law of effect, the view that reinforcers (gains) and punishers (losses) can be measured in conceptually analogous terms. Part III considers the utility of token reinforcement procedures in cross-species analysis of behavior more generally, showing how token procedures can be used to bridge the methodological gulf separating research with humans from that with other animals. Part IV discusses the relevance of token systems to the field of behavioral economics. Token systems have the potential to significantly advance research and theory in behavioral economics, permitting both a more refined analysis of the costs and benefits underlying standard economic models, and a common currency more akin to human monetary systems. Some implications for applied research and for broader theoretical integration across disciplines will also be considered.

Reinforcer accumulation in a token-reinforcement context with pigeons

Four pigeons were exposed to a token-reinforcement procedure with stimulus lights serving as tokens. Responses on one key (the token-production key) produced tokens that could be exchanged for food during an exchange period. Exchange periods could be produced by satisfying a ratio requirement on a second key (the exchange-production key). The exchange-production key was available any time after one token had been produced, permitting up to 12 tokens to accumulate prior to exchange. Token accumulation, measured in terms of both frequency (percent cycles with accumulation) and magnitude (mean number of tokens accumulated), decreased as the token-production ratio increased from 1 to 10 across conditions (with exchange-production ratio held constant), and increased as the exchange-production ratio increased from 1 to 250 across conditions (with token-production ratio held constant). When tokens were removed, accumulation decreased markedly compared to conditions with tokens and the same schedules. These data show that token accumulation is an orderly function of token-production and exchange-production schedules, and they are broadly consistent with a unit-price model based on local and global responses per reinforcer.

A unit price evaluation of severe problem behavior

We evaluated problem behavior exhibited by 6 individuals with developmental disabilities using the behavioral economic conceptualization of unit price. Descriptive observations were conducted during interactions between the participants and their primary care providers in a clinical laboratory, the participants' homes, or school. Data were recorded on potential reinforcers and problem behavior. After identifying reinforcers for each participant's problem behavior by way of functional analysis, the descriptive data were analyzed retrospectively, using the cost-benefit ratio described by the concept of unit price. Results showed that demand was elastic and produced monotonic response output curves. The results represent an extension of the concept of unit price to severe problem behavior under naturally occurring environmental conditions.

Second-order schedules of token reinforcement with pigeons: implications for unit price

Four pigeons were exposed to second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were earned according to a fixed-ratio (token-production) schedule, with the opportunity to exchange tokens for food (exchange period) occurring after a fixed number had been produced (exchange-production ratio). The token-production and exchange-production ratios were manipulated systematically across conditions. Response rates varied inversely with the token-production ratio at each exchange-production ratio. Response rates also varied inversely with the exchange-production ratio at each token-production ratio, particularly at the higher token-production ratios. At higher token-production and exchange-production ratios, response rates increased in token-production segments closer to exchange periods and food. Some conditions were conducted in a closed economy, in which the pigeons earned all their daily ration of food within the session. Relative to comparable open-economy conditions, response rates in the closed economy were less affected by changes in token-production ratio, resulting in higher levels of food intake and body weight. Some of the results are consistent with the economic concept of unit price, a cost-benefit ratio comprised of responses per unit of food delivery, but most are well accounted for by a consideration of the number of responses required to produce exchange periods, without regard to the amount of reinforcement available during those exchange periods.

Assessing unit-price related remifentanil choice in rhesus monkeys

Given a commodity available at different prices, a unit-price account of choice predicts preference for the cheaper alternative. This experiment determined if rhesus monkeys preferred remifentanil (an ultra-short-acting micro-opioid agonist) delivered at a lower unit price over a higher-priced remifentanil alternative (Phases 1 and 3). Choice between equal-priced alternatives also was assessed (Phase 2). A discrete-trials procedure was arranged in which three monkeys chose between two remifentanil alternatives by responding on one of two levers. Different prices were arranged by manipulating drug dose (0.3 and 0.1 microg/kg/injection) and/or the ratio requirement. Monkeys usually chose the larger-dose alternative even when it was more expensive. Only when unit prices were relatively high (e.g., large response requirements) did monkeys choose the cheaper (or equally priced) smaller-dose alternative. Employing larger doses (0.9 and 0.3 microg/kg/injection) attenuated the larger-dose preference. The results demonstrate that choice was not determined simply by unit price. An alternative model that employs demand-function analysis to generate choice predictions is proposed.

A steady-state test of the demand curve analysis of relative reinforcer efficacy

The demand curve analysis of relative reinforcer efficacy predicts that preference for 1 reinforcer over another may be predicted from relative levels of consumption under single-schedule conditions. Prior experiments with human cigarette smokers have largely confirmed this prediction, but because subjects completed only 1 session under each arrangement, questions remain about the stability of single and concurrent-schedule (choice) performances. In addition, no prior tests of this prediction of the demand curve analysis have used nonhuman subjects, so questions remain about the effects of instructions and demand characteristics. In the present study, the authors examined pigeons' steady-state preferences in a closed economy across a wide range of concurrent fixed-ratio and random-ratio (RR) schedules. On the basis of the pigeons' single-schedule consumption, the demand curve analysis qualitatively predicted indifference at low ratio values and preference for the RR alternative at higher ratio values. These predictions proved correct in 88% of the cases in which the analysis predicted preferences in individual subjects at specific concurrent-schedule values. These findings offer preliminary support for the position that the predictions of the demand curve analysis hold true when steady-state performances are assessed. They also offer preliminary evidence that previous confirmations of the demand curve analysis were not due to the influence of instructions or to demand characteristics of the human laboratory setting. These findings supplement prior questioning of the coherence and utility of traditional measures of relative reinforcer efficacy.

Morphine tolerance as a function of ratio schedule: response requirement or unit price?

Key pecking by 3 pigeons was maintained by a multiple fixed-ratio 10, fixed-ratio 30, fixed-ratio 90 schedule of food presentation. Components differed with respect to amount of reinforcement, such that the unit price was 10 responses per 1-s access to food. Acute administration of morphine, l-methadone, and cocaine dose-dependently decreased overall response rates in each of the components. When a rate decreasing dose of morphine was administered daily, tolerance, as measured by an increase in the dose that reduced response rates to 50% of control (i.e., the ED50 value), developed in each of the components; however, the degree of tolerance was smallest in the fixed-ratio 90 component (i.e., the ED50 value increased the least). When the l-methadone dose-effect curve was redetermined during the chronic morphine phase, the degree of cross-tolerance conferred to l-methadone was similar across components, suggesting that behavioral variables may not influence the degree of cross-tolerance between opioids. During the chronic phase, the cocaine dose-effect curve shifted to the right for 2 pigeons and to the left for 1 pigeon, which is consistent with predictions based on the lack of pharmacological similarity between morphine and cocaine. When the morphine, l-methadone, and cocaine dose-effect curves were redetermined after chronic morphine administration ended, the morphine and l-methadone ED50s replicated those obtained prior to chronic morphine administration. The morphine data suggest that the fixed-ratio value (i.e., the absolute output) determines the degree of tolerance and not the unit price.

Labor supply and consumption of food in a closed economy under a range of fixed- and random-ratio schedules: tests of unit price

The behavioral economic concept of unit price predicts that consumption and response output (labor supply) are determined by the unit price at which a good is available regardless of the value of the cost and benefit components of the unit price ratio. Experiment 1 assessed 4 pigeons' consumption and response output at a range of unit prices. In one condition, food was available according to a range of fixed-ratio schedules, whereas in the other condition, food was available according to a range of random-ratio schedules. Consistent with unit price predictions, consumption and response output were approximately equivalent across schedule types within the lower range of unit prices. However, at Unit Prices 64 (ratio value = 192) and greater, considerably more consumption and response output were observed in the random-ratio condition. Experiment 2 replicated these findings with 4 pigeons using the rapid demand curve assay procedure that is commonly used in the behavioral economics literature. Findings are integrated with two mathematical models of behavior under variable reinforcer delays.