The arithmetic of discounting

Genome-wide association study of delay discounting in Heterogeneous Stock rats

Delay discounting refers to the behavioral tendency to devalue rewards as a function of their delay in receipt. Heightened delay discounting has been associated with substance use disorders and multiple co-occurring psychopathologies. Human and animal genetic studies have established that delay discounting is heritable, but only a few associated genes have been identified. We aimed to identify novel genetic loci associated with delay discounting through a genome-wide association study (GWAS) using Heterogeneous Stock (HS) rats, a genetically diverse outbred population derived from eight inbred founder strains. We assessed delay discounting in 650 male and female HS rats using an adjusting amount procedure in which rats chose between smaller immediate sucrose rewards or a larger reward at various delays. Preference switch points were calculated and both exponential and hyperbolic functions were fitted to these indifference points. Area under the curve (AUC) and the discounting parameter k of both functions were used as delay discounting measures. GWAS for AUC, exponential k, and one indifference point identified significant loci on chromosomes 20 and 14. The gene Slc35f1, which encodes a member of the solute carrier family, was the sole gene within the chromosome 20 locus. That locus also contained an eQTL for Slc35f1, suggesting that heritable differences in the expression might be responsible for the association with behavior. Adgrl3, which encodes a latrophilin subfamily G-protein coupled receptor, was the sole gene within the chromosome 14 locus. These findings implicate novel genes in delay discounting and highlight the need for further exploration.

Hyperbolae

Hyperbolic relations between independent and dependent variables are ubiquitous in the experimental analysis of behavior, mentioned in over 150 articles in the Journal of the Experimental Analysis of Behavior. There are two principal forms of hyperbolae: The first describes the relation between response rate and reinforcement rate on variable-interval schedules of reinforcement; it rises asymptotically toward a maximum. The second describes the relation between the current equivalent value of an incentive and its delay or (im)probability; it falls from a maximum toward an asymptote of 0. Where do these come from? What do their parameters mean? How are they related? This article answers the first two questions and addresses the last.

Using behavioral economics to optimize safer undergraduate late-night transportation

Many universities sponsor student-oriented transit services that could reduce alcohol-induced risks but only if services adequately anticipate and adapt to student needs. Human choice data offer an optimal foundation for planning and executing late-night transit services. In this simulated choice experiment, respondents opted to either (a) wait an escalating delay for a free university-sponsored "safe" option, (b) pay an escalating fee for an on-demand rideshare service, or (c) pick a free, immediately available "unsafe" option (e.g., ride with an alcohol-impaired driver). Behavioral-economic nonlinear models of averaged-choice data describe preference across arrangements. Best-fit metrics indicate adequate sensitivity to contextual factors (i.e., wait time, preceding late-night activity). At short delays, students preferred the free transit option. As delays extend beyond 30 min, most students preferred competing alternatives. These data depict a policy-relevant delay threshold to better safeguard undergraduate student safety.

Prisoner's dilemma and the free operant: John Nash, I'd like you to meet Fred Skinner

In separate chambers, responding by two pairs of pigeons was reinforced under concurrent random-ratio schedules of reinforcement. For each pair, the birds' schedules were coupled in such a manner that left- and right-key reinforcement probabilities were determined by the key being pecked by the other pigeon of the pair. In this way, a reinforcement matrix, like that of the popular Prisoner's Dilemma game of game theory, was created. The responding of all subjects soon gravitated to the choice combination identified by the mathematician John Nash as the equilibrium of the Prisoner's Dilemma game. This was found both before and after reversal of contingencies on the keys. In a second experiment, with a single pair of pigeons, stimuli signaling the choice of the paired pigeon had little lasting effect: responding again gravitated to the game's equilibrium. The results affirm earlier findings, demonstrating that Skinner's principle of positive reinforcement, together with Nashian mathematics, entirely accounts for iterative game-theoretic behavior. They extend these findings to the so-called free operant: to schedules of reinforcement in which responding is not constrained by stimulus-response sequencing (i.e., a trials procedure). The coupled schedule of reinforcement introduced here offers significant promise for the experimental analysis of economic and social behaviors.

Time Cost and Demand: Implications for Public Policy

The success of policy involves not only good design but a good understanding of how the public will respond behaviorally to the benefits or detriments of that policy. Behavioral science has greatly contributed to how we understand the impact of monetary costs on behavior and has therefore contributed to policy design. Consumption taxes are a direct result of this; for example, cigarette taxes that aim to reduce cigarette consumption. In addition to monetary costs, time may also be conceptualized as a constraint on consumption. Time costs may therefore have policy implications, for example, long waiting times could deter people from accessing certain benefits. Recent data show that behavioral economic demand curve methods used to understand monetary cost may also be used to understand time costs. In this article we discuss how the impact of time cost can be conceptualized as a constraint on demand for public benefits utilization and public health when there are delays to receiving the benefits. Policy examples in which time costs may be relevant and demand curve methods may be useful are discussed in the areas of government benefits, public health, and transportation design.

Bidding for delayed rewards: Accumulation as delay discounting, delay discounting as regulation, demand functions as corollary

In the accumulation paradigm animals press one manipulandum to accumulate pellets or seconds of access to food, and then press another manipulandum, or run some distance, to collect it. The accumulation may be interpreted as delay discounting, with the animals adjusting the distal amount to compensate for its distance or delay. The amount accumulated before being collected is a linear function of the distance or time that the experimental paradigm stipulates for collecting it. That linear function follows from the Unit-Price/Unit-Amount axiom. The inverse of the linear compensation functions gives a delay-discount hyperbola. The advantages of the accumulation paradigm and analytic framework for delay discounting studies are noted. Compensation functions are then derived from a behavioral regulation model, which generalizes them to contexts where the individual's budget for response cost becomes over-taxed. In turn, such compensation/regulation models lead directly to representative demand functions. In sum, regulation models provide a theoretical grounding for demand functions, compensation functions, and delay discounting hyperbolas. The parameter that links them is the unit amount k, the slope of the compensation function and of the discount function, the setpoint for consumption in a regulation model, and the ideal quantity consumed at minimal price in a demand analysis.

In search of a definition of reinforcer value: Some successes and failures of the multiplicative hyperbolic model

The concept of 'value' has enjoyed a central position in many theoretical accounts of choice behaviour. Several definitions of 'value' are contrasted in this paper, and one particular approach is defended, whereby value is defined as a dimensionless intervening variable. This definition is a cornerstone of the multiplicative hyperbolic model of choice (MHM), which was proposed twenty years ago as a modification of Mazur's (1987) hyperbolic model of delay discounting. This paper reviews some of the merits and shortcomings of MHM, and suggests some ways in which MHM might be extended and improved. A formal link between 'value' and the related concept of 'response strength' is suggested, and revisions of the model are proposed which may enable it to accommodate several behavioural phenomena not considered in the original formulation. Broadening the scope of MHM comes at the cost of adding to its burden of free parameters, and it is emphasised that addition of any new parameters needs empirical justification. The status of value as a dimensionless intervening variable is upheld; however it is noted that a growing body of empirical evidence for links between neurobiological phenomena and value suggests that interpretation of value as a hypothetical construct may be warranted.

The effects of outcome unit framing on delay discounting

We examined the effects of outcome framing on delay discounting. In Experiment 1, participants completed four delay-discounting tasks. In one monetary task, money was framed in units of dollars (＄50), and in the other, money was framed in units of handfuls of quarters (equal to ＄50). In one food task, food was framed in clear units of food (e.g., 100 M&Ms), and in the other, food was framed in units of servings (e.g., 10 servings of M&Ms). When money was framed in units of dollars, participants discounted less by delay compared to discounting of handfuls of quarters. When food was framed as clear units, participants also discounted less compared to how they discounted servings. In Experiment 2, participants completed two delay-discounting tasks for dollars and quarters (e.g., ＄50 or 200 quarters) to determine if the results of Experiment 1 were due to the differences in handling costs. In one delay-discounting task, money was framed in units of dollars. In the other delay-discounting task, money was framed in units of quarters. There was no difference in how participants discounted delayed money framed as dollars or quarters. Clear unit framing may result in less discounting by delay than fuzzy unit framing.

Investigating the Impact of Cognitive Load and Motivation on Response Control in Relation to Delay Discounting in Children with ADHD

Attention-deficit/hyperactivity disorder (ADHD) is characterized by deficits in impulse control across a range of behaviors, from simple actions to those involving complex decision-making (e.g., preference for smaller-sooner versus larger later rewards). This study investigated whether changes in motor response control with increased cognitive load and motivational contingencies are associated with decision-making in the form of delay discounting among 8-12 year old children with and without ADHD. Children with ADHD (n = 26; 8 girls) and typically developing controls (n = 40; 11 girls) completed a standard go/no-go (GNG) task, a GNG task with motivational contingencies, a GNG task with increased cognitive load, and two measures of delay discounting: a real-time task in which the delays and immediately consumable rewards are experienced in real-time, and a classic task involving choices about money at longer delays. Children with ADHD, particularly girls, exhibited greater delay discounting than controls during the real-time discounting task, whereas diagnostic groups did not significantly differ on the classic discounting task. The effect of cognitive load on response control was uniquely associated with greater discounting on the real-time task for children with ADHD, but not for control children. The effect of motivational contingencies on response control was not significantly associated with delay discounting for either diagnostic group. The findings from this study help to inform our understanding of the factors that influence deficient self-control in ADHD, suggesting that impairments in cognitive control may contribute to greater delay discounting in ADHD.

Automating Scoring of Delay Discounting for the 21- and 27-Item Monetary Choice Questionnaires

Delay discounting describes the process wherein rewards lose value as a function of their delayed receipt; how quickly rewards lose value is termed the rate of delay discounting. Rates of delay discounting are robust predictors of much behavior of societal importance. One efficient approach to obtaining a human subject's rate of delay discounting is via the 21- and 27-item Monetary Choice Questionnaires, brief dichotomous choice tasks that assess preference between small immediate and larger delayed monetary outcomes. Unfortunately, the scoring procedures for the Monetary Choice Questionnaires are rather complex, which may serve as a barrier to their use. This report details a freely available Excel-based spreadsheet tool that automatically scores Monetary Choice Questionnaire response sets, using both traditional and contemporary/advanced approaches. An overview of the Monetary Choice Questionnaire and its scoring algorithm is provided. We conclude with general considerations for using the spreadsheet tool.

The amount effect and marginal value

The amount effect of delay discounting (by which the value of larger reward amounts is discounted by delay at a lower rate than that of smaller amounts) strictly implies that value functions (value as a function of amount) are steeper at greater delays than they are at lesser delays. That is, the amount effect and the difference in value functions at different delays are actually a single empirical finding. Amount effects of delay discounting are typically found with choice experiments. Value functions for immediate rewards have been empirically obtained by direct judgment. (Value functions for delayed rewards have not been previously obtained.) The present experiment obtained value functions for both immediate and delayed rewards by direct judgment and found them to be steeper when the rewards were delayed--hence, finding an amount effect with delay discounting.

Models of ADHD: Five ways smaller sooner is better

BACKGROUND

In the delay discounting paradigm choices are made between a good of small utility available soon (SS) vs. a good of greater utility available later (LL). Versions of the task may be used with human and non-human subjects, they generate characteristic data, and are of relevance the topic of this special issue, the imputed impulsivity of organisms categorized as ADHD, and animal models of ADHD.

METHOD

Mathematical models of judgment were formulated. The first model adds the utility of a good to the utility of the delay. It provides distinct information about future perspective, the marginal utility of the goods offered, and the relative weight on delay. It grounds hyperbolic discounting. It predicts slower rates of discount for more preferred goods. It can reduce to an exponential function on future perspective time. When choice is mediated by conditioning, not judgment, a model of conditioned reinforcement strength is written. For studies where only the degree of preference for the LL is reported, a Thurstonian model of discrimination provides a shell that embraces all of the earlier models, providing additional information about the precision and bias of those preferential judgments.

COMPARISON WITH OTHER MODELS

All other prominent models are nested within, or approximations to, the models described here.

RESULTS

The models are fit to exemplary data. They provide some support for theories of delay aversion, and explanations for some inconsistencies in the literature. They involve parameters of general utility and provide connection to economic models.