Bidding for delayed rewards: Accumulation as delay discounting, delay discounting as regulation, demand functions as corollary

Hyperbolae

Hyperbolic relations between independent and dependent variables are ubiquitous in the experimental analysis of behavior, mentioned in over 150 articles in the Journal of the Experimental Analysis of Behavior. There are two principal forms of hyperbolae: The first describes the relation between response rate and reinforcement rate on variable-interval schedules of reinforcement; it rises asymptotically toward a maximum. The second describes the relation between the current equivalent value of an incentive and its delay or (im)probability; it falls from a maximum toward an asymptote of 0. Where do these come from? What do their parameters mean? How are they related? This article answers the first two questions and addresses the last.

From data through discount rates to the area under the curve

The rate of discounting future goods is a crucial factor in intertemporal trade-offs, upon which depends not only individual well-being but also that of our planet: How much privation now for a temperate future for our grandchildren? What is the best way to measure how the value of future goods decreases with its delay? The most accurate discount functions involve several covarying parameters, making interpretation equivocal. A universal and robust measure is the area under the discount curve, the AuC. The AuC of a hyperbolic discount function is a logarithmic function of the discount rate, k. The same integral also approximates the area under a hyperboloid function. A simple technique converts each datum into estimates of the discount rate, eliminating rogue data points in the process. These trimmed estimates are converted into areas and tested against data, where they succeed at predicting the AuC and its relation to log(k).

Theory of reinforcement schedules

The three principles of reinforcement are (1) events such as incentives and reinforcers increase the activity of an organism; (2) that activity is bounded by competition from other responses; and (3) animals approach incentives and their signs, guided by their temporal and physical conditions, together called the "contingencies of reinforcement." Mathematical models of each of these principles comprised mathematical principles of reinforcement (MPR; Killeen, 1994). Over the ensuing decades, MPR was extended to new experimental contexts. This article reviews the basic theory and its extensions to satiation, warm-up, extinction, sign tracking, pausing, and sequential control in progressive-ratio and multiple schedules. In the latter cases, a single equation balancing target and competing responses governs behavioral contrast and behavioral momentum. Momentum is intrinsic in the fundamental equations, as behavior unspools more slowly from highly aroused responses conditioned by higher rates of incitement than it does from responses from leaner contexts. Habits are responses that have accrued substantial behavioral momentum. Operant responses, being predictors of reinforcement, are approached by making them: The sight and feel of a paw on a lever is approached by placing paw on lever, as attempted for any sign of reinforcement. Behavior in concurrent schedules is governed by approach to momentarily richer patches (melioration). Applications of MPR in behavioral pharmacology and delay discounting are noted.

Hyperbolic discounting and exponentiated demand: Modeling demand for cigarettes in three dimensions

Behavioral economics has been a fruitful area of research in substance use. Mathematical descriptions of how individuals temporally discount the value of a commodity have been correlated with substance use and mathematical descriptions of drug consumption decreasing as a function of price (i.e., demand) predict maladaptive substance use. While there is a logical assumption that temporal factors affect demand for a drug, little has been done to merge these models. Thus, the purpose of this study was to combine models of discounting and demand, extending Howard Rachlin's work and contributions to novel areas of study. Data from 85 participants from Amazon Mechanical Turk (MTurk) who completed a hypothetical cigarette purchase task that included price of and delay to cigarettes were analyzed. Multilevel modeling was used to determine descriptive accuracy of combined additive and multiplicative models of discounting and demand. Of the discounting models used in conjunction with the exponentiated demand equation, the Rachlin hyperboloid best described the delay dimension of consumption. The multiplicative version of the Rachlin equation applied to both delay and price outperformed other models tested. Therefore, existing models of discounting and demand can be extended to modeling consumption data from complex multidimensional experimental arrangements.

Ecological factors shape quantitative decision-making in coyotes

Much research has focused on the development and evolution of cognition in the realm of numerical knowledge in human and nonhuman animals but often fails to take into account ecological realities that, over time, may influence and constrain cognitive abilities in real-life decision-making. Cognitive abilities such as enumerating and timing are central to many psychological and ecological models of behavior, yet our knowledge of how these are affected by environmental fluctuations remains incomplete. Our research bridges the gap between basic cognitive research and ecological decision-making. We used coyotes (Canis latrans) as a model animal system to study decision-making about smaller, more proximal food rewards and larger, more distant food rewards; we tested animals across their four reproductive cycle phases to examine effects of ecological factors such as breeding status and environmental risk on quantitative performance. Results show that coyotes, similar to other species, spatially discount food rewards while foraging. The degree to which coyotes were sensitive to the risk of obtaining the larger food reward, however, depended on the season in which they completed the foraging task, the presence of unfamiliar humans (i.e., risk), and the presence of conspecifics. Importantly, our results support that seasonal variations drive many differences in nonhuman animal behavior and cognition (e.g., hibernation, breeding, food resource availability). Further, it may be useful in the future to extend this work to humans because seasons may influence human cognition as well, and this remains unexplored in the realms of enumeration, timing, and spatial thinking.

The essential value of the alternatives of the suboptimal choice procedure is different for pigeons and rats

Suboptimal choice is the preference for a discriminative alternative with low probability of reinforcement, over a non-discriminative alternative with higher probability of reinforcement. Pigeons consistently prefer the discriminative alternative, whereas rats prefer the non-discriminative; the variables accounting for this difference are not yet clear. The economic concepts related to demand curves have been used to calculate the essential value of different types of reinforcers, so they may be useful to compare the value of the alternatives in the suboptimal choice procedure. The goal of this study was to calculate the essential value of each of the alternatives presented in the suboptimal choice procedure to assess if pigeons (Experiment 1) and rats (Experiment 2) value them differently. In both experiments, the fixed ratio requirement in the initial link was increased throughout sessions in order to obtain the demand curve and calculate the essential value by fitting the exponential-demand model. A Bayesian Linear Mixed-Effects Model indicated that pigeons had higher essential values for the discriminative alternative, whereas rats obtained higher essential values for the non-discriminative alternative. These results suggest that preferences in the suboptimal choice procedure are indeed based on the essential value of the alternatives, and provide a new paradigm to study the variables affecting this phenomenon.