Hyperbolae

Hyperbolic relations between independent and dependent variables are ubiquitous in the experimental analysis of behavior, mentioned in over 150 articles in the Journal of the Experimental Analysis of Behavior. There are two principal forms of hyperbolae: The first describes the relation between response rate and reinforcement rate on variable-interval schedules of reinforcement; it rises asymptotically toward a maximum. The second describes the relation between the current equivalent value of an incentive and its delay or (im)probability; it falls from a maximum toward an asymptote of 0. Where do these come from? What do their parameters mean? How are they related? This article answers the first two questions and addresses the last.

Resistance to change, of behavior and of theory

The persistence of operant behavior when disrupted tends to be positively related to how often reinforcers were delivered in the past. Behavioral momentum theory describes this finding as the outcome of Pavlovian processes. That is, the relation between discriminative stimuli and reinforcers that were delivered in their presence strengthens behavior, thereby making it more likely to persist. If only the story were that simple. A growing number of findings challenge the basic tenets of behavioral momentum theory. Some even call into question whether Pavlovian relations contribute to persistence in the first place. In this paper, I will review behavioral momentum theory and some of the data that have been problematic for the theory. I will argue that despite these very real challenges, the theory provides important utility not only to basic analyses of response persistence but also to clinical interventions directed at long-term reductions in problem behavior. It, for example, has set the stage for the development of alternative conceptual analyses of resistance to change, two of which will be highlighted for readers. Moreover, behavioral momentum theory may tell us something important about the reasons it continues to have an influence on the field, despite the challenging data that deter it.

Theory of reinforcement schedules

The three principles of reinforcement are (1) events such as incentives and reinforcers increase the activity of an organism; (2) that activity is bounded by competition from other responses; and (3) animals approach incentives and their signs, guided by their temporal and physical conditions, together called the "contingencies of reinforcement." Mathematical models of each of these principles comprised mathematical principles of reinforcement (MPR; Killeen, 1994). Over the ensuing decades, MPR was extended to new experimental contexts. This article reviews the basic theory and its extensions to satiation, warm-up, extinction, sign tracking, pausing, and sequential control in progressive-ratio and multiple schedules. In the latter cases, a single equation balancing target and competing responses governs behavioral contrast and behavioral momentum. Momentum is intrinsic in the fundamental equations, as behavior unspools more slowly from highly aroused responses conditioned by higher rates of incitement than it does from responses from leaner contexts. Habits are responses that have accrued substantial behavioral momentum. Operant responses, being predictors of reinforcement, are approached by making them: The sight and feel of a paw on a lever is approached by placing paw on lever, as attempted for any sign of reinforcement. Behavior in concurrent schedules is governed by approach to momentarily richer patches (melioration). Applications of MPR in behavioral pharmacology and delay discounting are noted.

A Theory of the Extinction Burst

A preliminary theory of a temporary increase in the rate of an operant response with the transition to extinction (i.e., the extinction burst) is proposed. The theory assumes reinforcers are events permitting access to some valuable activity, and that such activity can compete for allocation with the target response under some conditions (e.g., very high reinforcement rates). With the transition to extinction, elimination of this competition for allocation can produce an increase in the the target response, but the increase is transient because the value of the target response decreases with exposure to extinction. The theory provides a way to understand why the extinction burst is not ubiquitous, seems more common following very small ratio schedules, occurs for a short period of time following the transition to extinction, and may be eliminated with the availability of alternative reinforcement. It appears to provide a reasonable starting point for a theory of the extinction burst that does not necessarily require inclusion of invigorating effects of frustration, and it is closely aligned with Resurgence as Choice theory. Additional research on factors modulating reinforcement-related activities and how they affect the extinction burst could help to further evaluate the theory.

Behavior Stability and Individual Differences in Pavlovian Extended Conditioning

How stable and general is behavior once maximum learning is reached? To answer this question and understand post-acquisition behavior and its related individual differences, we propose a psychological principle that naturally extends associative models of Pavlovian conditioning to a dynamical oscillatory model where subjects have a greater memory capacity than usually postulated, but with greater forecast uncertainty. This results in a greater resistance to learning in the first few sessions followed by an over-optimal response peak and a sequence of progressively damped response oscillations. We detected the first peak and trough of the new learning curve in our data, but their dispersion was too large to also check the presence of oscillations with smaller amplitude. We ran an unusually long experiment with 32 rats over 3,960 trials, where we excluded habituation and other well-known phenomena as sources of variability in the subjects' performance. Using the data of this and another Pavlovian experiment by Harris et al. (2015), as an illustration of the principle we tested the theory against the basic associative single-cue Rescorla–Wagner (RW) model. We found evidence that the RW model is the best non-linear regression to data only for a minority of the subjects, while its dynamical extension can explain the almost totality of data with strong to very strong evidence. Finally, an analysis of short-scale fluctuations of individual responses showed that they are described by random white noise, in contrast with the colored-noise findings in human performance.

Resurgence as Choice in Context: Treatment duration and on/off alternative reinforcement

Resurgence as Choice (RaC) is a quantitative theory suggesting that an increase in an extinguished target behavior with subsequent extinction of an alternative behavior (i.e., resurgence) is governed by the same processes as choice more generally. We present data from an experiment with rats examining a range of treatment durations with alternative reinforcement plus extinction and demonstrate that increases in treatment duration produce small but reliable decreases in resurgence. Although RaC predicted the relation between target responding and treatment duration, the model failed in other respects. First, contrary to predictions, the present experiment also replicated previous findings that exposure to cycling on/off alternative reinforcement reduces resurgence. Second, RaC did a poor job simultaneously accounting for target and alternative behaviors across conditions. We present a revised model incorporating a role for more local signaling effects of reinforcer deliveries or their absence on response allocation. Such signaling effects are suggested to impact response allocation above and beyond the values of the target and alternative behaviors as longer-term repositories of experience. The new model provides an excellent account of the data and can be viewed as an integration of RaC and a quantitative approximation of some aspects of Context Theory.

Behavioral momentum and resistance to extinction across repeated extinction tests

The present experiments assessed whether resistance to extinction of pigeons' key pecking decreased across repeated extinction tests. An additional impetus for this research was to determine how the quantitative framework provided by behavioral momentum theory might be used to describe any such changes across tests. Pigeons pecked keys in two-component multiple schedules (one component associated with a higher reinforcer rate and the other with a lower rate) in which baseline and extinction conditions alternated. In Experiment 1, baseline and extinction conditions alternated every session, and, in Experiment 2, these conditions lasted for 10 and 7 sessions, respectively. Resistance to extinction decreased across successive extinction conditions in both experiments. Fits of the behavioral-momentum based model of extinction to the data returned uncertain results in Experiment 1 but implicated both generalization decrement and response-reinforcer contingency termination as the possible mechanisms responsible for behavior change in Experiment 2. Thus, these data suggest that experimental manipulations that affect discrimination of changes in reinforcement contingencies may influence resistance to extinction by modulating the disruptive impacts of removing reinforcers from the experimental context and of suspending response-reinforcer contingencies.

An Information Theoretic Approach to Model Selection: A Tutorial with Monte Carlo Confirmation

A reliance on null hypothesis significance testing (NHST) and misinterpretations of its results are thought to contribute to the replication crisis while impeding the development of a cumulative science. One solution is a data-analytic approach called Information-Theoretic (I-T) Model Selection, which builds upon Maximum Likelihood estimates. In the I-T approach, the scientist examines a set of candidate models and determines for each one the probability that it is the closer to the truth than all others in the set. Although the theoretical development is subtle, the implementation of I-T analysis is straightforward. Models are sorted according to the probability that they are the best in light of the data collected. It encourages the examination of multiple models, something investigators desire and that NHST discourages. This article is structured to address two objectives. The first is to illustrate the application of I-T data analysis to data from a virtual experiment. A noisy delay-discounting data set is generated and seven quantitative models are examined. In the illustration, it is demonstrated that it is not necessary to know the "truth" is to identify the one that is closest to it and that the most likely models conform to the model that generated the data. Second, we examine claims made by advocates of the I-T approach using Monte Carlo simulations in which 10,000 different data sets are generated and analyzed. The simulations showed that 1) the probabilities associated with each model returned by the single virtual experiment approximated those that resulted from the simulations, 2) models that were deemed close to the truth produced the most precise parameter estimates, and 3) adding a single replicate sharpens the ability to identify the most probable model.