Concurrent choice: Effects of overall reinforcer rate and the temporal distribution of reinforcers

Theory of reinforcement schedules

The three principles of reinforcement are (1) events such as incentives and reinforcers increase the activity of an organism; (2) that activity is bounded by competition from other responses; and (3) animals approach incentives and their signs, guided by their temporal and physical conditions, together called the "contingencies of reinforcement." Mathematical models of each of these principles comprised mathematical principles of reinforcement (MPR; Killeen, 1994). Over the ensuing decades, MPR was extended to new experimental contexts. This article reviews the basic theory and its extensions to satiation, warm-up, extinction, sign tracking, pausing, and sequential control in progressive-ratio and multiple schedules. In the latter cases, a single equation balancing target and competing responses governs behavioral contrast and behavioral momentum. Momentum is intrinsic in the fundamental equations, as behavior unspools more slowly from highly aroused responses conditioned by higher rates of incitement than it does from responses from leaner contexts. Habits are responses that have accrued substantial behavioral momentum. Operant responses, being predictors of reinforcement, are approached by making them: The sight and feel of a paw on a lever is approached by placing paw on lever, as attempted for any sign of reinforcement. Behavior in concurrent schedules is governed by approach to momentarily richer patches (melioration). Applications of MPR in behavioral pharmacology and delay discounting are noted.

Effects of brief post-sample cues signaling presence or absence of reinforcers in delayed matching-to-sample

When short-term memory is assessed in the delayed matching-to-sample (DMTS) procedure, performance is better when cues signal larger reinforcer magnitudes or higher reinforcer probabilities for correct responding. Previous studies demonstrating signaled-magnitude or signaled-probability effects presented cues for a prolonged period during the sample stimulus and/or retention interval. The present study asked whether a signaled-probability effect would occur with brief post-sample cues that signaled the presence or absence of reinforcement. Five pigeons responded in a DMTS task in which sample stimuli were sometimes followed by a 0.5-s cue signaling that reinforcers would either be available or not available in the current trial, and the retention interval varied from 0.5s to 20s. A reliable signaled-probability effect was found when reinforcers were arranged independently and for all correct responses, whereas a smaller, less systematic effect was found when reinforcers were arranged dependently and probabilistically. These findings highlight the importance of reinforcement contingencies and contingency discriminability in remembering, and add to the evidence showing that cues signaling differential reinforcement in DMTS may affect processes during the retention interval and comparison phase, rather than attention to the sample stimulus.

Concurrent schedules: How responses per reinforcer affects estimates of sensitivity to reinforcement

Conventionally, when choice is measured under concurrent schedules, all responses are included. However, the class of all responses consists of 2 sub-classes which are discriminable by their properties: Reinforced responses always equal obtained reinforcers, while unreinforced responses are free to vary. As a result, the inclusion of reinforced responses in choice measures results in sensitivity values in generalized matching that are biased toward larger values, and this bias becomes extreme in some combinations of overall response and reinforcer rates. Different ways of varying concurrent-schedule values also affect estimated sensitivity and the linearity between choice based on all responses and reinforcer ratios. To avoid spurious results and comparisons, and as a matter of good practice, generalized-matching fits and measures should be done using only unreinforced responses.

Methodological improvements to a Procedure for Rapidly Establishing Steady-State Behavior

We performed three experiments to improve the quality and retention of data obtained from a Procedure for Rapidly Establishing Steady-State Behavior (PRESS-B; Klapes et al., 2020). In Experiment 1, 120 participants worked on nine concurrent random-interval random-interval (conc RI RI) schedules and were assigned to four conditions of varying changeover delay (COD) length. The 0.5-s COD condition group exhibited the fewest instances of exclusive reinforcer acquisition. Importantly, this group did not differ in generalized matching law (GML) fit quality from the other groups. In Experiment 2, 60 participants worked on nine conc RI RI schedules with a wider range of scheduled reinforcement rate ratios than was used in Experiment 1. Participants showed dramatic reductions in exclusive reinforcer acquisition. Experiment 3 entailed a replication of Experiment 2 wherein blackout periods were implemented between the schedule presentations and each schedule remained in operation until at least one reinforcer was acquired on each alternative. GML fit quality was slightly more consistent in Experiment 3 than in the previous experiments. Thus, these results suggest that future PRESS-B studies should implement a shorter COD, a wider and richer scheduled reinforcement rate ratio range, and brief blackouts between schedule presentations for optimal data quality and retention.

A multivariate assessment of the rapidly changing procedure with McDowell's Evolutionary Theory of Behavior Dynamics

A multivariate analysis is concerned with more than one dependent variable simultaneously. Models that generate event records have a privileged status in a multivariate analysis. From a model that generates event records, we may compute predictions for any dependent variable associated with those event records. However, because of the generality that is afforded to us by these kinds of models, we must carefully consider the selection of dependent variables. Thus, we present a conditional compromise heuristic for the selection of dependent variables from a large group of variables. The heuristic is applied to McDowell's Evolutionary Theory of Behavior Dynamics (ETBD) for fitting to a concurrent variable-interval schedule in-transition dataset. From the parameters obtained from fitting ETBD, we generated predictions for a wide range of dependent variables. Overall, we found that our ETBD implementation accounted well for various flavors of the log response ratio, but had difficulty accounting for the overall response rates and cumulative reinforcer effects. Based on these results, we argue that the predictions of our ETBD implementation could be improved by decreasing the base response probabilities, either by increasing the response latencies or by decreasing the sizes of the operant classes.

Choosing among multiple alternatives: Relative and overall reinforcer rates

Choice behavior among two alternatives has been widely researched, but fewer studies have examined the effect of multiple (more than two) alternatives on choice. Two experiments investigated whether changing the overall reinforcer rate affected preference among three and four concurrently scheduled alternatives. Experiment 1 trained six pigeons on concurrent schedules with three alternatives available simultaneously. These alternatives arranged reinforcers in a ratio of 9:3:1 with the configuration counterbalanced across pigeons. The overall rate of reinforcement was varied across conditions. Preference between the pair of keys arranging the 9:3 reinforcer ratio was less extreme than the pair arranging the 3:1 reinforcer ratio regardless of overall reinforcer rate. This difference was attributable to the richer alternative receiving fewer responses per reinforcer than the other alternatives. Experiment 2 trained pigeons on concurrent schedules with four alternatives available simultaneously. These alternatives arranged reinforcers in a ratio of 8:4:2:1, and the overall reinforcer rate was varied. Next, two of the alternatives were put into extinction and the random interval duration was changed from 60 s to 5 s. The ratio of absolute response rates was independent of interval length across all conditions. In both experiments, an analysis of sequences of visits following each reinforcer showed that the pigeons typically made their first response to the richer alternative irrespective of which alternative was just reinforced. Performance on these three- and four-alternative concurrent schedules is not easily extrapolated from corresponding research using two-alternative concurrent schedules.

Choice among two and three alternatives

Although choice between two alternatives has been widely researched, fewer studies have examined choice across multiple (more than two) alternatives. Past models of choice behavior predict that the number of alternatives should not affect relative response allocation, but more recent research has found violations of this principle. Five pigeons were presented with three concurrently scheduled alternatives. Relative reinforcement rates across these alternatives were assigned 9:3:1. In some conditions three keys were available; in others, only two keys were available. The number of available alternatives did not affect relative response rates for pairs of alternatives; there were no significant differences in behavior between the two and three key conditions. For two birds in the three-alternative conditions and three birds in the two-alternative conditions, preference was more extreme for the pair of alternatives with the lower overall pairwise reinforcer rate (3:1) than the pair with higher overall reinforcer rate (9:3). However, when responding during the changeover was removed three birds showed the opposite pattern in the three-alternative conditions; preference was more extreme for the pair of alternatives with the higher overall reinforcer rate. These findings differ from past research and do not support established theories of choice behavior.

Effects of changeover delay on response allocation during probe tests

The present study assessed whether a pattern of responding that develops when choosing between two alternatives generalizes to novel choice tests when alternatives are presented in new combinations. Pigeons were trained on a two-component multiple schedule. In both components, a concurrent variable-interval (VI) 40-s VI 80-s schedule was used. The COD was 1 s in one component and 10 s in the other. The long COD produced consistently longer dwell times than the short COD did. Following training, subjects were presented with four types of probe-test components in which one alternative was drawn from the component with the short COD and one alternative was drawn from the component with the long COD. When the schedule values of the two alternatives were identical (VI 40 vs. VI 40 and VI 80 vs. VI 80), subjects preferred the alternative trained with the long COD (Ms = .78 and .61, respectively). Additionally, subjects preferred the VI 40-s alternative trained with the long COD to the VI 80-s alternative that was trained with the short COD (M = .85). Systematic preference was not observed when subjects were given a choice between the VI 40-s alternative that was trained with the short COD and the VI 80-s alternative that was trained with the long COD. These results demonstrate that a stimulus associated with a longer COD, and thus longer dwell times in baseline training, may be more preferred during probe tests than expected on the basis of the rate of primary reinforcement associated with that stimulus.

Law of effect models and choice between many alternatives

Data from five experiments on choice between more than two variable-interval schedules were modeled with different equations for the Law of Effect. Navakatikyan's (2007) component-functions models with three, four and five free parameters were compared with Stevens' (1957), Herrnstein's (1970) and Davison and Hunter's (1976) equations. These latter models are consistent with the generalized-matching principle, whereas Navakatikyan's models are not. Navakatikyan's models performed better or on par with their competitors, especially in predicting residence-time data and generalized-matching sensitivities for time allocation. The models described well an observed decrease, in several of these data sets, in generalized-matching sensitivity between two alternatives when reinforcer rate increased on the other alternatives. Models built on the generalized-matching principle cannot do this. Navakatikyan's models also performed better, though to a lesser extent, than their competitors for data sets that are not obviously inconsistent with generalized matching.

On the joint control of preference by time and reinforcer-ratio variation

Five pigeons were trained in a procedure in which, with a specified probability, food was either available on a fixed-interval schedule on the left key, or on a variable-interval schedule on the right key. In Phase 1, we arranged, with a probability of 0.5, either a left-key fixed-interval schedule or a right-key variable-interval 30s, and varied the value of the fixed-interval schedule from 5s to 50s across 5 conditions. In Phase 2, we arranged either a left-key fixed-interval 20-s schedule or a right-key variable-interval 30-s schedule, and varied the probability of the fixed-interval schedule from 0.05 to 1.0 across 8 conditions. Phase 3 always arranged a fixed-interval schedule on the left key, and its value was varied over the same range as in Phase 1. In Phase 1, overall preference was generally toward the variable-interval schedule, preference following reinforcers was initially toward the variable-interval schedule, and maximum preference for the fixed-interval schedule generally occurred close to the arranged fixed-interval time, becoming relatively constant thereafter. In Phase 2, overall left-key preference followed the probability of the fixed-interval schedule, and maximum fixed-interval choice again occurred close to the fixed-interval time, except when the fixed-interval probability was 0.1 or less. The pattern of choice following reinforcers was similar to that in Phase 1, but the peak fixed-interval choice became more peaked with higher probabilities of the fixed interval. Phase 3 produced typical fixed-interval schedule responding. The results are discussed in terms of reinforcement effects, timing in the context of alternative reinforcers, and generalized matching. These results can be described by a quantitative model in which reinforcer rates obtained at times since the last reinforcer are distributed across time according to a Gaussian distribution with constant coefficient of variation before the fixed-interval schedule time, changing to extended choice controlled by extended reinforcer ratios beyond the fixed-interval time. The same model provides a good description of response rates on single fixed-interval schedules.

Obese and lean Zucker rats demonstrate differential sensitivity to rates of food reinforcement in a choice procedure

The obese Zucker rat carries two recessive fa alleles that result in the expression of an obese phenotype. Obese Zuckers have higher food intake than lean controls in free-feed studies in which rats have ready access to a large amount of one type of food. The present study examined differences in obese and lean Zucker rats using concurrent schedules of reinforcement, which more ecologically models food selection using two food choices that have limited, but generally predictable availability. Lever-pressing of ten lean (Fa/Fa or Fa/fa) and ten obese (fa/fa) Zucker rats was placed under three concurrent variable interval variable interval (conc VI VI) schedules of sucrose and carrot reinforcement, in which the programmed reinforcer ratios for 45-mg food pellets were 5:1, 1:1, and 1:5. Allocation of responses to the two food alternatives was characterized using the generalized matching equation, which allows sensitivity to reinforcer rates (a) and bias toward one alternative (log k) to be quantified. All rats showed a bias toward sucrose, though there were no differences between lean and obese Zucker rats. In addition, obese Zucker rats exhibited higher sensitivity to reinforcement rates than lean rats. This efficient pattern of responding was related to overall higher deliveries of food pellets. Effective matching for food, then, may be another behavioral pattern that contributes to an obese phenotype.

Differential reinforcement and resistance to change of divided-attention performance

Behavioral momentum theory provides a framework for understanding how conditions of reinforcement influence instrumental response strength under conditions of disruption (i.e., resistance to change). The present experiment examined resistance to change of divided-attention performance when different overall probabilities of reinforcement were arranged across two components of a multiple schedule. Pigeons responded in a delayed-matching-to-sample procedure with compound samples (color + line orientation) and element comparisons (two colors or two line orientations). Reinforcement ratios of 1:9, 1:1, and 9:1 for accurate matches on the two types of comparison trials were examined across conditions using reinforcement probabilities (color/lines) of .9/.1, .5/.5, and .1/.9 in the rich component and .18/.02, .1/.1, and .02/.18 in the lean component. Relative accuracy with color and line comparisons was an orderly function of relative reinforcement, but this relation did not depend on the overall rate of reinforcement between components. The resistance to change of divided-attention performance was greater for both trial types in the rich component with presession feeding and extinction, but not with decreases in sample duration. These findings suggest promise for the applicability of quantitative models of operant behavior to divided-attention performance, but they highlight the need to further explore conditions impacting the resistance to change of attending.

Reinforcement: food signals the time and location of future food

It has long been understood that food deliveries may act as signals of future food location, and not only as strengtheners of prefood responding as the law of effect suggests. Recent research has taken this idea further--the main effect of food deliveries, or other "reinforcers", may be signaling rather than strengthening. The present experiment investigated the ability of food deliveries to signal food contingencies across time after food. In Phase 1, the next food delivery was always equally likely to be arranged for a left- or a right-key response. Conditions were arranged such that the next food delivery was likely to occur either sooner on the left (or right) key, or sooner on the just-productive (or not-just-productive) key. In Phase 2, similar contingencies were arranged, but the last-food location was signaled by a red keylight. Preference, measured in 2-s bins across interfood intervals, was jointly controlled by the likely time and location of the next food delivery. In Phase 1, when any food delivery signaled a likely sooner next food delivery on a particular key, postfood preference was strongly toward that key, and moved toward the other key across the interreinforcer interval. In other conditions in which food delivery on the two keys signaled different subsequent contingencies, postfood preference was less extreme, and quickly moved toward indifference. In Phase 2, in all three conditions, initial preference was strongly toward the likely-sooner food key, and moved to the other key across the interfood interval. In both phases, at a more extended level of analysis, sequences of same-key food deliveries caused a small increase in preference for the just-productive key, suggesting the presence of a "reinforcement effect", albeit one that was very small.

Bias and undermatching in delinquent boys' verbal behavior as a function of their level of deviance

Eighty-one 13- to 14-year-old boys at risk for delinquency (target boys) engaged in brief dyadic conversations with their peer friends. The target boys' verbal behavior was coded into two mutually exclusive content categories, rule-break talk and normative talk. Positive social responses from peer boys for each category of talk were also recorded, and were presumed to reinforce the target boys' verbal behavior. A measure of child deviance was available for each target boy. The generalized matching law was fitted to the target boys' response and time allocation data and provided an excellent description of their verbal behavior, with an expected degree of undermatching and strong bias in favor of normative talk. When the boys' data were separated into groups of increasing child deviance, the matching law continued to provide an excellent description of the boys' verbal behavior regardless of their level of deviance, but undermatching became more severe and bias favoring normative talk became less strong as child deviance increased. Based on a selectionist theory of adaptive behavior dynamics from the basic science, it was suggested that the increasing degree of undermatching might be due to a decline in the reinforcing value of positive social responses with increasing child deviance. It was also suggested that the trend in the bias parameters might be due to different histories of reinforcement and punishment of rule-break and normative behavior for boys characterized by different levels of child deviance.

The identities hidden in the matching laws, and their uses

Various theoretical equations have been proposed to predict response rate as a function of the rate of reinforcement. If both the rate and probability of reinforcement are considered, a simple identity, defining equation, or "law" holds. This identity places algebraic constraints on the allowable forms of our mathematical models and can help identify the referents for certain empirical or theoretical coefficients. This identity can be applied to both single and compound schedules of reinforcement, absolute and relative measures, and to local, global and overall rates and probabilities. The rate matching equations of Hernstein and Catania appear to have been approximations to, and to have been evolving toward, one form of this algebraic identity. Estimates of the bias and sensitivity terms in the generalized ratio and logarithmic matching models are here held to be averaging artifacts arising from fitting procedures applied to models that violate or conceal the underlying identities.

Within-session transitions in choice: a structural and quantitative analysis

The present study used within-session transitions between two concurrent schedules to evaluate choice in transition. Eight female Long-Evans rats were trained to respond under concurrent schedules of reinforcement during experimental sessions that lasted 22 hr. The generalized matching equation was used to model steady-state behavior at the end of each session, while transitional behavior that emerged following the change in reinforcement schedules was modeled using a logistic equation. The generalized matching and logistic equations were appropriate models for behavior generated during single-session transitions. A local analysis of behavior on the two response alternatives during acquisition was used to determine the source of preference as revealed in response ratios. The number of "low-response" visits, those containing three to five responses, remained stable. Preference ratios largely reflected a sharp increase in the number of visits with long response bouts on the rich alternative and a decrease in the number of such visits to the leaner alternative.

Some effects of procedural variables on operant choice behavior

Procedural variables are concerned with the way subjects contact stimulus, response, and reinforcer variables and relations in an experiment. Procedural variables may therefore be understood as tools that allow researchers and theorists to explore the generality of their conceptions of independent variables, behavioral processes, and quantitative laws of behavior. The present article reviews data from several studies showing how procedural variables can influence the outcome of operant choice experiments, even when nominal independent variables remain constant. An important theoretical issue raised by these data is whether researchers have most usefully identified the independent variables that they incorporate into their quantitative models. Particular attention is paid to how procedural variables can influence the measure of reinforcement frequency over time that best correlates with the distribution of choice responding over time.

The dynamics of the law of effect: a comparison of models

Dynamical models based on three steady-state equations for the law of effect were constructed under the assumption that behavior changes in proportion to the difference between current behavior and the equilibrium implied by current reinforcer rates. A comparison of dynamical models showed that a model based on Navakatikyan's (2007) two-component functions law-of-effect equations performed better than models based on Herrnstein's (1970) and Davison and Hunter's (1976) equations. Navakatikyan's model successfully described the behavioral dynamics in schedules with negative-slope feedback functions, concurrent variable-ratio schedules, Vaughan's (1981) melioration experiment, and experiments that arranged equal, and constant-ratio unequal, local reinforcer rates.

Relative reinforcer rates and magnitudes do not control concurrent choice independently

One assumption of the matching approach to choice is that different independent variables control choice independently of each other. We tested this assumption for reinforcer rate and magnitude in an extensive parametric experiment. Five pigeons responded for food reinforcement on switching-key concurrent variable-interval variable-interval schedules. Across conditions, the ratios of reinforcer rates and of reinforcer magnitudes on the two alternatives were both manipulated. Control by each independent variable, as measured by generalized-matching sensitivity, changed significantly with the ratio of the other independent variable. Analyses taking the model-comparison approach, which weighs improvement in goodness-of-fit against increasing number of free parameters, were inconclusive. These analyses compared a model assuming constant sensitivity to magnitude across all reinforcer-rate ratios with two alternative models. One of those alternatives allowed sensitivity to magnitude to vary freely across reinforcer-rate ratios, and was less efficient than the common-sensitivity model for all pigeons, according to the Schwarz-Bayes information criterion. The second alternative model constrained sensitivity to magnitude to be equal for pairs of reinforcer-rate ratios that deviated from unity by proportionately equal amounts but in opposite directions. This model was more efficient than the common-magnitude-sensitivity model for 2 of the pigeons, but not for the other 3. An analysis of variance, carried out independently of the generalized-matching analysis, also showed a significant interaction between the effects of reinforcer rate and reinforcer magnitude on choice. On balance, these results suggest that the assumption of independence inherent in the matching approach cannot be maintained. Relative reinforcer rates and magnitudes do not control choice independently.

Local effects of delayed food

Five pigeons were trained on a procedure in which seven concurrent variable-interval schedules arranged seven different food-rate ratios in random sequence in each session. Each of these components lasted for 10 response-produced food deliveries, and components were separated by 10-s blackouts. We varied delays to food (signaled by blackout) between the two response alternatives in an experiment with three phases: In Phase 1, the delay on one alternative was 0 s, and the other was varied between 0 and 8 s; in Phase 2, both delays were equal and were varied from 0 to 4 s; in Phase 3, the two delays summed to 8 s, and each was varied from 1 to 7 s. The results showed that increasing delay affected local choice, measured by a pulse in preference, in the same way as decreasing magnitude, but we found also that increasing the delay at the other alternative increased local preference. This result casts doubt on the traditional view that a reinforcer strengthens a response depending only on the reinforcer's value discounted by any response-reinforcer delay. The results suggest that food guides, rather than strengthens, behavior.

Maternal nutrition and four-alternative choice

Two groups of 10 male rats were trained to nose poke for food pellets at four alternatives that provided differing rates of pellet delivery on aperiodic schedules. After a fixed number of pellets had been delivered, 5, 10 or 20 in different conditions of the experiment, a 10-s blackout occurred, and the locations of the differing rates of pellet delivery were randomized for the next component. Two groups of rats were used: The AD group consisted of 10 rats born to dams that had normal (ad libitum) nutrition during pregnancy, whereas the 10 rats in the UN group were from dams exposed to reduced food availability during pregnancy. All pups received normal nutrition after birth. Choice between the nose-poke alternatives quickly adapted when the rates of pellet delivery were changed in both groups, but there were no consistent differences in the speed of adaptation between the two groups. The generalized matching relation failed to describe the allocation of responses among alternatives, but the contingency-discriminability model provided a precise description of performance.

Global undernutrition during gestation influences learning during adult life

Intrauterine growth restriction can lead to significant long-term health consequences such as metabolic and cardiovascular disorders, but less is known about its effects on choice and behavioral adaptation in later life. Virgin Wistar rats were time mated and randomly assigned to receive either ad-libitum access to chow or 30% of that level of nutrition during pregnancy to generate growth-restricted offspring. At 60 days of age, 6 female offspring from each group were trained on concurrent variable-interval schedules. Sessions consisted of seven randomly arranged concurrent-schedule components, each with a different reinforcer ratio that varied from 27:1 to 1:27, and each component lasting for 10 reinforcer deliveries. Behavioral change across reinforcers in components, measured by sensitivity to reinforcement, was consistently lower for offspring of undernourished mothers, showing that their behavior was less adaptable to environmental change. These results provide direct experimental evidence for a link between prenatal environmental conditions and reduced behavioral adaptability--learning--in later life.

A model for residence time in concurrent variable interval performance

A component-functions model of choice behavior is proposed for performance on interdependent concurrent variable-interval (VI) variable-interval schedules based on the product of two component functions, one that enhances behavior and one that reduces behavior. The model is the solution to the symmetrical pair of differential equations describing behavioral changes with respect to two categories of reinforcers: enhancing and reducing, or excitatory and inhibitory. The model describes residence time in interdependent concurrent VI VI schedules constructed from arithmetic and exponential distributions. The model describes the data reported by Alsop and Elliffe (1988) and Elliffe and Alsop (1996) with a variance accounted for of 87% compared to 64% accounted for by the Davison and Hunter (1976) model and 42% by Herrnstein's (1970) hyperbola. The model can explain matching, undermatching, and overmatching in the same subject under different procedures and has the potential to be extended to performance on concurrent schedules with more than two alternatives, multiple schedules, and single schedules. Thus, it can be considered as an alternative to Herrnstein's quantitative law of effect.

Some effects of overall rate of earning reinforcers on run lengths and visit durations

In a concurrent schedule, responding at each alternative is controlled by a pair of schedules that arrange reinforcers for staying at that alternative and reinforcers for switching to the other alternative. Each pair of schedules operates only while at the associated alternative. When only one pair of stay and switch schedules is presented, the rates of earning reinforcers for staying divided by the rates of earning reinforcers for switching controls the mean number responses in a visit and the mean duration of visits. The purpose of the present experiment was to see whether the sum of the rates of earning stay and switch reinforcers changed the way that run length and visit duration were affected by the ratio of the rates of stay to switch reinforcers. Rats were exposed to pairs of stay and switch schedules that varied both the ratio of the rates of earning stay and switch reinforcers and the sum of the rates of earning stay and switch reinforcers. Run lengths and visit durations were joint functions of the ratio of the rates of earning stay and switch reinforcers and the sum of the rates of earning stay and switch reinforcers. These results shows that the effect of the ratio of the sum of the rates of earning stay and switch reinforcers results from processes operating at the alternative, rather than from processes operating at both alternatives.

Momentary maximizing and optimal foraging theories of performance on concurrent VR schedules

Optimal foraging theory proposes that animals obtain the highest rate of reinforcers for the least effort and momentary maximizing theory proposes that animals make the response that at that instant is most likely to be reinforced. While each theory may account for matching on concurrent schedules, the data supporting each theory are weak. Two experiments assessed these theories by considering concurrent choice as consisting of two pairs of stay and switch schedules. Symmetrical arrangements, which are equivalent to standard concurrent schedules, maintained behavior described by the generalized matching law. Weighted arrangements, in which the programmed rate of earning reinforcers was always greater at one alternative, maintained behavior that was biased towards the weighted alternative, yet the bias was less than that predicted by optimal foraging theory. Asymmetrical arrangements, in which the stay and switch schedules operating at an alternative are the same, maintained behavior that favored one alternative, even though momentary maximizing predicted indifference. The generalized matching law poorly described each rat's pooled data from all conditions but these data were described by an equation based on the stay and switch reinforcers earned per-visit and included elements of optimal foraging and momentary maximizing theories of choice.

On science and the discriminative law of effect

This article considers the process of the dissemination of scientific findings from the point of view of the discriminative law of effect. We assume that the purpose of science is to describe the state of the world in an unbiased and accurate manner. We then consider a number of challenges to the unbiased consensual development of science that arise from differences between science that is done, submitted for publication, and published. These challenges arise from the differential reinforcers for both research and publication delivered by journals and editors for novel results, the undervaluation of systematic replication and findings of invariance, and general lack of reinforcers for failed replications. All these challenges bias science toward searching for, reporting, and valuing novel results and consequently lead to a biased and erroneous view of the world. We suggest that science should be approached more conservatively, and that a reevaluation of the value of replication, and especially failed replication, is in order.

Independence of terminal-link entry rate and immediacy in concurrent chains

In Phase 1, 4 pigeons were trained on a three-component multiple concurrent-chains procedure in which components differed only in terms of relative terminal-link entry rate. The terminal links were variable-interval schedules and were varied across four conditions to produce immediacy ratios of 4:1, 1:4, 2:1, and 1:2. Relative terminal-link entry rate and relative immediacy had additive and independent effects on initial-link response allocation, and the data were well-described by a generalized-matching model. Regression analyses showed that allowing sensitivity to immediacy to vary across components produced only trivial increases in variance accounted for. Phase 2 used a three-component concurrent-schedules procedure in which the schedules were the same as the initial links of Phase 1. Across two conditions, the relative reinforcer magnitude was varied. Sensitivity to relative reinforcer rate was independent of relative magnitude, confirming results of prior studies. Sensitivity to relative reinforcer rate in Phase 2 did not vary systematically across subjects compared to sensitivity to relative entry rate in Phase 1, and regression analyses confirmed again that only small increases in variance accounted for were obtained when sensitivities were estimated independently compared with a single estimate for both phases. Overall, the data suggest that conditioned and primary reinforcers have functionally equivalent effects on choice and support the independence of relative terminal-link entry rate and immediacy as determiners of response allocation. These results are consistent with current models for concurrent chains, including Grace's (1994) contextual choice model and Mazur's (2001) hyperbolic value-added model.

Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.

Every reinforcer counts: reinforcer magnitude and local preference

Six pigeons were trained on concurrent variable-interval schedules. Sessions consisted of seven components, each lasting 10 reinforcers, with the conditions of reinforcement differing between components. The component sequence was randomly selected without replacement. In Experiment 1, the concurrent-schedule reinforcer ratios in components were all equal to 1.0, but across components reinforcer-magnitude ratios varied from 1:7 through 7:1. Three different overall reinforcer rates were arranged across conditions. In Experiment 2, the reinforcer-rate ratios varied across components from 27:1 to 1:27, and the reinforcer-magnitude ratios for each alternative were changed across conditions from 1:7 to 7:1. The results of Experiment 1 replicated the results for changing reinforcer-rate ratios across components reported by Davison and Baum (2000, 2002): Sensitivity to reinforcer-magnitude ratios increased with increasing numbers of reinforcers in components. Sensitivity to magnitude ratio, however, fell short of sensitivity to reinforcer-rate ratio. The degree of carryover from component to component depended on the reinforcer rate. Larger reinforcers produced larger and longer postreinforcer preference pulses than did smaller reinforcers. Similar results were found in Experiment 2, except that sensitivity to reinforcer magnitude was considerably higher and was greater for magnitudes that differed more from one another. Visit durations following reinforcers measured either as number of responses emitted or time spent responding before a changeover were longer following larger than following smaller reinforcers, and were longer following sequences of same reinforcers than following other sequences. The results add to the growing body of research that informs model building at local levels.

Concurrent schedules: reinforcer magnitude effects

Five pigeons were trained on pairs of concurrent variable-interval schedules in a switching-key procedure. The arranged overall rate of reinforcement was constant in all conditions, and the reinforcer-magnitude ratios obtained from the two alternatives were varied over five levels. Each condition remained in effect for 65 sessions and the last 50 sessions of data from each condition were analyzed. At a molar level of analysis, preference was described well by a version of the generalized matching law, consistent with previous reports. More local analyses showed that recently obtained reinforcers had small measurable effects on current preference, with the most recently obtained reinforcer having a substantially larger effect. Larger reinforcers resulted in larger and longer preference pulses, and a small preference was maintained for the larger-magnitude alternative even after long inter-reinforcer intervals. These results are consistent with the notion that the variables controlling choice have both short- and long-term effects. Moreover, they suggest that control by reinforcer magnitude is exerted in a manner similar to control by reinforcer frequency. Lower sensitivities when reinforcer magnitude is varied are likely to be due to equal frequencies of different sized preference pulses, whereas higher sensitivities when reinforcer rates are varied might result from changes in the frequencies of different sized preference pulses.