The Measurement and Functional Properties of Reinforcer Value in Single-Alternative Responding: A Test of Linear System Theory

Evolutionary theory prediction: Response rate as a joint function of reinforcement rate and reinforcer magnitude

Artificial organisms (AOs) animated by an evolutionary theory of behavior dynamics (ETBD) worked on concurrent interval schedules with a standard reinforcer magnitude on 1 alternative and a range of reinforcer magnitudes on the other. The reinforcer magnitudes on the second alternative were hedonically scaled using the generalized matching law. The AOs then worked on single interval schedules that arranged various combinations of the scaled reinforcer magnitudes and a range of nominal schedule values. This produced bivariate response rate data to which 5 candidate equations were fitted. One equation was found to provide the best description of the bivariate data in terms of percentage of variance accounted for, information criterion value, and residual profile. This equation consisted of 2 factors, 1 entailing the scaled magnitude, 1 entailing the obtained reinforcement rate, and both expressed in the form of exponentiated hyperbolas. The theory's prediction of the bivariate equation, along with additional predictions of the theory, were tested on data from an experiment in which rats pressed levers for various concentrations of sucrose pellets. The bivariate equation predicted by the theory was confirmed, as were all the additional predictions of the theory that could be tested on this data set.

Bias and undermatching in delinquent boys' verbal behavior as a function of their level of deviance

Eighty-one 13- to 14-year-old boys at risk for delinquency (target boys) engaged in brief dyadic conversations with their peer friends. The target boys' verbal behavior was coded into two mutually exclusive content categories, rule-break talk and normative talk. Positive social responses from peer boys for each category of talk were also recorded, and were presumed to reinforce the target boys' verbal behavior. A measure of child deviance was available for each target boy. The generalized matching law was fitted to the target boys' response and time allocation data and provided an excellent description of their verbal behavior, with an expected degree of undermatching and strong bias in favor of normative talk. When the boys' data were separated into groups of increasing child deviance, the matching law continued to provide an excellent description of the boys' verbal behavior regardless of their level of deviance, but undermatching became more severe and bias favoring normative talk became less strong as child deviance increased. Based on a selectionist theory of adaptive behavior dynamics from the basic science, it was suggested that the increasing degree of undermatching might be due to a decline in the reinforcing value of positive social responses with increasing child deviance. It was also suggested that the trend in the bias parameters might be due to different histories of reinforcement and punishment of rule-break and normative behavior for boys characterized by different levels of child deviance.

Toward a mechanics of adaptive behavior: evolutionary dynamics and matching theory statics

One theory of behavior dynamics instantiates the idea that behavior evolves in response to selection pressure from the environment in the form of reinforcement. This computational theory implements Darwinian principles of selection, reproduction, and mutation, which operate on a population of potential behaviors by means of a genetic algorithm. The behavior of virtual organisms animated by this theory may be studied in any experimental environment. The evolutionary theory was tested by comparing the steady-state behavior it generated on concurrent schedules to the description of steady state behavior provided by modern matching theory. Ensemble fits of modern matching theory that enforced its constant-k requirement and the parametric identities required by its equations, accounted for large proportions of data variance, left random residuals, and yielded parameter estimates with values and properties similar to those obtained in experiments with live organisms. These results indicate that the dynamics of the evolutionary theory and the statics of modern matching theory together constitute a good candidate for a mechanics of adaptive behavior.

Undermatching is an emergent property of selection by consequences

A computational theory of selection by consequences [McDowell, J.J, 2004. A computational model of selection by consequences. J. Exp. Anal. Behav. 81, 297-317] was tested by studying the responding of virtual organisms that were animated by the theory on random interval schedules of reinforcement. The theory generated responding by applying principles of selection, reproduction, and mutation to a population of potential behaviors that evolved in response to the selection pressure exerted by reinforcement. The organisms' equilibrium response rates were well described by the modern version of the Herrnstein hyperbola, which includes an exponent on reinforcement rate. Under strong selection pressure this exponent decreased with increasing mutation rate from a value near 1.0 at 1% mutation to an asymptotic value of 0.83 at mutation rates of 10% and greater. This asymptotic value is consistent with values obtained by fitting the equation to data from live organisms responding on single schedules, and with the value of about 0.80 that is expected on the basis of extensive research with live organisms responding on concurrent schedules. These results show that the computational theory is consistent with the modern theory of matching [McDowell, J.J, 2005. On the classic and modern theories of matching. J. Exp. Anal. Behav. 84, 111-127], and that it is a viable candidate for a mathematical dynamics of behavior.

A test of the formal and modern theories of matching

The present study tested a formal, or purely mathematical, theory of matching, and a modern account derived by McDowell (1986) that incorporates deviations from strict matching-bias and sensitivity. Six humans pressed a lever for monetary reinforcers on five concurrent variable interval (VI) schedules of reinforcement. All schedules were presented during each session. The magnitude on one alternative remained constant, and five magnitudes were presented across sessions on the other alternative. To test the formal account, two absolute response rate equations were fitted to the response and reinforcement rates at each alternative at each magnitude. Although the equations accounted for a high percentage of variance, there was a significant negative correlation between the standardized residuals and the predicted response rates. To test the modern account, an ensemble of four equations was fitted to the data. The equations predicted relative and absolute responding, and the independent variables in each equation were adjusted for bias and sensitivity. The equations accounted for a high percentage of variance, and the standardized residuals were not correlated with the predicted response rates. The values of the parameters were consistent with empirical findings and theoretical predictions, including the prediction that k should remain constant across changes in reinforcer magnitude. The results suggest that the formal theory of matching does not describe the data, and that the modern theory may provide an accurate and coherent description of concurrent and single-alternative responding.

Effects of adding a second reinforcement alternative: implications for Herrnstein's interpretation of r(e)

Herrnstein's hyperbola describes the relation between response rate and reinforcer rate on variable-interval (VI) schedules. According to Herrnstein's (1970) interpretation, the parameter r(e) represents the reinforcer rate extraneous to the alternative to which the equation is fitted (the target alternative). The hyperbola is based on an assumption that extraneous reinforcer rate remains constant with changes in reinforcer rate on the target alternative (the constant-r(e) assumption) and that matching with no bias and perfect sensitivity occurs between response and reinforcer ratios. In the present experiment, 12 rats pressed levers for food on a series of 10 VI schedules arranged on the target alternative. Across conditions, six VI values and extinction were arranged on a second alternative. Reinforcer rate on the second alternative, r2, negatively covaried with reinforcer rate on the target alternative for five of the six VI values on the second alternative, and significant degrees of bias and undermatching occurred in response ratios. Given covariation of reinforcer rate on the second and target alternatives, the constant-r(e) assumption can be maintained only by assuming that reinforcer rate from unmeasured background sources, rb, covaries with reinforcer rate on the second alternative such that their sum, r(e), remains constant. In a single-schedule arrangement, however, r(e) equals rb and thus rb is assumed to remain constant, forcing a conceptual inconsistency between single- and concurrent-schedule arrangements. Furthermore, although an alternative formulation of the hyperbola can account for variations in bias and sensitivity, the modified equation also is based on the constant-r(e) assumption and therefore suffers from the same logical problem as the hyperbola when reinforcer rate on the second alternative covaries with reinforcer rate on the target alternative.

On the classic and modern theories of matching

Classic matching theory, which is based on Herrnstein's (1961) original matching equation and includes the well-known quantitative law of effect, is almost certainly false. The theory is logically inconsistent with known experimental findings, and experiments have shown that its central constant-k assumption is not tenable. Modern matching theory, which is based on the power function version of the original matching equation, remains tenable, although it has not been discussed or studied extensively. The modern theory is logically consistent with known experimental findings, it predicts the fact and details of the violation of the classic theory's constant-k assumption, and it accurately describes at least some data that are inconsistent with the classic theory.

Herrnstein??s hyperbolic matching equation and behavioral pharmacology: review and critique

Behavioral pharmacologists have enlisted Herrnstein's (1970) hyperbolic matching equation to understand the behavioral effects of drugs. Herrnstein's hyperbola describes the relation between absolute response rate and reinforcement rate. The equation has two fitted parameters. The parameter k represents the asymptotic response rate, and the parameter r(e) represents the reinforcement rate necessary to obtain half the asymptotic response rate. According to one interpretation of the equation, changes in k should reflect changes in response or motoric variables, and changes in r(e) should reflect changes in reinforcer or motivational variables, or changes in reinforcement from sources extraneous to the instrumental response. We review research that has applied Herrnstein's equation to distinguish the motoric from the motivational effects of drugs, and to identify additional independent variables responsible for drug effects, such as extraneous reinforcement. The validity of inferences about drug effects depends on the consistency of how k and r(e) respond to environmental manipulations: k should change only with response or motoric variables, and r(e) should change with reinforcer or motivational variables and with the rate of extraneous reinforcement. Empirical tests of these predictions, however, have produced inconsistent results. The review suggests that Herrnstein's theory has not fulfilled its promise of identifying the behavioral mechanisms of drug action. Modifications to the equation, known as bias and sensitivity, may explain some of these inconsistent results, and the modified equation may have utility in behavioral pharmacology.