Matching, statistics, and common sense

Concurrent schedules: How responses per reinforcer affects estimates of sensitivity to reinforcement

Conventionally, when choice is measured under concurrent schedules, all responses are included. However, the class of all responses consists of 2 sub-classes which are discriminable by their properties: Reinforced responses always equal obtained reinforcers, while unreinforced responses are free to vary. As a result, the inclusion of reinforced responses in choice measures results in sensitivity values in generalized matching that are biased toward larger values, and this bias becomes extreme in some combinations of overall response and reinforcer rates. Different ways of varying concurrent-schedule values also affect estimated sensitivity and the linearity between choice based on all responses and reinforcer ratios. To avoid spurious results and comparisons, and as a matter of good practice, generalized-matching fits and measures should be done using only unreinforced responses.

Special issue on computational models of classical conditioning guest editors' introduction

In the present special issue, the performance of current computational models of classical conditioning was evaluated under three requirements: (1) Models were to be tested against a list of previously agreed-upon phenomena; (2) the parameters were fixed across simulations; and (3) the simulations used to test the models had to be made available. These requirements resulted in three major products: (a) a list of fundamental classical-conditioning results for which there is a consensus about their reliability; (b) the necessary information to evaluate each of the models on the basis of its ordinal successes in accounting for the experimental data; and (c) a repository of computational models ready to generate simulations. We believe that the contents of this issue represent the 2012 state of the art in computational modeling of classical conditioning and provide a way to find promising avenues for future model development.

Two modern developments in matching theory

Matching theory is a mathematical theory of choice behavior, parts of which have been shown to hold in natural human environments and to have important therapeutic applications. Two modern developments in matching theory are discussed in this article. The first is the mathematical description of behavior in asymmetrical choice situations, which are situations where different reinforcers and/or different behaviors are associated with concurrently available response alternatives. Most choice situations in natural human environments are probably asymmetrical. The second development in matching theory is the mathematical description of a tendency toward indifferent responding in all choice situations. Behavior in asymmetrical choice situations and the tendency toward indifferent responding in all choice situations can be described by modifications of the matching equations, which change the equations from lines into power functions. These modern forms have been extraordinarily successful in describing behavior in choice situations, and are the forms most likely to accurately describe human behavior in naturally occurring environments.

Matching theory in natural human environments

Matching theory is a mathematical account of behavior, many aspects of which have been confirmed in laboratory experiments with nonhuman and human subjects. The theory asserts that behavior is distributed across concurrently available response alternatives in the same proportion that reinforcement is distributed across those alternatives. The theory also asserts that behavior on a single response alternative is a function not only of reinforcement contingent on that behavior, but also of reinforcement contingent on other behaviors and of reinforcement delivered independently of behavior. These assertions constitute important advances in our understanding of the effects of reinforcement on behavior. Evidence from the applied literature suggests that matching theory holds not only in laboratory environments, but also in natural human environments. In addition, the theory has important therapeutic implications. For example, it suggests four new intervention strategies, and it can be used to improve treatment planning and management. Research on matching theory illustrates the progression from laboratory experimentation with nonhuman subjects to therapeutic applications in natural human environments.

The identities hidden in the matching laws, and their uses

Various theoretical equations have been proposed to predict response rate as a function of the rate of reinforcement. If both the rate and probability of reinforcement are considered, a simple identity, defining equation, or "law" holds. This identity places algebraic constraints on the allowable forms of our mathematical models and can help identify the referents for certain empirical or theoretical coefficients. This identity can be applied to both single and compound schedules of reinforcement, absolute and relative measures, and to local, global and overall rates and probabilities. The rate matching equations of Hernstein and Catania appear to have been approximations to, and to have been evolving toward, one form of this algebraic identity. Estimates of the bias and sensitivity terms in the generalized ratio and logarithmic matching models are here held to be averaging artifacts arising from fitting procedures applied to models that violate or conceal the underlying identities.

Concurrent-schedule performance: Effects of relative and overall reinforcer rate

Six pigeons were trained to respond on two keys, each of which provided reinforcers on an arithmetic variable-interval schedule. These concurrent schedules ran nonindependently with a 2-s changeover delay. Six sets of conditions were conducted. Within each set of conditions the ratio of reinforcers available on the two alternatives was varied, but the arranged overall reinforcer rate remained constant. Each set of conditions used a different overall reinforcer rate, ranging from 0.22 reinforcers per minute to 10 reinforcers per minute. The generalized matching law fit the data from each set of conditions, but sensitivity to reinforcer frequency (a) decreased as the overall reinforcer rate decreased for both time allocation and response allocation based analyses of the data. Overall response rates did not vary with changes in relative reinforcer rate, but decreased with decreases in overall reinforcer rate. Changeover rates varied as a function of both relative and overall reinforcer rates. However, as explanations based on changeover rate seem unable to deal with the changes in generalized matching sensitivity, discrimination accounts of choice may offer a more promising interpretation.

Concurrent variable-ratio schedules: Implications for the generalized matching law

Rats' responses were reinforced on concurrent variable-ratio variable-ratio schedules in which responses on one lever incremented the ratio counter and responses on a second lever changed the schedule and correlated stimulus. The relative frequency of reinforcement was varied from .10 to .99. In one set of conditions, responding on the main lever incremented both ratio counters, but reinforcement required a response in the presence of the stimulus correlated with the ratio that had been completed. In a second set of conditions, responses on the main lever incremented only the ratio correlated with the stimulus that was currently present. When main-lever responses incremented both ratio counters, subjects distributed responding and time in a manner consistent with the generalized matching law. When responses on the main lever incremented only the schedule currently in effect, the rats responded almost exclusively on the schedule producing the higher frequency of reinforcement. These results extend the applicability of the generalized matching law to dependent ratio schedules.

Relative reinforcer rates and magnitudes do not control concurrent choice independently

One assumption of the matching approach to choice is that different independent variables control choice independently of each other. We tested this assumption for reinforcer rate and magnitude in an extensive parametric experiment. Five pigeons responded for food reinforcement on switching-key concurrent variable-interval variable-interval schedules. Across conditions, the ratios of reinforcer rates and of reinforcer magnitudes on the two alternatives were both manipulated. Control by each independent variable, as measured by generalized-matching sensitivity, changed significantly with the ratio of the other independent variable. Analyses taking the model-comparison approach, which weighs improvement in goodness-of-fit against increasing number of free parameters, were inconclusive. These analyses compared a model assuming constant sensitivity to magnitude across all reinforcer-rate ratios with two alternative models. One of those alternatives allowed sensitivity to magnitude to vary freely across reinforcer-rate ratios, and was less efficient than the common-sensitivity model for all pigeons, according to the Schwarz-Bayes information criterion. The second alternative model constrained sensitivity to magnitude to be equal for pairs of reinforcer-rate ratios that deviated from unity by proportionately equal amounts but in opposite directions. This model was more efficient than the common-magnitude-sensitivity model for 2 of the pigeons, but not for the other 3. An analysis of variance, carried out independently of the generalized-matching analysis, also showed a significant interaction between the effects of reinforcer rate and reinforcer magnitude on choice. On balance, these results suggest that the assumption of independence inherent in the matching approach cannot be maintained. Relative reinforcer rates and magnitudes do not control choice independently.

Sensitivity of conditional-discrimination performance to within-session variation of reinforcer frequency

The present experiment developed a methodology for assessing sensitivity of conditional-discrimination performance to within-session variation of reinforcer frequency. Four pigeons responded under a multiple schedule of matching-to-sample components in which the ratio of reinforcers for correct S1 and S2 responses was varied across components within session. Initially, five components, each arranging a different reinforcer-frequency ratio (from 1:9 to 9:1), were presented randomly within a session. Under this condition, sensitivity to reinforcer frequency was low. Sensitivity failed to improve after extended exposure to this condition, and under a condition in which only three reinforcer-frequency ratios were varied within session. In a later condition, three reinforcer-frequency ratios were varied within session, but the reinforcer-frequency ratio in effect was differentially signaled within each component. Under this condition, values of sensitivity were similar to those traditionally obtained when reinforcer-frequency ratios for correct responses are varied across conditions. The effects of signaled vs. unsignaled reinforcer-frequency ratios were replicated in two subsequent conditions. The present procedure could provide a practical alternative to parametric variation of reinforcer frequency across conditions and may be useful in characterizing the effects of a variety of manipulations on steady-state sensitivity to reinforcer frequency.

Concurrent schedule control of human observing during auditory vigilance

Six adult humans experienced a vigilance task which required them to detect auditory signals and report their occurrence by pressing a button. A subject sat on a chair with his or her head between two small panels. When the subjects pressed his or her head against one of the panels, white noise was presented from a speaker set behind the panel. Signals for the left speaker were presented according to a variable-interval schedule and those for the right speaker were presented according to another variable-interval schedule (concurrent variable-interval variable-interval schedule). All subjects showed systematic variations of time allocation, although slopes of choice proportions showed severe undermatching according to the generalized matching law. The remaining three subjects, who were first exposed to some concurrent schedules, also showed such a tendency. Then, they experienced a training in which they were exposed to each variable-interval schedule successively before being exposed to them concurrently. This procedure reduced the amount of undermatching. These findings were in good agreement with previous studies reporting conformity of human observing responses to the generalized matching law.

Determinants of human performance on concurrent schedules

Six experiments, each with 5 human adults, were conducted to investigate the determinants of human performance on multiple concurrent variable-interval schedules. A two-key procedure was employed in which subjects' key presses produced points exchangeable for money. Variables manipulated across experiments were (a) changeover delay (Experiments 2, 4, and 6), (b) ordinal cues related to scheduled reinforcement frequencies (Experiments 3 and 4), and (c) instructions describing the ordinal relations between schedule-correlated stimuli and scheduled reinforcement frequency (Experiments 5 and 6). The performances of only 13 of the 30 subjects could be described by the generalized matching equation and were within a range of values typical of those reported in the animal literature. Eight subjects showed indifference, 9 undermatched, 7 approximated matching, 3 overmatched, and a further 3 responded exclusively to the richer component of the concurrent schedules. These differing modes of responding were closely related to the different types of performance rules reported by subjects in postexperimental questionnaires. The results are in good agreement with those from studies of human performance on single schedules, suggesting that rule-governed behavior, in interaction with contingencies, may be an important determinant of human choice.