Effects of different acquisition procedures on response variability

A Critical Review of the Support for Variability as an Operant Dimension

There is abundant evidence that behavioral variability is more predominant when reinforcement is contingent on it than when it is not, and the interpretation of direct reinforcement of variability suggested by Page and Neuringer, Journal of Experimental Psychology: Animal Behavior Processes, 11(3), 429-452 (1985) has been widely accepted. Even so, trying to identify the underlying mechanisms in the emergence of stochastic-like variability in a variability contingency is intricate. There are several challenges to characterizing variability as directly reinforced, most notably because reinforcement traditionally has been found to produce repetitive responding, but also because directly reinforced variability does not always relate to independent variables the same way as more commonly studied repetitive responding does. The challenging findings in variability experiments are discussed, along with alternative hypotheses on how variability contingencies may engender the high variability that they undeniably do. We suggest that the typical increase in behavioral variability that is often demonstrated when reinforcement is contingent on it may be better explained in terms of a dynamic interaction of reinforcement and extinction working on several specific responses rather than as directly reinforced.

Unexpected downshifts in reward magnitude induce variation in human behavior

We investigated how changes in outcome magnitude affect behavioral variation in human volunteers. Our participants entered strings of characters using a computer keyboard, receiving feedback (gaining a number of points) for any string at least ten characters long. During a "surprise" phase in which the number of points awarded was changed, participants only increased their behavioral variability when the reward value was downshifted to a lower amount, and only when such a shift was novel. Upshifts in reward did not have a systematic effect on variability.

The effects of delayed reinforcement on variability and repetition of response sequences

Four experiments examined the effects of delays to reinforcement on key peck sequences of pigeons maintained under multiple schedules of contingencies that produced variable or repetitive behavior. In Experiments 1, 2, and 4, in the repeat component only the sequence right-right-left-left earned food, and in the vary component four-response sequences different from the previous 10 earned food. Experiments 1 and 2 examined the effects of nonresetting and resetting delays to reinforcement, respectively. In Experiment 3, in the repeat component sequences had to be the same as one of the previous three, whereas in the vary component sequences had to be different from each of the previous three for food. Experiment 4 compared postreinforcer delays to prereinforcement delays. With immediate reinforcement sequences occurred at a similar rate in the two components, but were less variable in the repeat component. Delays to reinforcement decreased the rate of sequences similarly in both components, but affected variability differently. Variability increased in the repeat component, but was unaffected in the vary component. These effects occurred regardless of the manner in which the delay to reinforcement was programmed or the contingency used to generate repetitive behavior. Furthermore, the effects were unique to prereinforcement delays.

Response stability and variability induced in humans by different feedback contingencies

In two experiments, the behavioral effects of different response-feedback contingencies were examined with a task requiring human subjects to repeatedly type three-key sequences on a computer keyboard. In Experiment 1, the subjects first received positive feedback for response variability, followed by no feedback, or vice versa In Experiment 2, the subjects first received positive feedback for response variability, followed by response-independent positive feedback, or vice versa. Response stability and variability were examined using different measures, such as percentage of trials meeting the variability criteria, frequency of use of the different response alternatives, and autocorrelations as an index of response randomness. The subjects' behavior in the first phase in each condition came to reflect the current feedback contingency. Depending on the measure examined, responding after each contingency change was characterized by both response stability and decreases or increases in response variability. The collective results are discussed in the framework of previous animal and human studies on behavioral stability and variability.

Operant variability: evidence, functions, and theory

Although responses are sometimes easy to predict, at other times responding seems highly variable, unpredictable, or even random. The inability to predict is generally attributed to ignorance of controlling variables, but this article is a review of research showing that the highest levels of behavioral variability may result from identifiable reinforcers contingent on such variability. That is, variability is an operant. Discriminative stimuli and reinforcers control it, resulting in low or high variability, depending on the contingencies. Schedule-of-reinforcement effects are orderly, and choosing to vary or repeat is lawfully governed by relative reinforcement frequencies. The operant nature of variability has important implications. For example, learning, exploring, creating, and problem solving may partly depend on it. Abnormal levels of variability, including those found in psychopathologies such as autism, depression, and attention deficit hyperactivity disorder, may be modified through reinforcement. Operant variability may also help to explain some of the unique attributes of voluntary action.

An optimal period for setting sustained variability levels

In two experiments, we investigated how explicit reinforcement of highly variable behavior at different points in training affected performance after the requirement was eliminated. Two versions of a computer game, differing in the number of possible solution paths, were used. In each, an optimal period of training for producing sustained high variability was found. Exposure to a high lag requirement shortly after acquisition sustained variability. Rewarding variability at other times did not have a sustained effect. The implications for learning and problem solving are discussed.

Comparing choices and variations in people and rats: two teaching experiments

Two pairs of experiments enabled students to compare their own operant behaviors with those of rats. The students played computer games for points, and the rats pressed levers for food. The first pair of experiments showed that, under concurrent schedules of reinforcement, relative frequencies of choices between two alternatives increased linearly in rats and people as functions of relative frequencies of reinforcement, with similar biases and undermatching observed in both species. The second pair of experiments showed that behavioral variability was controlled by reinforcers contingent on variability, this again true for both species. These experiments helped demonstrate the relevance of animal operant research to an explanation of human operant behavior.