Reinforcement of variations and repetitions along three independent response dimensions

Effects of variability requirements on difficult sequence learning

The present study investigated the effects of variability requirements on learning difficult target sequences in humans. Twenty university students emitted five-response sequences. For the experimental groups, 30 nontarget sequences were reinforced according to the Lag-10 variation criterion or the Lag-3 repetition criterion across conditions. For the control groups, the probability of reinforcers for nontarget sequences was yoked to that obtained by the experimental groups. In addition, for both groups, two difficult target sequences were continuously reinforced. U values were higher with the Lag-10 variation criterion than with the Lag-3 repetition criterion for the experimental groups and were unsystematic for the control groups. Higher U values ​​were accompanied by a random pattern in the emission of nontarget sequences for all groups. Higher levels of variability, regardless of whether they were directly produced by reinforcement or were contingency induced, facilitated learning of difficult target sequences.

An investigation of resurgence of reinforced behavioral variability in humans

The present study examined resurgence of reinforced variability in college students, who completed a 3-phase computer-based variability task. In the first phase, baseline, points were delivered for drawing rectangles that sufficiently differed from previous rectangles in terms of a target dimension (size or location, counterbalanced) but were sufficiently similar in terms of the alternative dimension. In the second phase, alternative, points were only delivered for rectangles that were sufficiently different in terms of the alternative dimension, but repetitive in terms of the target dimension. In the third phase, extinction, no points were delivered. In baseline, participants made rectangles that were highly varied in terms of the target dimension and less varied in terms of the alternative dimension, and vice versa in the alternative phase. During extinction, levels of variability increased for the target dimension, providing evidence for resurgence of reinforced variability of a specific dimension of behavior. However, levels of variability also remained high for the alternative dimension, indicating that extinction-induced response variability may also have impacted the results. Although future research is needed to explore other explanations, the results of this study replicate prior research with pigeons and provide some support for the notion of variability as an operant.

Generalization of learned variability across multiple dimensions in humans

This study examined whether trained variability would generalize across dimensions of the target response. Two experiments used a computerized rectangle drawing task that required participants to click and drag a mouse cursor to create rectangles on a computer screen. In Experiment 1, one group received points when successive rectangles varied in their size, shape and location (VAR), another group were yoked to the VAR group and received points that were allocated to them using a yoking procedure (YOKE), regardless of the variability in the size, shape or location of the rectangle drawn. Variability was higher for a dimension when variability on that dimension was directly reinforced. In Experiment 2, three groups of participants received points when rectangles varied on two dimensions; each group differed in the two dimensions that required variation. Variability was again higher for the reinforced dimensions for two of the three groups. Comparison with the YOKE group showed that the variability on those dimensions where variability was not directly reinforced was affected by reinforcement for variability on the other dimensions. Specifically, the variability in Shape and Location was significantly higher when these two dimensions occurred with other dimensions where variability was reinforced (as in Experiment 2) compared to when they were not required to vary (as in the YOKE group). This suggests that, for these two groups, the reinforced variability on the other two dimensions generalized to the third dimension. Implications of this finding to our understanding of factors that promote behavioral variability are discussed.

Information: theory, brain, and behavior

In the 65 years since its formal specification, information theory has become an established statistical paradigm, providing powerful tools for quantifying probabilistic relationships. Behavior analysis has begun to adopt these tools as a novel means of measuring the interrelations between behavior, stimuli, and contingent outcomes. This approach holds great promise for making more precise determinations about the causes of behavior and the forms in which conditioning may be encoded by organisms. In addition to providing an introduction to the basics of information theory, we review some of the ways that information theory has informed the studies of Pavlovian conditioning, operant conditioning, and behavioral neuroscience. In addition to enriching each of these empirical domains, information theory has the potential to act as a common statistical framework by which results from different domains may be integrated, compared, and ultimately unified.

Operant variability: a conceptual analysis

Some researchers claim that variability is an operant dimension of behavior. The present paper reviews the concept of operant behavior and emphasizes that differentiation is the behavioral process that demonstrates an operant relation. Differentiation is conceived as change in the overlap between two probability distributions: the distribution of reinforcement probability as a function of some response property (S distribution) and the probability distribution of the response property itself (R distribution). This concept implies that the differentiation process can be measured only if S distribution and R distribution are both established on the same response property. To determine whether the differentially reinforced behavioral variability fits the proposed concept of operant behavior, I examine the main procedures (lag n and threshold procedures) and the main dependent variable (U value) employed in the studies of operant variability. Because lag n and threshold procedures establish their S distributions on properties distinct from U value, differentiation cannot be measured over the change in U value. I conclude that studies of operant variability have failed to provide a direct demonstration that variability is an operant dimension of behavior. Hence, studies in which measures of variability provide a basis to measure differentiation can better support the claim that variability is an operant dimension of behavior.

A novel task for the investigation of action acquisition

We present a behavioural task designed for the investigation of how novel instrumental actions are discovered and learnt. The task consists of free movement with a manipulandum, during which the full range of possible movements can be explored by the participant and recorded. A subset of these movements, the 'target', is set to trigger a reinforcing signal. The task is to discover what movements of the manipulandum evoke the reinforcement signal. Targets can be defined in spatial, temporal, or kinematic terms, can be a combination of these aspects, or can represent the concatenation of actions into a larger gesture. The task allows the study of how the specific elements of behaviour which cause the reinforcing signal are identified, refined and stored by the participant. The task provides a paradigm where the exploratory motive drives learning and as such we view it as in the tradition of Thorndike [1]. Most importantly it allows for repeated measures, since when a novel action is acquired the criterion for triggering reinforcement can be changed requiring a new action to be discovered. Here, we present data using both humans and rats as subjects, showing that our task is easily scalable in difficulty, adaptable across species, and produces a rich set of behavioural measures offering new and valuable insight into the action learning process.

Reinforced variability in animals and people: implications for adaptive action

Although reinforcement often leads to repetitive, even stereotyped responding, that is not a necessary outcome. When it depends on variations, reinforcement results in responding that is diverse, novel, indeed unpredictable, with distributions sometimes approaching those of a random process. This article reviews evidence for the powerful and precise control by reinforcement over behavioral variability, evidence obtained from human and animal-model studies, and implications of such control. For example, reinforcement of variability facilitates learning of complex new responses, aids problem solving, and may contribute to creativity. Depression and autism are characterized by abnormally repetitive behaviors, but individuals afflicted with such psychopathologies can learn to vary their behaviors when reinforced for so doing. And reinforced variability may help to solve a basic puzzle concerning the nature of voluntary action.

Response stability and variability induced in humans by different feedback contingencies

In two experiments, the behavioral effects of different response-feedback contingencies were examined with a task requiring human subjects to repeatedly type three-key sequences on a computer keyboard. In Experiment 1, the subjects first received positive feedback for response variability, followed by no feedback, or vice versa In Experiment 2, the subjects first received positive feedback for response variability, followed by response-independent positive feedback, or vice versa. Response stability and variability were examined using different measures, such as percentage of trials meeting the variability criteria, frequency of use of the different response alternatives, and autocorrelations as an index of response randomness. The subjects' behavior in the first phase in each condition came to reflect the current feedback contingency. Depending on the measure examined, responding after each contingency change was characterized by both response stability and decreases or increases in response variability. The collective results are discussed in the framework of previous animal and human studies on behavioral stability and variability.

Operant variability: evidence, functions, and theory

Although responses are sometimes easy to predict, at other times responding seems highly variable, unpredictable, or even random. The inability to predict is generally attributed to ignorance of controlling variables, but this article is a review of research showing that the highest levels of behavioral variability may result from identifiable reinforcers contingent on such variability. That is, variability is an operant. Discriminative stimuli and reinforcers control it, resulting in low or high variability, depending on the contingencies. Schedule-of-reinforcement effects are orderly, and choosing to vary or repeat is lawfully governed by relative reinforcement frequencies. The operant nature of variability has important implications. For example, learning, exploring, creating, and problem solving may partly depend on it. Abnormal levels of variability, including those found in psychopathologies such as autism, depression, and attention deficit hyperactivity disorder, may be modified through reinforcement. Operant variability may also help to explain some of the unique attributes of voluntary action.