Development of complex, stereotyped behavior in pigeons

Exploring the Influencing Paths of Villagers' Participation in the Creation of Micro-Landscapes: An Integrative Model of Theory of Planned Behavior and Norm Activation Theory

Villager participation has become a key breakthrough in rural landscape governance. Using the theory of planned behavior and the norm activation theory as frameworks, this study adopts the structural equation model to explore the influencing mechanism of villager participation in rural micro-landscapes based on data gathered from 414 villagers in a rural micro-landscape construction survey in Jinjiang, China. The results indicate that (1) integrated planned behavior theory and norm activation theory can better explain the influencing mechanism of villagers' participation in rural micro-landscape construction; (2) perception, norm, attitude, and control dimensions significantly influence villagers' participation behavior intention. The attitude dimension had the greatest influence, followed by the normative and control dimensions, while the perception dimension had the least influence on the procedure; and (3) according to the mediation results, natural environment perception, social environment perception, personal norm, social norm, participation attitude, result awareness, and self-efficacy all exert indirect effects on participation behavior based on villagers' participation behavioral intention. The largest median effect value was result awareness, followed by personal norm, participation attitude, natural environment awareness, self-efficacy, and social norm. This study expands the theoretical framework and research content of planned behavior and clarifies the mechanism of the influencing factors of villagers' participation in rural micro-landscapes, extending the theory of planned behavior to the research field of villagers' participation, which has a guiding role in promoting the co-construction, co-governance, and sharing of rural landscapes.

A Critical Review of the Support for Variability as an Operant Dimension

There is abundant evidence that behavioral variability is more predominant when reinforcement is contingent on it than when it is not, and the interpretation of direct reinforcement of variability suggested by Page and Neuringer, Journal of Experimental Psychology: Animal Behavior Processes, 11(3), 429-452 (1985) has been widely accepted. Even so, trying to identify the underlying mechanisms in the emergence of stochastic-like variability in a variability contingency is intricate. There are several challenges to characterizing variability as directly reinforced, most notably because reinforcement traditionally has been found to produce repetitive responding, but also because directly reinforced variability does not always relate to independent variables the same way as more commonly studied repetitive responding does. The challenging findings in variability experiments are discussed, along with alternative hypotheses on how variability contingencies may engender the high variability that they undeniably do. We suggest that the typical increase in behavioral variability that is often demonstrated when reinforcement is contingent on it may be better explained in terms of a dynamic interaction of reinforcement and extinction working on several specific responses rather than as directly reinforced.

Proprioceptive stimuli and habit formation: Interresponse time mediated behavior in CD-1 mice

The consolidation of behavioral sequences into relatively ballistic habits is thought to involve the formation of stimulus - response associations. Typically, the stimuli in these associations are assumed to be exteroceptive, i.e., external to the organism. However, responses, themselves, also possess stimulus properties that can mediate behavior. Indeed, these "proprioceptive cues" have long been hypothesized to underlie habit formation (Hull, 1934a, 1934b). One such stimulus involves the time durations between responses - a stimulus termed interresponse time (IRT). We hypothesize that IRTs can come to serve as stimuli that differentially control response elements during habit formation. To examine this hypothesis we report on two experiments that asked whether CD-1 mice utilize IRTs to structure behavior in a two-choice environment. In experiment 1, eight mice were exposed to a free-operant concurrent variable-interval (VI) 30-s VI 60-s reinforcement schedule. We found that switch and stay responses were differentially correlated with IRT durations. In Experiment 2 we directly and differentially reinforced stay/switch responses based on IRT durations in a two-lever procedure. For four of the subjects, the probability of receiving reinforcement after switch responses was proportional to IRT duration. For five of the subjects, these reinforcement probabilities were inversely proportional to IRT duration. Regardless, all of our subjects learned to emit IRT-mediated switching behavior that matched the reinforcement contingencies. Together, Experiments 1 and 2 provide the first evidence of which we are aware that IRTs can come to control sequential choice behavior in mice.

Undesirable Choice Biases with Small Differences in the Spatial Structure of Chance Stimulus Sequences

In two-alternative discrimination tasks, experimenters usually randomize the location of the rewarded stimulus so that systematic behavior with respect to irrelevant stimuli can only produce chance performance on the learning curves. One way to achieve this is to use random numbers derived from a discrete binomial distribution to create a 'full random training schedule' (FRS). When using FRS, however, sporadic but long laterally-biased training sequences occur by chance and such 'input biases' are thought to promote the generation of laterally-biased choices (i.e., 'output biases'). As an alternative, a 'Gellerman-like training schedule' (GLS) can be used. It removes most input biases by prohibiting the reward from appearing on the same location for more than three consecutive trials. The sequence of past rewards obtained from choosing a particular discriminative stimulus influences the probability of choosing that same stimulus on subsequent trials. Assuming that the long-term average ratio of choices matches the long-term average ratio of reinforcers, we hypothesized that a reduced amount of input biases in GLS compared to FRS should lead to a reduced production of output biases. We compared the choice patterns produced by a 'Rational Decision Maker' (RDM) in response to computer-generated FRS and GLS training sequences. To create a virtual RDM, we implemented an algorithm that generated choices based on past rewards. Our simulations revealed that, although the GLS presented fewer input biases than the FRS, the virtual RDM produced more output biases with GLS than with FRS under a variety of test conditions. Our results reveal that the statistical and temporal properties of training sequences interacted with the RDM to influence the production of output biases. Thus, discrete changes in the training paradigms did not translate linearly into modifications in the pattern of choices generated by a RDM. Virtual RDMs could be further employed to guide the selection of proper training schedules for perceptual decision-making studies.

Learning new response sequences

Four rats were required to press either a right or left lever to complete various three-response sequences. After extended exposure to a training sequence, subjects were shifted to a new target sequence. The new target sequences always differed from the previous sequence by the response required in the first or last position of the sequence. Subjects were repeatedly exposed to all possible combinations of training and new target sequences. Learning of new sequences occurred more rapidly when the change in the new target sequence was in the last position. Errors persisted longer in new sequences in which the change was in the first position. Extinction of the training sequence occured faster when the change was in the last position. Responses in the last position were considerably more sensitive to the shift to new target sequences than were responses in the first position. Even though response sequences may form new behavioral units from the training sequence, reinforcement and extinction acted differentially on the individual lever presses within new target sequences rather than on the sequences as a whole. These findings support the hypothesis that response strength is determined by contiguity to reinforcement.

Disruption of component processes of spatial working memory by electroconvulsive shock but not magnetic seizure therapy

Self-ordered spatial working memory measures provide important information regarding underlying cognitive strategies, such as stereotypy. This strategy is based on repetitive sequential selection of a spatial pattern once a correct sequence has been identified. We previously reported that electroconvulsive shock (ECS) but not magnetic seizure therapy (MST) impaired performance on a spatial working memory task in a preclinical model. Here we tested the hypothesis that ECS disrupted stereotyped patterns in the selection of spatial stimuli. In a within-subject study design, we assessed the effects of ECS, MST, and sham on stereotypy and reaction time in a preclinical model. Stereotypy was assessed by the correlation of actual and predicted response patterns of spatial stimuli. Predicted patterns were based on performance during baseline sessions. ECS resulted in lower correlations between predicted and actual responses to spatial stimuli in two of the three subjects, and it also disrupted stereotypy. For one subject, there was change in the predictability of the spatial locus of responses between experimental conditions. For all three subjects, reaction time was significantly longer in ECS, relative to MST and sham. This is the first study to examine the effect of ECS, and to contrast the effects of ECS and MST, on spatial working memory component processes. Our preliminary findings show that ECS, but not MST decreased stereotypy and increased reaction time. This line of investigation may have significant implications in our understanding cognitive component processes of memory function and impairment.

Variable sequencing is actively maintained in a well learned motor skill

Variation in sequencing of actions occurs in many natural behaviors, yet how such variation is maintained is poorly understood. We investigated maintenance of sequence variation in adult Bengalese finch song, a learned skill with rendition-to-rendition variation in the sequencing of discrete syllables (i.e., syllable "b" might transition to "c" with 70% probability and to "d" with 30% probability). We found that probabilities of transitions ordinarily remain stable but could be modified by delivering aversive noise bursts following one transition (e.g., "b→c") but not the alternative (e.g., "b→d"). Such differential reinforcement induced gradual, adaptive decreases in probabilities of targeted transitions and compensatory increases in alternative transitions. Thus, the normal stability of transition probabilities does not reflect hardwired premotor circuitry. While all variable transitions could be modified by differential reinforcement, some were less readily modified than others; these were cases that exhibited more alternation between possible transitions than predicted by chance (i.e., "b→d " would tend to follow "b→c " and vice versa). These history-dependent transitions were less modifiable than more stochastic transitions. Similarly, highly stereotyped transitions (which are completely predictable) were not modifiable. This suggests that stochastically generated variability is crucial for sequence modification. Finally, we found that, when reinforcement ceased, birds gradually restored transition probabilities to their baseline values. Hence, the nervous system retains a representation of baseline probabilities and has the impetus to restore them. Together, our results indicate that variable sequencing in a motor skill can reflect an end point of learning that is stably maintained via continual self-monitoring.

Using a lag schedule to increase variability of verbal responding in an individual with autism

Research has shown that reinforcing novel behaviors can increase the number of different ways that an individual behaves (Goetz & Baer, 1973; Pryor, Haag, & O'Reilly, 1969). However, it was not until more recently that researchers began to consider variability to be a reinforceable operant in and of itself (Neuringer, 2002). More specifically, Neuringer suggested that variability can be taught using a Lag x schedule of reinforcement, in which x refers to the number of previous responses from which the current response must differ in order for reinforcement to occur (Page & Neuringer, 1985). The purpose of the present study was to extend one of the first studies of a Lag x schedule on verbal responses with human subjects (Lee, McComas, & Jawor, 2002), by increasing the lag criteria while attempting to address some of methodological limitations of the study. The participant was a 7-year-old male with autism. A changing criterion design was used and results showed that 3 novel responses were acquired and varied according to the lag schedule of reinforcement.

Effects of reinforcing preselected approximations on the topography of the rat's bar press

The effects of different shaping approximations on the topography of the rat's bar press were investigated in two experiments. Behavior was classified into discrete components, and changes in components and their sequential organization were analyzed. Experiment 1 examined response form early in training and found that specific components reinforced during shaping were incorporated into press sequences. Experiment 2 investigated how response form changed when a shaping contingency was relaxed later in training. Two topographies were selected for reinforcement, and both appeared in the press sequences of all subjects by the end of shaping. Subsequently, all variations of bar pressing were reinforced, and neither topography was necessary to satisfy the contingency. Although the frequency of the topographies reinforced during shaping declined for 3 of 4 subjects during this phase, the most frequent press sequence for 2 rats at the end of training included both unnecessary topographies. Variability in press topographies declined when all emitted variants were reinforced. However, all subjects emitted novel response forms throughout training. The results demonstrate that specific response-reinforcer contingencies influence response form by modulating component availability and organization.

Spatiotemporal patterns of behavior produced by variable-interval schedules of reinforcement

The spatiotemporal patterns of behavior exhibited by two pigeons during a variable-interval 15-second schedule of food reinforcement, a variable-interval 5-minute schedule, and then extinction of key pecking were recorded using an apparatus that continuously tracked the position of the bird in the experimental chamber. The variable-interval 15-second schedule produced a close-to-key pattern between reinforcements with two types of regular excursions from the region of the key frequently occurring after reinforcement. Subsequent exposure to the variable-interval 5-minute schedule produced more extended and extremely regular patterns between responses. Reinstatement of the variable-interval 15-second schedule reestablished the close-to-key pattern with regular excursions frequently occurring after reinforcement. During extinction the spatiotemporal patterns that had developed during the variable-interval 5-minute schedule reappeared and gradually dissipated. These patterns may have been a form of superstitious behavior.

Behavioral stereotypy and the generalized matching equation

The development of behavioral stereotypy is a common result of exposure to both response-dependent and response-independent reinforcement procedures. The generalized matching equation and two dynamic versions of that equation, which take into account the time differential between reinforcements and their effect on behavior, predict this outcome of many procedures involving reinforcement. Following from the assumption that distinct response topographies, distinct response sequences, or orientations to distinct stimuli can be treated in the equations as distinct classes of behavior, the equations predict that-at least for matching and undermatching-the behavior class that is most biased relative to other behavior classes of the same type will tend to predominate to the exclusion or near exclusion of those behavior classes.

Choice and behavioral patterning

Ten pigeons pecked left and right keys in a discrete-trials experiment in which access to food was contingent upon changeovers to the right key after particular runs of left-key pecks. In each of three sets of conditions, two run lengths were reinforced according to a concurrent variable-interval schedule: reinforcement followed runs of either 1 or 2, 1 or 4, or 2 or 4 left-key pecks preceding changeovers. The intertrial interval separating successive pecks was varied from .5 to 10.0 sec, and the relative frequency of reinforcement for the shorter of the two reinforced runs was varied from 0 to .75. The contingencies established local behavioral patterning that roughly approximated that required for reinforcement. For a fixed pair of reinforced run lengths, preference for the shorter of the two frequently increased as the intertrial interval increased and therefore as the minimum temporal durations of both reinforced runs increased. Preference for the shorter of the two also increased as its corresponding relative frequency of reinforcement increased. Both of these effects on preference were qualitatively similar to corresponding effects in previous research with two different kinds of reinforced behavioral patterns, interresponse times and interchangeover times. In all these experiments, analytical units were found in the temporal patterns of behavior, not in the behavior immediately contiguous with a reinforcer. It is suggested that a particular local temporal pattern of behavior is established to the extent to which it is repeatedly remembered when reinforcers are delivered, regardless of whether the delivery of a reinforcer is explicitly contingent upon that pattern.

Interval and ratio reinforcement of a complex sequential operant in pigeons

Pigeons were required to produce exactly four pecks on each of two keys in any order for reinforcement. Correct response sequences were reinforced on either fixed-interval two-minute or fixed-ratio four schedules, with each correct sequence treated as a single response. Each pigeon developed a particular dominant sequence that accounted for more than 80% of all sequences. Sequence stereotypy was relatively unaffected by the temporal properties of the fixed-interval and fixed-ratio schedules. Response time (time from the first response in each sequence to the last) was also relatively unaffected by the temporal properties of the schedules. In contrast, response latency (time from end of one sequence to the beginning of the next) was markedly affected by the schedules. Latencies were long early in the interreinforcement interval and got shorter as the interreinforcement interval progressed. These data suggest that stereotyped response sequences become functional behavioral units, resistant to disruption or alteration by reinforcement variables that ordinarily influence the temporal spacing of individual responses.

The operant conditioning of response variability: Free-operant versus discrete-response procedures

The operant conditioning of response variability under free-operant and discrete-response procedures was investigated. Two pigeons received food only if their pattern of four pecks on two response keys differed from the patterns emitted on the two immediately preceding trials. Under the free-operant procedure, the keys remained illuminated and operative throughout each trial. There was little variability in the response patterns that resulted, and the pigeons received fewer than one third of the available reinforcers. Under the discrete-response procedure, a brief timeout period followed each response. Variability increased under this procedure, and the pigeons obtained three fourths of the available reinforcers. Previous successes and failures to produce response variability may have been due to the use or failure to use, respectively, a discrete-response procedure. Respondent effects inherent in the free-operant procedure may encourage the development of response stereotypy and, in turn, prevent the development of response variability.

Maximizing versus matching on concurrent variable-interval schedules

Maximization and matching predictions were examined for a time-based analogue of the concurrent variable-interval variable-ratio schedule. One alternative was a variable interval whose time base operated relatively independent of the schedule chosen, and the other was a discontinuous variable interval for which timing progressed only when selected. Pigeons switched between schedules by pecking a changeover key. The maximization hypothesis predicts that subjects will show a bias toward the discontinuous variable interval and undermatching; however the obtained results conformed closely to the predictions of the matching law. Finally, a quantitative comparison was made of the bias and sensitivity estimates obtained in published concurrent variable-interval variable-ratio analogue studies. Results indicated that only the ratio-based analogue of the concurrent variable interval variable ratio studied by Green, Rachlin, and Hanson (1983) produced significant bias toward the variable-ratio alternative and undermatching, as predicted by reinforcement maximization.

Failure to produce response variability with reinforcement

Two experiments attempted to train pigeons to produce variable response sequences. In the first, naive pigeons were exposed to a procedure requiring four pecks on each of two keys in any order, with a reinforcer delivered only if a given sequence was different from the preceding one. In the second experiment, the same pigeons were exposed to this procedure after having been trained successfully to alternate between two specific response sequences. In neither case did any pigeon produce more than a few different sequences or obtain more than 50% of the possible reinforcers. Stereotyped sequences developed even though stereotypy was not reinforced. It is suggested that reinforcers have both hedonic and informative properties and that the hedonic properties are responsible for sterotyped repetition of reinforced responses, even when stereotypy is negatively related to reinforcer delivery.

Allocation of complex, sequential operants on multiple and concurrent schedules of reinforcement

Pigeons could produce food by pecking exactly four times on each of two keys, in any order. In the first experiment, these response sequences were reinforced on a series of multiple schedules of variable-interval reinforcement. In the second experiment, these response sequences were reinforced on a series of concurrent schedules of reinforcement. In both experiments, highly stereotyped response sequences developed. If these response sequences were treated as individual responses, the resulting data conformed to what is typically reported in studies of multiple and concurrent schedules involving individual responses. For example, behavioral contrast was observed with the multiple schedules, and matching was observed with the concurrent schedules. However, schedule manipulation had no effect on within-sequence characteristics of responses like accuracy, stereotypy, or rate. These data constitute further evidence that response sequences can become functional behavioral units.

Some factors that influence the acquisition of complex, stereotyped, response sequences in pigeons

Two pigeons were required to peck six to nine illuminated response keys. A response on any one of the keys darkened that key. When each key had been darkened, a reinforcer was delivered. No specific sequence of key pecking was ever required. The keys were presented in various matrices: three by two, three by three, horizontal rows, and vertical columns. The keys either presented the same stimulus, white light; or each key presented a different stimulus, a color or form. The results indicated that although there were 720 to 362,880 different sequences that would produce reinforcement, each bird developed a particular, stereotyped sequence that dominated its behavior. Variability among the birds across phases yielded less than 60 sequences, .0001 to 6 percent of the possible sequences. The data suggest that a reinforcement contingency that includes "free choice" of response sequence will produce stereotypical response sequences that function as complex "units" of behavior.

Choice with a fixed requirement for food, and the generality of the matching relation

Pigeons were trained on choice procedures in which responses on each of two keys were reinforced probabilistically, but only after a schedule requirement had been met. Under one arrangement, a fixed-interval choice procedure was used in which responses were not reinforced until the interval was over; then a response on one key would be reinforced, with the effective key changing irregularly from interval to interval. Under a second, fixed-ratio choice procedure, responses on either key counted towards completion of the ratio and then, once the ratio had been completed, a response on the probabilistically selected key would produce food. In one experiment, the schedule requirements were varied for both fixed-interval and fixed-ratio schedules. In the second experiment, relative reinforcement rate was varied. And in a third experiment, the duration of an intertrial interval separating choices was varied. The results for 11 pigeons across all three experiments indicate that there were often large deviations between relative response rates and relative reinforcement rates. Overall performance measures were characterized by a great deal of variability across conditions. More detailed measures of choice across the schedule requirement were also quite variable across conditions. In spite of this variability, performance was consistent across conditions in its efficiency of producing food. The absence of matching of behavior allocation to reinforcement rate indicates an important difference between the present procedures and other choice procedures; that difference raises questions about the specific conditions that lead to matching as an outcome.

Reinforced variability in animals and people: implications for adaptive action

Although reinforcement often leads to repetitive, even stereotyped responding, that is not a necessary outcome. When it depends on variations, reinforcement results in responding that is diverse, novel, indeed unpredictable, with distributions sometimes approaching those of a random process. This article reviews evidence for the powerful and precise control by reinforcement over behavioral variability, evidence obtained from human and animal-model studies, and implications of such control. For example, reinforcement of variability facilitates learning of complex new responses, aids problem solving, and may contribute to creativity. Depression and autism are characterized by abnormally repetitive behaviors, but individuals afflicted with such psychopathologies can learn to vary their behaviors when reinforced for so doing. And reinforced variability may help to solve a basic puzzle concerning the nature of voluntary action.

Variation, repetition, and choice

Experiment 1 investigated the controlling properties of variability contingencies on choice between repeated and variable responding. Pigeons were exposed to concurrent-chains schedules with two alternatives. In the REPEAT alternative, reinforcers in the terminal link depended on a single sequence of four responses. In the VARY alternative, a response sequence in the terminal link was reinforced only if it differed from the n previous sequences (lag criterion). The REPEAT contingency generated low, constant levels of sequence variation whereas the VARY contingency produced levels of sequence variation that increased with the lag criterion. Preference for the REPEAT alternative tended to increase directly with the degree of variation required for reinforcement. Experiment 2 examined the potential confounding effects in Experiment 1 of immediacy of reinforcement by yoking the interreinforcer intervals in the REPEAT alternative to those in the VARY alternative. Again, preference for REPEAT was a function of the lag criterion. Choice between varying and repeating behavior is discussed with respect to obtained behavioral variability, probability of reinforcement, delay of reinforcement, and switching within a sequence.

Differential effects of midazolam and pentylenetetrazole on behavioral repetition and variation

The present study investigated the effects of midazolam and pentylenetetrazole upon repeating and varying sequences of responses. Rats were exposed to a multiple schedule with two components. Under the VARY component, water was provided if the current sequence of four responses differed from each of the five previous ones; under the REPEAT component, water delivery was contingent upon a particular four-response sequence. Each component was signaled by specific exteroceptive stimuli. Overall, sequence variability was affected by midazolam, and to a lesser degree, by pentylenetetrazole. With the REPEAT contingency, drug administration increased behavior variation, thus reducing accuracy; with the VARY contingency, behavior variation and accuracy were not altered. These results were explained on the basis of the disruptive effects of both drugs upon the discriminative control exerted by previous responses within a sequence and by exteroceptive stimuli.

Operant variability: evidence, functions, and theory

Although responses are sometimes easy to predict, at other times responding seems highly variable, unpredictable, or even random. The inability to predict is generally attributed to ignorance of controlling variables, but this article is a review of research showing that the highest levels of behavioral variability may result from identifiable reinforcers contingent on such variability. That is, variability is an operant. Discriminative stimuli and reinforcers control it, resulting in low or high variability, depending on the contingencies. Schedule-of-reinforcement effects are orderly, and choosing to vary or repeat is lawfully governed by relative reinforcement frequencies. The operant nature of variability has important implications. For example, learning, exploring, creating, and problem solving may partly depend on it. Abnormal levels of variability, including those found in psychopathologies such as autism, depression, and attention deficit hyperactivity disorder, may be modified through reinforcement. Operant variability may also help to explain some of the unique attributes of voluntary action.

Reinforcement of variations and repetitions along three independent response dimensions

Reinforcement was presented contingent upon human subjects simultaneously varying three dimensions of an operant response. The response was drawing rectangles on a computer screen. The dimensions were area of the rectangle, its location on the screen, and its shape. In Experiment 1, an experimental group was reinforced for satisfying the three-part variability contingency. A control group was equally reinforced for drawing rectangles but independently of levels of variability. Results showed that the experimental group varied significantly more along each of the dimensions than did the control group. In Experiment 2, another group of subjects was reinforced for repeating instances along one of the dimensions, e.g. repeatedly draw a rectangle in approximately the same location, while simultaneously varying along the other two dimensions. The subjects learned to satisfy these contingencies as well. These results show reinforcement simultaneously and independently controls the variability of three orthogonal dimensions of a response.

Resistance to change of operant variation and repetition

A multiple chained schedule was used to compare the relative resistance to change of variable and fixed four-peck response sequences in pigeons. In one terminal link, a response sequence produced food only if it occurred infrequently relative to 15 other response sequences (vary). In the other terminal link, a single response sequence produced food (repeat). Identical variable-interval schedules operated in the initial links. During baseline, lower response rates generally occurred in the vary initial link, and similar response and reinforcement rates occurred in each terminal link. Resistance of responding to prefeeding and three rates of response-independent food delivered during the intercomponent intervals then was compared between components. During each disruption condition, initial- and terminal-link response rates generally were more resistant in the vary component than in the repeat component. During the response-independent food conditions, terminal-link response rates were more resistant than initial-link response rates in each component, but this did not occur during prefeeding. Variation (in vary) and repetition (in repeat) both decreased during the response-independent food conditions in the respective components, but with relatively greater disruption in repeat. These results extend earlier findings demonstrating that operant variation is more resistant to disruption than is operant repetition and suggest that theories of response strength, such as behavioral momentum theory, must consider factors other than reinforcement rate. The implications of the results for understanding operant response classes are discussed.

Is instrumental variability abnormally high in children exhibiting ADHD and aggressive behavior?

To test whether instrumental behavior of some children with attention deficit hyperactivity disorder (ADHD) is more variable than control subjects, the sequences of responses by three groups of children were compared: (i) those diagnosed with ADHD who lived in a residential treatment facility for children demonstrating aggressive behavior; (ii) age-matched children from the same facility who did not have an ADHD diagnosis but who had also demonstrated abnormal levels of aggression; and (iii) normal children from a local school. The experiment consisted of rewarding children for responding in a computer 'game' on two keys of a keyboard during each of three phases. In phase 1, rewards were provided independently of sequence variability (IND). In phase 2, rewards depended upon highly variable sequences of left and right responses. Phase 3 was a return to the IND contingencies. The results showed that sequence variability was higher in the sequence variability (VARY) phase than in the preceding IND phase, but remained high when the contingencies returned to IND, thus replicating previous findings. In none of the phases did the ADHD children respond more variably than the controls. However, ADHD subjects made more off-task responses than did controls. Thus, although the present research showed differences between ADHD and control subjects, there was no evidence to support higher behavioral variability in the ADHD subjects.

Developing a technology for the use of operant extinction in clinical settings: an examination of basic and applied research

Extinction of operant behavior, which involves terminating the reinforcement contingency that maintains a response, is important to the development, generalization, and reduction of behavior in clinical settings. We review basic and applied research findings on variables that influence the direct and indirect effects of extinction and discuss the potential value of a general technology for the use of extinction. We suggest that current research findings are not sufficient for the development of a comprehensive, applied technology of extinction and provide extensive guidelines for further studies on factors that may affect the course of extinction in clinical settings.

Effects of fixed and variable ratios on human behavioral variability

The effect that ratio schedules of reinforcement had upon variability of responding was investigated in college students. Subjects were paid $0.02 contingent upon completion of eight presses, distributed in any combination across two push buttons; 256 different sequences were possible. Sequence emission was reinforced according to fixed- and variable-ratio schedules. Ratio requirements of 1, 2, 4 and 8 were presented in alternate components of a multiple schedule. The variability engendered by variable-ratio schedules was also compared to that engendered by fixed ratios. Variability increased with ratio size, irrespective of whether the schedule requirement was fixed or variable. The data demonstrate the similarity between the determinants of human and nonhuman variability, and they illustrate the role of ratio size in determining variability in operant behavior.

Acquisition of a spatially defined operant with delayed reinforcement

Two experiments investigated the role of an immediate, response-produced auditory stimulus during acquisition, via delayed reinforcement, of a response selected to control for possible unprogrammed, operandum-related sources of response feedback. Experimentally naive rats were exposed to a delayed-food reinforcement condition, specifically a tandem fixed-ratio 1 differential-reinforcement-of-other-behavior 30-s schedule. The response was defined as breaking a photocell beam located near the ceiling at the rear of the operant conditioning chamber. In Experiment 1, rates of photobeam breaking by each rat increased from near zero, regardless of the presence or absence of a tone that immediately followed the response initiating the delay interval. Though not essential, the tone facilitated response acquisition and resulted in more efficient response patterns at stability. Experiment 2 demonstrated that photobeam-breaking response rates under the delayed reinforcement contingency exceeded those in a preceding baseline condition in which no food was delivered. In addition, upon introduction of the delayed reinforcement procedure, correspondence between response patterns and the requirements of the reinforcement schedule increased over baseline levels in the absence of a food contingency. Together with a previous report of Lattal and Gleeson (1990), the present results suggest that response acquisition with delayed reinforcement is a robust phenomenon that may not depend on a mechanically defined response or an immediate external stimulus change to mediate the temporal gap between response and reinforcer.

Variation and selection: The evolutionary analogy and the convergence of cognitive and behavioral psychology

The empirical and theoretical work of both operant and cognitive researchers has increasingly appealed to evolutionary concepts. In particular, both traditional operant studies of extinction-induced behavior and cognitive investigations of creativity and problem solving converge on the fundamental evolutionary principles of variation and selection. These contemporary developments and their implications for the alleged preparadigmatic status of psychology are discussed.

Behavioral variability and frequency-dependent selection

In Experiment 1, two conditions were compared: (a) a variability schedule in which food reinforcement was delivered for the fourth peck in a sequence that differed from the preceding N four-peck sequences, with the value of N continuously adjusted to maintain reinforcement probability approximately constant; and (b) a control condition in which the variability constraint was dropped but reinforcement probability remained constant. Pigeons responded approximately randomly under the variability schedule but showed strong stereotyped behavior under the control condition. Experiments 2 and 3 tested the idea that variability is the outcome of a type of frequency-dependent selection, namely differential reinforcement of infrequent behavior patterns. The results showed that pigeons alternate when frequency-dependent selection is applied to single pecks because alternation is an easy-to-learn stable pattern that satisfies the frequency-dependent condition. Nevertheless, 2 of 4 pigeons showed random behavior when frequency-dependent selection was applied to two pecks, even though double alternation is a permissible and stable stereotype under these conditions. It appears that random behavior results when pigeons are unable to acquire the stable stereotyped behavior under a given frequency-dependent schedule.

Effects of self-generated rules on the development of schedule-controlled behavior

College students responded under a multiple differential-reinforcement-of-low-rate 5-s fixed-ratio 8 schedule, with components alternating every 2 min. After 40 programmed minutes of acquisition and 12 min of maintenance, without notice, both schedules changed to extinction for 28 min. During acquisition, between alternations of the multiple schedule, some subjects were asked to develop rules describing the schedule contingencies. Other subjects were given these same rules between alternations, and a third group neither received nor were asked to develop rules. By the end of the acquisition phase, self-generated-rule subjects were more likely to show schedule-typical behavior than were subjects not asked to generate rules. The behavior of those given rules was similar to those asked to generate rules at the end of acquisition, but yoked-rule subjects acquired schedule-typical behavior at a quicker rate. By the end of extinction, during the period corresponding to the previous fixed-ratio interval, all no-rule subjects who had earned points during acquisition and maintenance were responding at a rate of less than 30 responses per minute. Only 3 of the 9 self-generated-rule subjects and 2 of the 5 yoked-rule subjects were similarly responding at this low rate. Results suggest that asking subjects to develop self-rules facilitates acquisition, but can retard extinction. Results also suggest that self-generated rules function similarly to external rules.