Observing and conditioned reinforcement

Selective associations produced solely with appetitive contingencies: the stimulus-reinforcer interaction revisited

In studies reporting stimulus-reinforcer interactions in traditional conditioning paradigms, when a tone-light compound was associated with food the light gained stimulus control, but when the compound was paired with shock avoidance the tone gained control. However, the physical nature of the reinforcer-related events (food vs. shock) presented in the presence of the tone-light compound was always confounded with the conditioned hedonic value of the compound's presence relative to its absence. When the compound was paired with shock, its presence was negative relative to its absence (which was shock-free). In contrast, when the compound was paired with food, its presence was positive relative to its absence (which was food-free). The present experiment dealt with this confounding effect by conditioning a tone-light compound to be positive or negative, relative to its absence, solely with food reinforcement. One group of rats received food for responding in the presence of the tone-light compound and no food in its absence. The other group also responded in the presence of the compound, but received food only in its absence. These rats were trained on a chained schedule in which responding in the presence of the tone-light compound produced a terminal link signaled by the absence of the compound; responding ceased in the terminal link because it delayed food delivery. In a test session to assess stimulus control by the elements of the compound, tone and light were presented separately under extinction conditions. Rats that had been exposed to a positive correlation between food and the compound emitted almost double the responses in the presence of the light as in the presence of the tone. In comparison, rats that had been exposed to a negative correlation emitted only two thirds as many responses in the presence of the light as in the presence of the tone. Because this selective association was produced using only food, it appears that the contingencies under which a reinforcer is presented, rather than (or as well as) its physical properties, can generate the selective associations previously attributed to "stimulus-reinforcer interactions." This could mean that regardless of the class of reinforcer that ultimately maintains responding (appetitive or aversive), the contingency-generated hedonic value of the compound stimulus may influence the dominant modality of stimulus control.

The effects of diazepam and triazolam on repeated acquisition and performance of response sequences with an observing response

Drugs often disrupt the acquisition of new response sequences at doses that fail to disrupt the performance of a previously acquired response sequence. This selective drug effect may result from differences in the control exerted by the stimuli presented after each response in the acquisition and performance sequences. To examine the function of these stimuli, an observing procedure was incorporated into a multiple schedule of repeated acquisition and performance of response sequences, in which stimulus presentations were contingent upon an observing response. Three experiments were conducted with humans. Experiment 1 compared responding with and without the observing contingency. No difference was found in the overall percentage of errors across the two conditions. Within the observing condition, observing behaviour was maintained in the acquisition component as long as errors occurred, but was not maintained in the performance component. Experiment 2 examined whether a contingency that increased errors also would increase observing in both the acquisition and performance components. Specifically, reinforcer delivery in each component was contingent upon emitting 10 correct responses and one, two, or four errors. Observing responses increased in the acquisition component as the error requirement increased, whereas observing responses in the performance component increased only when the error requirement was four. Experiment 3 assessed the effects of diazepam (0, 7.5, 15, and 30 mg/70 kg, p.o.) and triazolam (0, 0.375, and 0.75 mg/70 kg, p.o.) on repeated acquisition and performance baselines with the observing contingency. Selective drug effects were obtained in this modified procedure; that is, the percentage of errors in the acquisition component increased at doses that failed to affect the percentage of errors in the performance components. Importantly, drug effects were selective, even though observing responses were not emitted in the performance component and, hence, the stimulus presentations did not occur in that component. These findings suggest that alternative explanations for these differential effects are needed; in that regard, a response-unit account of the selective drug effects is discussed.

Observing behavior in a computer game

Contingencies studied in lever-pressing procedures were incorporated into a popular computer game, "Star Trek," played by college students. One putative reinforcer, the opportunity to destroy Klingon invaders, was scheduled independently of responding according to a variable-time schedule that alternated unpredictably with equal periods of Klingon unavailability (mixed variable time, extinction schedule of reinforcement). Two commands ("observing responses") each produced stimuli that were either correlated or uncorrelated with the two components. In several variations of the basic game, an S-, or bad news, was not as reinforcing as an S+, or good news. In addition, in other conditions for the same subjects observing responses were not maintained better by bad news than by an uninformative stimulus. In both choices, more observing tended to be maintained by an S- for response-independent Klingons when its information could be (and was) used to advantage with respect to other types of reinforcement in the situation (Parts 1 and 2) than when the information could not be so used (Part 3). The findings favor the conditioned reinforcement hypothesis of observing behavior over the uncertainty-reduction hypothesis. This extends research to a more natural setting and to multialternative concurrent schedules of events of seemingly intrinsic value.

Learning and rhythmic human EMG in ecological perspective

Previous evidence of strong interactions between learning and human treadmill locomotion led to a simplified system for studying learned rhythms in a framework of behavioral ecology. Motor control combined with instrumental conditioning in a rhythmic hand task with repeating trials, blocks, and complete regimens. Regimen contexts differed with respect to the pattern of stimulation before and after an electromyographic (EMG) response. Both an antecedent stimulus (a light flash) and a consequent stimulus (a tone indicating success or failure) were necessary for conditioning. Arguments were given for defining reinforcement as a composite of interdependent and size-scaled processes, some including knowledge of results, instead of as a single event after a response.

Choice with uncertain outcomes: conditioned reinforcement effects

Pigeons responded on concurrent chains with equal initial- and terminal-link durations. In all conditions, the terminal links of one chain ended reliably in reinforcement; the terminal links on the alternative chain ended in either food or blackout. In Experiment 1, the terminal-link stimuli were correlated with (signaled) the outcome, and the durations of the initial and terminal links were varied across conditions. Preference did not vary systematically across conditions. In Experiment 2, terminal-link durations were varied under different stimulus conditions. The initial links were variable-interval 80-s schedules. Preference for the reliable alternative was generally higher in unsignaled than in signaled conditions. Preference increased with terminal-link durations only in the unsignaled conditions. There were no consistent differences between conditions with and without a common signal for reinforcement on the two chains. In the first series of conditions in Experiment 3, a single response was required in the initial links, and the stimulus conditions during 50-s terminal links were varied. Preference for the reliable outcome approached 1.0 in unsignaled conditions and was considerably lower (below .50 for 3 of 5 subjects) in signaled conditions. In a final series of signaled conditions with relatively long terminal links, preference varied with duration of the initial links. The results extend previous findings and are discussed in terms of the delay reduction signaled by terminal-link stimuli.

Conditioned reinforcement by conditional discriminative stimuli

A concurrent-chains schedule was used to examine how a delay to conditional discriminative stimuli affects conditioned reinforcement strength. Pigeons' key-peck responses in the initial link produced either of two terminal links according to independent variable-interval 30-s schedules. Each terminal link involved an identical successive conditional discrimination and was segmented into three links: a delay interval (green), a color conditional discriminative stimulus (blue or red), and a line conditional discriminative stimulus (vertical or horizontal lines). Food delivery occurred 45 s after entering the terminal link with a probability of .5, but its conditional probability (1.0 or 0) depended on the combination of the color and the line stimuli. One of the color stimuli occurred independently of further responding, 5 s after entry into the right terminal link, but it occurred 35 s after entry into the left terminal link. One of the line stimuli occurred independently of responding 40 s after entry into either terminal link, synchronized with the offset of the color stimulus. The initial-link relative response rate for the right was consistently higher in comparison with a control condition in which the color stimuli occurred 20 s after entry into either terminal link. The preference for the short delay to the color conditional discriminative stimuli suggests the possibility of conditioned reinforcement by information about the relation between the line conditional discriminative stimuli and the outcomes.

Measurement of cognitive abilities in senescent animals

The behavioral paradigms used in investigating for differences in the cognitive abilities of young and aged animals are critically reviewed with regard to their power to discriminate between young and aged animals' mental capacity. Irrespective of the kind of task, geronto-behavioral research especially is afflicted with difficulties in controlling motivational and emotional influences on cognitive processes. It is hypothesized-somewhat provocatively-that most of the findings indicating an age-related decline are better attributed to the altered motivational status and/or emotional reactiveness than to impaired cognitive processes of senescent animals. Of the common tasks used in this field, it is concluded that complex mazes and different delayed response tasks seem to represent appropriate paradigms in order to study changed capacities in short- and long-term memory (working- and reference-memory, respectively).

Operant conditioning in motor and neural integration

Empirical and theoretical reasons were given to investigate operant conditioning in a new, integrative approach within motor control physiology. Elements of inborn and learned behavior were presented in a framework specifying their stimuli and responses. The operant was redefined as a controlling discriminative stimulus, Sd, together with the response, R, it produces, on the basis of a previous literature of operant and instrumental research. Complex motor and neural activity were reviewed in accordance with partitioning of: responses, controlling stimulation, reinforcement, and functions of movement-produced stimulation. Schematics portrayed reinforcement principles through analysis of a fast pathway from Ia muscle spindle afferents to motor outflow. Methods were suggested to minimize operant units through selective reinforcement and establish them to defined end points of learning within composite, ongoing behavior. It was argued that operant neural mechanisms can be investigated efficiently only by starting with individual operants that are thoroughly characterized.

The role of observing and attention in establishing stimulus control

Early theorists (Skinner, Spence) interpreted discrimination learning in terms of the strengthening of the response to one stimulus and its weakening to the other. But this analysis does not account for the increasing independence of the two performances as training continues or for increases in control by dimensions of a stimulus other than the one used in training. Correlation of stimuli with different densities of reinforcement produces an increase in the behavior necessary to observe them, and greater observing of and attending to the relevant stimuli may account for the increase in control by these stimuli. The observing analysis also encompasses errorless training, and the selective nature of observing explains the feature-positive effect and the relatively shallow gradients of generalization generated by negative discriminative stimuli. The effectiveness of the observing analysis in handling these special cases adds to the converging lines of evidence supporting its integrative power and thus its validity.

The effects of methadone on operant behavior maintained with and without conditioned reinforcement in the pigeon

Effects of methadone on key pecking supplemented with brief stimuli either correlated with or independent of unconditioned reinforcement was investigated. On average, key pecks by pigeons produced brief stimuli (BS) once per minute and food once per 4 min during both components of a multiple schedule (i.e., VI1 : BS, VI4 : Food). Brief stimuli were paired with food presentation during one component and not related to food during the second component. Acute methadone administration (0.56, 1.0, 1.7, and 3.0 mg/kg) decreased response rates during both components; however, the decrease was smaller by a constant amount during the paired brief stimulus component, regardless of drug dose. These results suggest conditioned reinforcement is not a primary mechanism through which methadone exerts behavioral effects and that reinforcer-correlated stimuli have potential for diminishing the reduced behavioral output observed following methadone administration.