Compounding of discriminative stimuli from the same and different sensory modalities

Relationship between reinforcement rate and response rate in Pavlovian and operant conditioning with compound stimuli in rats

Harris, Andrew, and Livesey (2012) proposed that response rate (R) to a stimulus (e.g., A) of a compound (e.g., AB) would be directly related to the difference between reinforcement rate (r) during AB and the other stimulus (e.g., B). To explore the extent of this proposal to operant conditioning, three experiments were carried out. Experiment 1 replicated Harris et al. (2012 - Experiment 1) using sucrose as US in Pavlovian conditioning. In Experiment 2 rats pressed a lever under a variable interval schedule in two phases. In Training, responses during a compound stimulus AB100% were reinforced 100%, responses during B50% and C50% 50% and responses during D25%, 25% of the trials. Then, nonreinforced A trials were introduced (Probe). Results from Experiment 2 showed that r controlled the R to AB100%, B50%, C50% and D25% during training and probe, however, were inconclusive about r and R relationship during A. In Experiment 3, responses during A100% were reinforced 100% and during B50% and C50% 50% of the trials (Training). Next, nonreinforced BC trials were presented (Probe). Results from Experiment 3 showed that response rate during BC were similar to A100%, but higher than B50% and C50%, suggesting summation of B50% and C50% reinforcement rates. Overall, the results showed that Harris et al. (2012) proposal extends to discriminative operant relations.

Special issue on computational models of classical conditioning guest editors' introduction

In the present special issue, the performance of current computational models of classical conditioning was evaluated under three requirements: (1) Models were to be tested against a list of previously agreed-upon phenomena; (2) the parameters were fixed across simulations; and (3) the simulations used to test the models had to be made available. These requirements resulted in three major products: (a) a list of fundamental classical-conditioning results for which there is a consensus about their reliability; (b) the necessary information to evaluate each of the models on the basis of its ordinal successes in accounting for the experimental data; and (c) a repository of computational models ready to generate simulations. We believe that the contents of this issue represent the 2012 state of the art in computational modeling of classical conditioning and provide a way to find promising avenues for future model development.

Resistance to extinction and relapse in combined stimulus contexts

Reinforcing an alternative response in the same context as a target response reduces the rate of occurrence but increases the persistence of that target response. Applied researchers who use such techniques to decrease the rate of a target problem behavior risk inadvertently increasing the persistence of the same problem behavior. Behavioral momentum theory asserts that the increased persistence is a function of the alternative reinforcement enhancing the Pavlovian relation between the target stimulus context and reinforcement. A method showing promise for reducing the persistence-enhancing effects of alternative reinforcement is to train the alternative response in a separate stimulus context before combining with the target stimulus in extinction. The present study replicated previous findings using pigeons by showing that combining an "alternative" richer VI schedule (96 reinforcers/hr) with a "target" leaner VI schedule (24 reinforcers/hr) reduced resistance to extinction of target responding compared with concurrent training of the alternative and target responses (totaling 120 reinforcers/hr). We also found less relapse with a reinstatement procedure following extinction with separate-context training, supporting previous findings that training conditions similarly influence both resistance to extinction and relapse. Finally, combining the alternative stimulus context was less disruptive to target responding previously trained in the concurrent schedule, relative to combining with the target response trained alone. Overall, the present findings suggest the technique of combining stimulus contexts associated with alternative responses with those associated with target responses disrupts target responding. Furthermore, the effectiveness of this disruption is a function of training context of reinforcement for target responding, consistent with assertions of behavioral momentum theory.

Additive and suppressive response summation with a chain schedule

Rats responded in a two-segment (variable-interval variable-interval) chain schedule. In one experiment, three subjects had either clicker, light, or clicker plus light as terminal-segment stimuli. All three responded at the highest rate when clicker plus light were present, thus showing additive summation. For three other subjects, initial-segment stimuli were either clicker, light, or clicker plus light. Two subjects responded at the lowest rate when clicker plus light were present, thus showing suppressive summation. In a second experiment, three subjects had either clicker, light, or neither clicker nor light as terminal-segment stimuli. None of these subjects showed reliable additive summation. Three other subjects had clicker, light, or neither as the initial-segment stimulus, and all three showed suppressive summation. Additive and suppressive summation both can be demonstrated with chain schedules, but stimulus parameters may be major variables in producing the effect.

Discriminated response and incentive processes in operant conditioning: a two-factor model of stimulus control

Understanding stimulus control generated in instrumental learning requires the direct investigation of discriminated response and reinforcer (incentive) processes acquired exclusively through the response-reinforcer contingencies operating on complex (multicomponent) baselines. Two series of stimulus-compounding studies accomplished this direct investigation. In one series, the independent variable was the relative reinforcement between schedule components; in the second series, it was relative response rate between components. Stimulus-compounding tests revealed that response and incentive processes enhanced each other when in agreement, counteracted each other when in opposition, and produced intermediate results when only one factor was operating. This pattern of results led to the conclusion that these factors were algebraically combining and to the development of a response/incentive matrix reflecting these dynamics. This two-factor analysis was extended to the peak-shift effect in stimulus generalization experiments and to the generation of inhibitory control. Two decades of stimulus compounding and peak-shift research were organized within this two-factor framework, extending this traditional approach to learning to active research areas heretofore not systematically considered in these terms.

Combined-stimulus control as a function of the response rate controlled by the absence of the single stimuli

Rat's bar-press responses were maintained at moderate rates during separate presentations of light and tone by separate but concurrent variable-interval schedules of food and shock presentation. The relative response rate maintained during light-out-no-tone was alternated in four successive phases: in Phases 1 and 3 responding was maintained at a higher rate than that during light and tone alone by a variable-interval food schedule, while in Phases 2 and 4 responding was reduced to a lower rate by a differential-reinforcement-of-other-behavior food schedule. In test presentations of light, tone and a light-plus-tone combination, administered at the end of each phase, the proportion of responses emitted during light-plus-tone was an inverse function of the relative response rate controlled by light-out-no-tone, indicating that the relative training response rate controlled by the absence of the single stimuli determined the control exerted by the combined stimuli. Different relative response rates maintained in training may also be partly responsible for previously observed differences in the form of generalization gradients following the establishment of multi-stimulus control.

Averaging of temporal memories by rats

Rats were trained on a mixed fixed-interval schedule in which stimulus A (tone or light) indicated food availability after 10 s and stimulus B (the other stimulus) indicated food availability after 20 s. Testing consisted of nonreinforced probe trials in which the stimulus was A, B, or the compound AB. On single-stimulus trials, rats responded with a peak of activity around the programmed reinforced time. On compound-stimulus trials, rats showed a single scalar peak of responding at a time midway between those for stimulus A and B. These results suggest that when provided with discrepant information regarding the temporal predictability of reinforcement, rats compute an average of the scheduled reinforcement times for the A and B stimuli and use this average to generate an expectation of reward for the compound stimuli.

Reinstatement of a food-maintained operant produced by compounding discriminative stimuli

After a tone and a light were established as discriminative stimuli for food-reinforced responding in rats, presenting these stimuli simultaneously produced over three times as many responses as either the tone or light alone. Following this stimulus compounding test, responses during the tone and during the light were not reinforced (extinction) for 20 sessions, essentially eliminating responding. On stimulus compounding tests administered after the 10th and 20th extinction sessions, tone-plus-light continued to produce significantly more responding than the tone or light alone. The compound even produced responses when the individual stimuli no longer did. These results suggest that the simultaneous presentation of multiple extinguished discriminative stimuli may also contribute to the reinstatement of other positively-reinforced behaviors, such as drug taking.

Conditioned inhibition of cocaine seeking in rats

Despite its potential relevance to the treatment of drug abuse, conditioned inhibition of drug seeking has not been systematically investigated before. In this study, rats could self-administer cocaine by lever pressing whenever a click or tone was present. Responding was not reinforced when a light was present. The light was presented simultaneously with the click (i.e., in an excitatory context) in 1 group, but the light was always presented alone in another group. When it was later presented in compound with the tone, the light was a highly effective conditioned inhibitor, suppressing cocaine seeking by 92% in the former group and by 74% in the latter. These results suggest ways to improve cue-oriented behavioral treatments for drug abuse.

The importance of a compound stimulus in conditioned drug-seeking behavior following one week of extinction from self-administered cocaine in rats

Previous studies have demonstrated that conditioned stimuli can increase responding on a drug-associated lever after extinction from drug self-administration. The present study investigated singular stimuli (tone or light) or a compound stimulus (tone + light) for their ability to increase extinguished responding following chronic cocaine self-administration. Rats self-administered cocaine for 2 weeks on a fixed ratio (FR1) schedule of reinforcement, in which lever responding resulted in varied presentation of a tone, light, or tone + light combination. The rats were then exposed to 1 week of daily extinction sessions. Presentation of the tone + light on day 8 of extinction in the absence of cocaine reinforcement resulted in a significant increase in responding, while either stimulus component alone was much weaker or failed to produce any changes from extinction rates of responding. In addition, changing the duration of the single elements of the compound did not affect the magnitude of increased responding to the compound. Following three final extinction sessions, robust lever responding for cocaine infusions on day 12 of extinction was seen across all groups. These findings suggest that compound stimuli may be critical to fully activate drug-seeking behavior in conditions of craving and relapse following prolonged extinction.

Humans' responses to novel stimulus compounds and the effects of training

Twenty-four college students participated in an experiment with stimulus compounds. Subjects learned to discriminate color stimuli that correlated with varying probabilities of reinforcement. Reinforcement consisted of points. For all subjects, two colors signaled a .80 reinforcement probability, and two others signaled a .20 probability. For compound-trained subjects, a fifth compound stimulus (composed of a high-probability color and a low-probability color) was correlated with a .10 reinforcement probability. During testing, interspersed probe trials required subjects to choose between two alternatives: a compound stimulus and either one of its constituent stimuli. Compound-untrained subjects preferred the compound over either individual stimulus, thus showing response summation. However, compound-trained subjects, having had experience with an exemplar compound, showed significantly lower choice proportions for a test compound, indicating that subjects' responding to novel stimulus compounds is modifiable by experience with a single similar compound.

Additive summation following intradimensional discrimination training

Rats were exposed to intradimensional composite stimuli presented on the response lever that varied in both light intensity and flicker rate. For all subjects, pressing the lever was reinforced when it was illuminated at a high intensity and flickered at a low rate (I + f) or when it was illuminated at a low intensity and flickered at a high rate (i + F). For half the subjects, lever responding was not reinforced when it was illuminated at a low intensity and flickering at a low rate (i + f). For the remaining subjects, lever presses were not reinforced when the lever was illuminated at a high intensity and flickered at a high rate (I + F). When the composite stimulus composed of the light intensity and flicker rates that had been associated only with reinforced responding was displayed (I + F for half the subjects and i + f for the remaining subjects), it controlled the highest response rate of all stimuli (additive summation). The results demonstrated that similar attentional processes control intra- and interdimensional composite-stimulus discriminations in a manner consistent with Weiss' (1972) analysis of summation.