Rate and temporal pattern of key pecking under autoshaping and omission schedules of reinforcement

Accounting for negative automaintenance in pigeons: a dual learning systems approach and factored representations

Animals, including Humans, are prone to develop persistent maladaptive and suboptimal behaviours. Some of these behaviours have been suggested to arise from interactions between brain systems of Pavlovian conditioning, the acquisition of responses to initially neutral stimuli previously paired with rewards, and instrumental conditioning, the acquisition of active behaviours leading to rewards. However the mechanics of these systems and their interactions are still unclear. While extensively studied independently, few models have been developed to account for these interactions. On some experiment, pigeons have been observed to display a maladaptive behaviour that some suggest to involve conflicts between Pavlovian and instrumental conditioning. In a procedure referred as negative automaintenance, a key light is paired with the subsequent delivery of food, however any peck towards the key light results in the omission of the reward. Studies showed that in such procedure some pigeons persisted in pecking to a substantial level despite its negative consequence, while others learned to refrain from pecking and maximized their cumulative rewards. Furthermore, the pigeons that were unable to refrain from pecking could nevertheless shift their pecks towards a harmless alternative key light. We confronted a computational model that combines dual-learning systems and factored representations, recently developed to account for sign-tracking and goal-tracking behaviours in rats, to these negative automaintenance experimental data. We show that it can explain the variability of the observed behaviours and the capacity of alternative key lights to distract pigeons from their detrimental behaviours. These results confirm the proposed model as an interesting tool to reproduce experiments that could involve interactions between Pavlovian and instrumental conditioning. The model allows us to draw predictions that may be experimentally verified, which could help further investigate the neural mechanisms underlying theses interactions.

Automaintenance without stimulus-change reinforcement: Temporal control of key pecks

Yoked pairs of experimentally naive pigeons were exposed to a modified autoshaping procedure in which key pecking by the leader birds postponed both keylight termination and access to grain for the leader and the follower bird. Key pecking developed and was maintained in all birds and continued through two reversals of roles in the yoked procedure. Although temporal control developed more slowly in follower birds, asymptotic temporal distributions of key pecking were similar for all birds in both leader and follower roles; maximum responding occurred soon after keylight onset and decreased to a minimum prior to reinforcement. Response distributions for both leader and follower birds were described by Killeen's (1975) mathematical model of temporal control. Follower birds received response-independent reinforcement, and the development by these birds of temporal distributions which are minimal immediately prior to reinforcement is without precedent in Pavlovian appetitive conditioning. However, maintenance of key pecking by the leader birds, whose responses postponed both stimulus-change and food reinforcement, supports an interpretation of autoshaped and automaintained key pecking as responding elicited by signaled grain presentation.

Automaintenance in guinea pigs: effects of feeding regimen and omission training

Behavior maintained by stimulus-reinforcer pairings was examined. Guinea pigs maintained at 85 per cent of free-feeding weights reliably contacted a retractable lever presented before delivery of a single piece of guinea-pig chow or a 45-milligram guinea-pig pellet. When animals were given free access to one food and received the second food preceded by the lever, contact responses persisted. Such responses seldom occurred when a single food was freely available and was also delivered after lever presentation. Introduction of an omission training (negative automaintenance) procedure, in which lever contacts resulted in lever retraction and prevented food delivery, strongly reduced lever contacts. Observation indicated that mouthing the food cup, instead of the lever, became the prominent behavior during the prefood stimulus under the omission training procedure.

Negative automaintenance omission training is effective

Twelve pigeons were exposed to negative automaintenance contingencies for 17-27 sessions immediately after brief (14-16 sessions) or extended (168-237 sessions) exposure to positive automaintenance contingencies, or after 4-10 sessions of instrumental training. In all conditions, negative automaintenance contingencies virtually eliminated responding, reducing response rates to an average 1.3 responses per min. This reduction in response rate was validated by a model of transition between early and late response rates that assumed exponential transition of rates from one set of contingencies to the next. The model faithfully reproduced cumulative records, and yielded estimates of terminal rates under negative automaintenance that were close to operant level.

Locating reward cue at response manipulandum (CAM) induces symptoms of drug abuse

Appetitive instrumental discrimination learning procedures provide for CAM (cue and manipulandum) when the reward cue (discriminative stimulus positively correlated with positive reinforcement) is located at the response manipulandum (object that when contacted or manipulated defines the performance of the instrumental response). Evidence reviewed shows that CAM induces excessive and compulsive instrumental responding relative to otherwise comparable non-CAM control procedures. In humans, symptoms of drug abuse are particularly likely when the drug-taking implement (response manipulandum at which instrumental drug-taking is directed) is also predictive of the drug's rewarding effects (reward cue). Evidence that the predictive relationship between a drug-taking implement and drug reward relates to drug abuse is reviewed, and implications for treatment and prevention are considered. CAM is related to neurobiological models of drug abuse that emphasize the role of the neurotransmitter dopamine (DA). CAM produces convergence of DA-mediated responding for conditioned reinforcement with DA mediation of psychomotor activation and incentive-motivational processes to yield reflexive cue-directed responding not observed in non-CAM controls.

"Turning back the clock" on serial-stimulus sign tracking

Two experiments examined the effects of a negative (setback) response contingency on key pecking engendered by a changing light-intensity stimulus clock (ramp stimulus) signaling fixed-time 30-s food deliveries. The response contingency specified that responses would immediately decrease the light-intensity value, and, because food was delivered only after the highest intensity value was presented, would delay food delivery by 1 s for each response. The first experiment examined the acquisition and maintenance of responding for a group trained with the contingency in effect and for a group trained on a response-independent schedule with the ramp stimulus prior to introduction of the contingency. The first group acquired low rates of key pecking, and, after considerable exposure to the contingency, those rates were reduced to low levels. The rates of responding for the second group were reduced very rapidly (within four to five trials) after introduction of the setback contingency. For both groups, rates of responding increased for all but 1 bird when the contingency was removed. A second experiment compared the separate effects of each part of the response contingency. One group was exposed only to the stimulus setback (stimulus only), and a second group was exposed only to the delay of the reinforcer (delay only). The stimulus-only group's rates of responding were immediately reduced to moderate levels, but for most of the birds, these rates recovered quickly when the contingency was removed. The delay-only groups's rates decreased after several trials, to very low levels, and recovery of responding took several sessions once the contingency was removed. The results suggest that (a) sign-tracking behavior elicited by an added clock stimulus may be reduced rapidly and persistently when a setback contingency is imposed, and (b) the success of the contingency is due both to response-dependent stimulus change and response-dependent alterations in the frequency of food delivery. The operation of the contingency is compared with the effects of secondary reinforcement and punishment procedures.

Discrimination and differentiation of response number in stimulus directed pecking of pigeons

In Experiment 1, autoshaping trials terminated with food only if pigeons emitted more than a target number of responses during a trial in one condition and fewer than a target number in another. The median number of responses per trial shifted in accordance wtih the requirements. The responding of yoked-control birds that received response-independent reinforcers did not vary with the response requirements. In Experiment 2, the number of responses in autoshaping trial became the discriminative stimulus for reinforcement in the second component of a chained schedule. In one condition, responding was reinforced only if the number of responses in the first component was above a target value; in the other condition, responding was reinforced only if the number was below the target value. The distribution of the first-component response numbers did not shift systematically between discrimination conditions, but response rates in the second component indicated that the number of responses in the autoshaping trial was a discriminable property behavior.