Increased reinforcement when timeout from avoidance includes access to a safe place

Assessing the role of effort reduction in the reinforcing efficacy of timeout from avoidance

Rats responded on concurrent schedules of shock-postponement or deletion (avoidance) and timeout from avoidance. In Experiment 1, 3 rats' responses on one lever postponed shocks for 20 s and responses on a second lever produced a 1-min timeout according to a variable-interval 45-s schedule. Across conditions, a warning signal (white noise) was presented 19.5 s, 16 s, 12 s, 8 s, or 4 s before an impending shock. Raising the duration of the warning signal increased both avoidance and timeout response rates. Timeout responding, although positively correlated with avoidance responding, was not correlated with the prevailing shock rate. In Experiment 2, 3 rats' responses on one lever deleted scheduled shocks according to a variable-cycle 30-s schedule and responses on a second lever produced a 2-min timeout as described above. After this baseline condition, the avoidance lever was removed and noncontingent shocks were delivered at intervals yoked to the receipt of shocks in the baseline sessions. Timeout responding decreased when the avoidance lever was removed, even though the shock-frequency reduction afforded by the timeout remained constant. These results suggest that a key factor in the reinforcing efficacy of timeout is suspension of the requirement to work to avoid shock, rather than the reduction in shock frequency associated with timeout.

The role of dopamine in the context of aversive stimuli with particular reference to acoustically signaled avoidance learning

Learning from punishment is a powerful means for behavioral adaptation with high relevance for various mechanisms of self-protection. Several studies have explored the contribution of released dopamine (DA) or responses of DA neurons on reward seeking using rewards such as food, water, and sex. Phasic DA signals evoked by rewards or conditioned reward predictors are well documented, as are modulations of these signals by such parameters as reward magnitude, probability, and deviation of actually occurring from expected rewards. Less attention has been paid to DA neuron firing and DA release in response to aversive stimuli, and the prediction and avoidance of punishment. In this review, we first focus on DA changes in response to aversive stimuli as measured by microdialysis and voltammetry followed by the change in electrophysiological signatures by aversive stimuli and fearful events. We subsequently focus on the role of DA and effect of DA manipulations on signaled avoidance learning, which consists of learning the significance of a warning cue through Pavlovian associations and the execution of an instrumental avoidance response. We present a coherent framework utilizing the data on microdialysis, voltammetry, electrophysiological recording, electrical brain stimulation, and behavioral analysis. We end by outlining current gaps in the literature and proposing future directions aimed at incorporating technical and conceptual progress to understand the involvement of reward circuit on punishment based decisions.

Theoretical implications of the neurotic paradox as a problem in behavior theory: An experimental resolution

Why do human phobias last for months or years when such behavior should undergo extinction? This failure of extinction or persistence of self-defeating behavior of human disorders was labeled by Mowrer as the neurotic paradox. The paradox is cited by an ever-increasing number of critics who challenge any laboratory-based learning model of human psychopathology. Laboratory research, of course, omits essential requirements in the analysis of behavior, and the principles derived from such analyses must be combined in order to explain complex human behavaior. Validation for a behavioral model can thus be achieved if (a) basic principles inferred from observation of humans treated with a laboratory-derived extinction procedure (e.g., implosive therapy) are combined with (b) principles examined in laboratory research that are combined to generate unique predictions that correspond to known features of human phobic behavior. The latter evidence is briefly reviewed in research demonstrating sustained responding over one thousand consecutive active avoidance responses with complete avoidance of the "phobic" CS for an initial single shock trial. Differential reinforcement for responses to early sequential stimuli depends on minimal work requirement, and reinforcement by timeout from avoidance. This combination of factors effectively precludes extinction to main conditioned aversive stimuli for nonhumans, as it does for human phobias. Support for a laboratory model of human phobia is thereby attained.

Vertical jumping and signaled avoidance

This paper reports an experiment intended to demonstrate that the vertical jumping response can be learned using a signaled-avoidance technique. A photoelectric cell system was used to record the response. Twenty female rats, divided equally into two groups, were exposed to intertrial intervals of either 15 or 40 s. Subjects had to achieve three successive criteria of acquisition: 3, 5, and 10 consecutive avoidance responses. Results showed that both groups learned the avoidance response, requiring increasingly larger numbers of trials as the acquisition criteria increased. No significant effect of intertrial interval was observed.

Variable-interval schedules of timeout from avoidance

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. Throughout, the shock-postponement schedule maintained proficient levels of avoidance. Nevertheless, in Experiment 1 responding on the timeout lever was established rapidly, was maintained at stable levels on variable-interval schedules, was extinguished by withholding timeout, was reestablished when timeout was reintroduced, and was brought under discriminative control with a multiple variable-interval extinction schedule of timeout. These results are in contrast with Verhave's (1962) conclusion that timeout is an ineffective reinforcer when presented to rats on intermittent schedules. In Experiment 2 the consequence of responding on the timeout lever was altered so that the shock-postponement schedule remained in effect even though the stimulus conditions associated with timeout were produced for 2 min. Responding extinguished, indicating that suspension of the shock-postponement schedule, not stimulus change, was the source of reinforcement. By establishing the reinforcing efficacy of timeout with standard variable-interval schedules, these experiments illustrate a procedure for studying negative reinforcement in the same way as positive reinforcement.

The defense motivation system: A theory of avoidance behavior

A motivational system approach to avoidance behavior is presented. According to this approach, a motivational state increases the probability of relevant response patterns and establishes the appropriate or “ideal” consummatory stimuli as positive reinforcers. In the case of feeding motivation, for example, hungry rats are likely to explore and gnaw, and to learn to persist in activities correlated with the reception of consummatory stimuli produced by ingestion of palatable substances. In the case of defense motivation, fearful rats are likely to flee or freeze, and to learn to persist in activities correlated with consummatory stimuli produced by flight from a dangerous place. Defense system activation is distinct from alarm reactions. The latter prepare the animal for probable noxious events, involve relatively intense negative affect and extinguish rapidly in situations where the noxious event no longer occurs. In contrast, defense system activation potentiates innate and modified defense reactions, thus preparing the animal for possible, but not necessarily probable, noxious events; it involves little or no negative affect and extinguishes very slowly when the noxious event no longer occurs. With these assumptions and the resulting model we attempt to resolve several long-standing problems in avoidance learning, including the low correlation between negative affect and avoidance performance, differential rates of extinction for avoidance performance and conditioned emotional responses, and evidence that some avoidance responses are much more easily learned than others. In addition, the model has implications for the study of parallels between appetitive and aversive motivation, sign tracking in aversive conditioning, and orientation of flight responses. Historical antecedents and alternative approaches are discussed.

The combination of appetitive and aversive reinforcers and the nature of their interaction during auditory learning

Learned changes in behavior can be elicited by either appetitive or aversive reinforcers. It is, however, not clear whether the two types of motivation, (approaching appetitive stimuli and avoiding aversive stimuli) drive learning in the same or different ways, nor is their interaction understood in situations where the two types are combined in a single experiment. To investigate this question we have developed a novel learning paradigm for Mongolian gerbils, which not only allows rewards and punishments to be presented in isolation or in combination with each other, but also can use these opposite reinforcers to drive the same learned behavior. Specifically, we studied learning of tone-conditioned hurdle crossing in a shuttle box driven by either an appetitive reinforcer (brain stimulation reward) or an aversive reinforcer (electrical footshock), or by a combination of both. Combination of the two reinforcers potentiated speed of acquisition, led to maximum possible performance, and delayed extinction as compared to either reinforcer alone. Additional experiments, using partial reinforcement protocols and experiments in which one of the reinforcers was omitted after the animals had been previously trained with the combination of both reinforcers, indicated that appetitive and aversive reinforcers operated together but acted in different ways: in this particular experimental context, punishment appeared to be more effective for initial acquisition and reward more effective to maintain a high level of conditioned responses (CRs). The results imply that learning mechanisms in problem solving were maximally effective when the initial punishment of mistakes was combined with the subsequent rewarding of correct performance.

Explaining avoidance: two factors are still better than one

Two-factor theory remains a viable account of avoidance behavior. By emphasizing the interplay of respondent and operant contingencies, two-factor theory encourages the analysis of stimuli that mediate molar consequences and incorporates control by local events as well as events that are temporally remote, improbable, or cumulative.

Stimuli inevitably generated by behavior that avoids electric shock are inherently reinforcing

A molecular analysis based on the termination of stimuli that are positively correlated with shock and the production of stimuli that are negatively correlated with shock provides a parsimonious count for both traditional discrete-trial avoidance behavior and the data derived from more recent free-operant procedures. The necessary stimuli are provided by the intrinsic feedback generated by the subject's behavior, in addition to those presented by the experimenter. Moreover, all data compatible with the molar principle of shock-frequency reduction as reinforcement are also compatible with a delay-of-shock gradient, but some data compatible with the delay gradient are not compatible with frequency reduction. The delay gradient corresponds to functions relating magnitude of behavioral effect to the time between conditional and unconditional stimuli, the time between conditioned and primary reinforcers, and the time between responses and positive reinforcers.

Conjoint schedules of timeout deletion in pigeons

This experiment attempted to bring behavior under joint control of two distinct contingencies, one that provided food and a second that extended the periods during which that food was available. Pigeons' responses on each of two keys were reinforced according to a single random-interval schedule of food presentation except during signaled timeout periods during which the schedule was temporarily disabled. By means of a conjoint schedule, responses on the initially less preferred key not only produced food but also canceled impending timeouts. When behavior came to predominate on this conjoint alternative, the consequences of responding on the two keys were reversed. Responding in 3 of 4 pigeons proved sensitive to the conjoint scheduled consequences, as evidenced by systematic shifts in response rates favoring the conjoint key. In 2 of these 3 pigeons, sensitivity to the conjoint contingency was evident under time-in:timeout ratios of 2:1 (time-in = 120 s, timeout = 60 s) and 1:5 (time-in = 30 s, timeout = 150 s), whereas for the other pigeon preference for the conjoint key was observed only under the latter sequence of conditions. There was only weak evidence of control by the conjoint scheduled consequences in the 4th subject, despite extended training and forced exposure to the conjoint alternative. The overall pattern of results is consistent with studies of timeout avoidance but also shares features in common with positively reinforced behavior.

Variable-interval schedules of timeout from avoidance: effects of chlordiazepoxide, CGS 8216, morphine, and naltrexone

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally (variable-interval schedule) produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. These procedures provided a baseline for studying the effects of drugs on behavior maintained by different sources of negative reinforcement (shock avoidance and timeout from avoidance). Experiment 1 studied a benzodiazepine agonist, chlordiazepoxide, and antagonist, CGS 8216. Chlordiazepoxide (2.5-30 mg/kg) had little effect on avoidance responding except at higher doses, when it reduced responding. By comparison, responding on the timeout lever was increased in 5 of 6 rats. These effects were reversed by CGS 8216 (2.5-5 mg/kg) in the 2 rats tested, but CGS 8216 had no effect by itself. Experiment 2 studied an opiate agonist, morphine, and antagonist, naltrexone, with 3 rats. Morphine's (2.5-20 mg/kg) effects were opposite those of chlordiazepoxide: At doses that either increased or had no effect on avoidance responding, morphine depressed timeout responding. Naltrexone (5 mg/kg) reversed these actions but had no effect by itself.