The dual role of the context in postpeak performance decrements resulting from extended training

Behavior Stability and Individual Differences in Pavlovian Extended Conditioning

How stable and general is behavior once maximum learning is reached? To answer this question and understand post-acquisition behavior and its related individual differences, we propose a psychological principle that naturally extends associative models of Pavlovian conditioning to a dynamical oscillatory model where subjects have a greater memory capacity than usually postulated, but with greater forecast uncertainty. This results in a greater resistance to learning in the first few sessions followed by an over-optimal response peak and a sequence of progressively damped response oscillations. We detected the first peak and trough of the new learning curve in our data, but their dispersion was too large to also check the presence of oscillations with smaller amplitude. We ran an unusually long experiment with 32 rats over 3,960 trials, where we excluded habituation and other well-known phenomena as sources of variability in the subjects' performance. Using the data of this and another Pavlovian experiment by Harris et al. (2015), as an illustration of the principle we tested the theory against the basic associative single-cue Rescorla–Wagner (RW) model. We found evidence that the RW model is the best non-linear regression to data only for a minority of the subjects, while its dynamical extension can explain the almost totality of data with strong to very strong evidence. Finally, an analysis of short-scale fluctuations of individual responses showed that they are described by random white noise, in contrast with the colored-noise findings in human performance.

Roles of context in acquisition of human instrumental learning: Implications for the understanding of the mechanisms underlying context-switch effects

Four experiments in human instrumental learning explored the associations involving the context that develop after three trials of training on simple discriminations. Experiments 1 and 4 found a deleterious effect of switching the learning context that cannot be explained by the context-outcome binary associations commonly used to explain context-switch effects after short training in human predictive learning and in animal Pavlovian conditioning. Evidence for context-outcome (Experiment 2), context-discriminative stimulus (Experiment 3), and context-instrumental response (Experiment 4) binary associations was found within the same training paradigm, suggesting that contexts became associated with all the elements of the situation, regardless of whether those associations played a role in a specific context-switch effect detected on performance.

ecco: An error correcting comparator theory

Building on the work of Ralph Miller and coworkers (Miller and Matzel, 1988; Denniston et al., 2001; Stout and Miller, 2007), I propose a new formalization of the comparator hypothesis that seeks to overcome some shortcomings of existing formalizations. The new model, dubbed ecco for "Error-Correcting COmparisons," retains the comparator process and the learning of CS-CS associations based on contingency. ecco assumes, however, that learning of CS-US associations is driven by total error correction, as first introduced by Rescorla and Wagner (1972). I explore ecco's behavior in acquisition, compound conditioning, blocking, backward blocking, and unovershadowing. In these paradigms, ecco appears capable of avoiding the problems of current comparator models, such as the inability to solve some discriminations and some paradoxical effects of stimulus salience. At the same time, ecco exhibits the retrospective revaluation phenomena that are characteristic of comparator theory.

Attention as an acquisition and performance variable (AAPV)

This article introduces a new model of Pavlovian conditioning, attention as an acquisition and performance variable (AAPV), which, like several other so-called attentional models, emphasizes the role of variation of cue salience, together with associative strength, in accounting for conditioning phenomena. AAPV is primarily (but not exclusively) a performance-focused model in that it assumes not only that both the saliences and associative strengths of cue representations change during acquisition, but also that they are both influential at the time of test in determining responding. Different weights are given to the representations' associative strengths according to the representations' respective saliences at test. The model also treats the representation of a stimulus that is directly activated by presentation of that stimulus as distinct from the representation of the same stimulus that is activated by presenting a companion of the stimulus. Additionally, extinction is viewed as resulting from a decrease in the salience of the cue's representation, rather than a decrease in associative strength. Simulations of several Pavlovian phenomena are presented in order to illustrate the model and assess its robustness.

The effect of the amount of blocking cue training on blocking of appetitive conditioning in mice

•

Blocking of appetitive conditioning in mice has rarely been demonstrated.

•

Blocking occurred when there was 200, but not 80 trials with a visual blocking cue.

•

Blocking occurred independent of trial number with an auditory blocking cue.

•

Post-asymptotic training is necessary under certain conditions for blocking.

Conditioning of a target cue is blocked when it occurs in compound with another cue (blocking cue) that has already received conditioning. Although blocking of appetitive conditioning is commonly used in rodents as a test of selective learning, it has been demonstrated rarely in mice. In order to investigate the conditions that result in blocking in mice two studies tested the effect of the extent of prior blocking cue training on blocking of appetitive conditioning. Mice received either 80 or 200 trials of blocking cue training prior to compound conditioning. A control group received only compound training. Experiment 1 assessed the ability of a visual cue to block conditioning to an auditory target cue. Exposure to the context and the unconditioned stimulus, sucrose pellets, was equated across groups. Blocking was evident in mice that received 200, but not 80 training trials with the visual blocking cue. Responding to the blocking cue was similar across groups. Experiment 2 assessed the ability of an auditory cue to block conditioning to a visual target cue. Blocking was evident in mice trained with 80 and 200 auditory blocking cue trials. The results demonstrate that the strength of blocking in mice is dependent on the modality and experience of the blocking cue. Furthermore, prolonged training of the blocking cue after asymptotic levels of conditioned responding have been reached is necessary for blocking to occur under certain conditions suggesting that the strength of conditioned responding is a limited measure of learning.

Conditioned suppression is an inverted-U function of footshock intensity

Most theories of associative learning assert that conditioned responding to a target cue is a monotonically increasing function of unconditioned-stimulus (US) intensity. In a lick suppression preparation with rats, a cue was paired with a 0.4-, 0.6-, 0.8-, 1.0-, 1.2-, or 1.4-mA footshock in Experiment 1a, and with a 0.3-, 0.8-, 1.3-, or 1.8-mA footshock in Experiment 1b. Subsequent suppression in response to the cue was an inverted-U function of the US intensity. In Experiment 2, we demonstrated that massive extinction of the training context and compound conditioning can each attenuate the response decrement caused by training with a high-intensity US. The sometimes-competing-retrieval model (Stout & Miller, Psychological Review 114:759-783, 2007) provides a better fit to these data than do several other models of associative learning.

The functions of contexts in associative learning

Although contexts play many roles during training and also during testing, over the last four decades theories of learning have predominantly focused on one or the other of the two families of functions served by contexts. In this selective review, we summarize recent data concerning these two functions and their interrelationship. The first function is similar to that of discrete cues, and allows contexts to elicit conditioned responses and compete with discrete events for behavioral control. The second function is modulatory, and similar to that of discrete occasion setters in that in this role contexts do not elicit conditioned responses by themselves, but rather modulate instrumental responding or responding to Pavlovian cues. We first present evidence for these two functions, and then suggest that the spacing of trials, amount of training, and contiguity are three determinants of the degree to which the context will play each function. We also conclude that these two functions are not mutually exclusive, and that future research would benefit from identifying the conditions under which their functions dominate behavioral control. We close by discussing some misconceptions concerning contexts. This article is part of a Special Issue entitled: SQAB 2013: Contextual Con.

Suboptimal choice by pigeons may result from the diminishing effect of nonreinforcement

Pigeons prefer an alternative that provides them with a stimulus 20% of the time that predicts 10 pellets of food and a different stimulus 80% of the time that predicts 0 pellets, over an alternative that provides them with a stimulus that always predicts 3 pellets of food, even though the preferred alternative provides them with considerably less food. It appears that the stimulus that predicts 10 pellets acts as a strong conditioned reinforcer, despite the fact that the stimulus that predicts 0 pellets occurs 4 times as often. In the present research, we tested the hypothesis that early in training conditioned inhibition develops to the 0-pellet stimulus, but later in training it dissipates. We trained pigeons with a hue as the 10-pellet stimulus and a vertical line as the 0-pellet stimulus. To assess the inhibitory value of the vertical line, we compared responding to the 10-pellet hue with responding to the compound of the 10-pellet hue and the vertical line early in training and once again late in training, using both a within-subject design (Experiment 1) and a between-groups design (Experiment 2). We found that there was a significant reduction in inhibition between the early test (when pigeons chose optimally) and late test (when choice was suboptimal). Thus, the increase in suboptimal choice may result from the decline in inhibition to the 0-pellet stimulus. Implications for human gambling behavior are considered.

The error in total error reduction

Most models of human and animal learning assume that learning is proportional to the discrepancy between a delivered outcome and the outcome predicted by all cues present during that trial (i.e., total error across a stimulus compound). This total error reduction (TER) view has been implemented in connectionist and artificial neural network models to describe the conditions under which weights between units change. Electrophysiological work has revealed that the activity of dopamine neurons is correlated with the total error signal in models of reward learning. Similar neural mechanisms presumably support fear conditioning, human contingency learning, and other types of learning. Using a computational modeling approach, we compared several TER models of associative learning to an alternative model that rejects the TER assumption in favor of local error reduction (LER), which assumes that learning about each cue is proportional to the discrepancy between the delivered outcome and the outcome predicted by that specific cue on that trial. The LER model provided a better fit to the reviewed data than the TER models. Given the superiority of the LER model with the present data sets, acceptance of TER should be tempered.

Preventing return of fear in an animal model of anxiety: additive effects of massive extinction and extinction in multiple contexts

Fear conditioning and experimental extinction have been presented as models of anxiety disorders and exposure therapy, respectively. Moreover, the return of fear serves as a model of relapse after exposure therapy. Here we present two experiments, with rats as subjects in a lick suppression preparation, in which we assessed the additive effects of two different treatments to attenuate the return of fear. First, we evaluated whether two phenomena known to generate return of fear (i.e., spontaneous recovery and renewal) summate to produce a stronger reappearance of extinguished fear. At test, rats evaluated outside the extinction context following a long delay after extinction (i.e., a delayed context shift) exhibited greater return of extinguished fear than rats evaluated outside the extinction context alone, but return of extinguished fear following a delayed context shift did not significantly differ from the return of fear elicited in rats tested following a long delay after extinction alone. Additionally, extinction in multiple contexts and a massive extinction treatment each attenuated the strong return of fear produced by a delayed context shift. Moreover, the conjoint action of these treatments was significantly more successful in preventing the reappearance of extinguished fear, suggesting that extensive cue exposure administered in several different therapeutic settings has the potential to reduce relapse after therapy for anxiety disorders, more than either manipulation alone.