Spatial midsession reversal learning in rats: Effects of egocentric Cue use and memory

Giving time a chance in the midsession reversal task

The midsession reversal task involves a simultaneous discrimination between stimuli S1 and S2. Choice of S1 but not S2 is reinforced during the first 40 trials, and choice of S2 but not S1 is reinforced during the last 40 trials. Trials are separated by a constant intertrial interval (ITI). Pigeons learn the task seemingly by timing the moment of the reversal trial. Hence, most of their errors occur around trial 40 (S2 choices before trial 41 and S1 choices after trial 40). It has been found that when the ITI is doubled on a test session, the reversal trial is halved, a result consistent with timing. However, inconsistent with timing, halving the ITI on a test session did not double the reversal trial. The asymmetry of ITI effects could be due to the intrusion of novel cues during testing, cues that preempt the timing cue. To test this hypothesis, we ran two types of tests after the regular training in the midsession reversal task, one with S1 and S2 choices always reinforced, and another with S1 always reinforced but S2 reinforced only after 20 trials when the ITI doubled or 40 trials when the ITI halved. For most pigeons, performance was consistent with timing both when the ITI doubled and when it was halved, but some pigeons appeared to follow strategies based on counting or on reinforcement contingencies.

Kea (Nestor notabilis) show flexibility and individuality in within-session reversal learning tasks

The midsession reversal paradigm confronts an animal with a two-choice discrimination task where the reward contingencies are reversed at the midpoint of the session. Species react to the reversal with either win-stay/lose-shift, using local information of reinforcement, or reversal estimation, using global information, e.g. time, to estimate the point of reversal. Besides pigeons, only mammalian species were tested in this paradigm so far and analyses were conducted on pooled data, not considering possible individually different responses. We tested twelve kea parrots with a 40-trial midsession reversal test and additional shifted reversal tests with a variable point of reversal. Birds were tested in two groups on a touchscreen, with the discrimination task having either only visual or additional spatial information. We used Generalized Linear Mixed Models to control for individual differences when analysing the data. Our results demonstrate that kea can use win-stay/lose-shift independently of local information. The predictors group, session, and trial number as well as their interactions had a significant influence on the response. Furthermore, we discovered notable individual differences not only between birds but also between sessions of individual birds, including the ability to quite accurately estimate the reversal position in alternation to win-stay/lose-shift. Our findings of the kea’s quick and flexible responses contribute to the knowledge of diversity in avian cognitive abilities and emphasize the need to consider individuality as well as the limitation of pooling the data when analysing midsession reversal data.

Step changes in the intertrial interval in the midsession reversal task: Predicting pigeons' performance with the learning-to-time model

Our goal was to assess the role of timing in pigeons' performance in the midsession reversal task. In discrete-trial sessions, pigeons learned to discriminate between 2 stimuli, S1 and S2. Choices of S1 were reinforced only in the first half of the session and choices of S2 were reinforced only in the second half. Typically, pigeons choose S2 before the contingency reverses (anticipatory errors) and S1 after (perseverative errors), suggesting that they time the interval from the beginning of the session to the contingency reversal. To test this hypothesis, we exposed pigeons to a midsession reversal task and, depending on the group, either increased or decreased the ITI duration. We then contrasted the pigeons' performance with the predictions of the Learning-to-Time (LeT) model: In both conditions, preference was expected to reverse at the same time as in the previous sessions. When the ITI was doubled, pigeons' preference reversal occurred at half the trial number but at the same time as in the previous sessions. When the ITI was halved, pigeons' preference reversal occurred at a later trial but at an earlier time than in the previous sessions. Hence, pigeons' performance was only partially consistent with the predictions of LeT, suggesting that besides timing, other sources of control, such as the outcome of previous trials, seem to influence choice.

Stimulus control depends on the subjective value of the outcome

Stimuli that provide information about likely future reinforcers tend to shift behavior, provided a reliable relation between the stimulus and the reinforcer can be discriminated. Stimuli that are apparently more reliable exert greater control over behavior. We asked how the subjective value (measured in terms of preference) of reinforcers associated with stimuli influences stimulus control. Five pigeons worked on a concurrent chains procedure in which half of all trials ended in a smaller reinforcer sooner, and the other half in a larger reinforcer later. In Signaled trials, the color and flash duration on the keys in the initial link signaled the outcome of the trial. In Conflicting probe trials, the color and the flash duration signaled conflicting information about the outcome of the trial. Choice in Signaled trials shifted toward the signaled outcome, but was never exclusive. In Conflicting probe trials, control was divided idiosyncratically between the 2 stimulus dimensions, but still favored the outcome with the higher subjective value. Thus, stimulus control depends not only on the perceived reliability of stimuli, but also on the subjective value of the outcome.

Past outcomes and time flexibly exert joint control over midsession reversal performance in the rat

In a midsession reversal task, subjects choose between two stimuli on every trial; only responses to one stimulus are reinforced. Halfway throughout the session, contingencies are reversed: previously reinforced responses are now extinguished and vice versa. Both, the outcome of the previous trial and the time elapsed since the beginning of the session, may predict the availability of reinforcement and determine choice. Thus, this task has typically been used to study cognitive flexibility and the temporal organization of behavior. This study assessed how past outcomes and time interact for behavioral control when each cue predicts the availability of reinforcement to a different extent. Eight rats were trained in four variations of the midsession reversal task differing in the reliability of outcomes and time as predictors of the reinforced response. We manipulated the reliability of the outcomes by providing either continuous or partial reinforcement, and the reliability of time by fixing the moment of reversal (middle of the session) or making the reversal unpredictable (semi-random trial). Results suggest that behavioral control alternates between outcomes and time according to the relative reliability of each cue. Model simulations show that outcomes and time may jointly determine behavior, and that momentary reinforcement rate may determine their relative influence.

The effect of reinforcement probability on time discrimination in the midsession reversal task

We examined how biasing time perception affects choice in a midsession reversal task. Given a simultaneous discrimination between stimuli S1 and S2, with choices of S1 reinforced during the first, but not the second half of the trials, and choices of S2 reinforced during the second, but not the first half of the trials, pigeons show anticipation errors (premature choices of S2) and perseveration errors (belated choices of S1). This suggests that choice depends on timing processes, on predicting when the contingency reverses based on session duration. We exposed 7 pigeons to a midsession reversal task and manipulated the reinforcement rate on each half of the session. Compared to equal reinforcement rates on both halves of the session, when the reinforcement rate on the first half was lower than on the second half, performance showed more anticipation and less perseveration errors, and when the reinforcement rate on the first half was higher than on the second half, performance showed a remarkable reduction of both types of errors. These results suggest that choice depends on both time into the session and the outcome of previous trials. They also challenge current models of timing to integrate local effects.