Differential outcome effects in pavlovian biconditional and ambiguous occasion setting tasks

Sex differences in discrimination behavior and orbitofrontal engagement during context-gated reward prediction

Animals, including humans, rely on contextual information to interpret ambiguous stimuli. Impaired context processing is a hallmark of several neuropsychiatric disorders, including schizophrenia, autism spectrum disorders, post-traumatic stress disorder, and addiction. While sex differences in the prevalence and manifestations of these disorders are well established, potential sex differences in context processing remain uncertain. Here, we examined sex differences in the contextual control over cue-evoked reward seeking and its neural correlates, in rats. Male and female rats were trained in a bidirectional occasion-setting preparation in which the validity of two auditory reward-predictive cues was informed by the presence, or absence, of a visual contextual feature (LIGHT: X+/DARK: X-/LIGHT: Y-/DARK: Y+). Females were significantly slower to acquire contextual control over cue-evoked reward seeking. However, once established, the contextual control over behavior was more robust in female rats; it showed less within-session variability (less influence of prior reward) and greater resistance to acute stress. This superior contextual control achieved by females was accompanied by an increased activation of the orbitofrontal cortex (OFC) compared to males. Critically, these behavioral and neural sex differences were specific to the contextual modulation process and not observed in simple, context-independent, reward prediction tasks. These results indicate a sex-biased trade-off between the speed of acquisition and the robustness of performance in the contextual modulation of cued reward seeking. The different distribution of sexes along the fast learning ↔ steady performance continuum might reflect different levels of engagement of the OFC, and might have implications for our understanding of sex differences in psychiatric disorders.

Occasion setting

Occasion setting refers to the ability of 1 stimulus, an occasion setter, to modulate the efficacy of the association between another, conditioned stimulus (CS) and an unconditioned stimulus (US) or reinforcer. Occasion setters and simple CSs are readily distinguished. For example, occasion setters are relatively immune to extinction and counterconditioning, and their combination and transfer functions differ substantially from those of simple CSs. Similarly, the acquisition of occasion setting is favored when stimuli are separated by longer intervals, by empty trace intervals, and are of different modalities, whereas the opposite conditions typically favor the acquisition of simple associations. Furthermore, the simple conditioning and occasion setting properties of a single stimulus can be independent, for example, that stimulus may simultaneously predict the occurrence of a reinforcer and indicate that another stimulus will not be reinforced. Many behavioral phenomena that are intractable to simple associative analysis are better understood within an occasion setting framework. Besides capturing the distinction between direct and modulatory control common to many arenas in neuroscience, occasion setting provides a model for the hierarchical organization of memory for events and event relations, and for contextual control more broadly. Although early lesion studies further differentiated between occasion setting and simple conditioning functions, little is known about the neurobiology of occasion setting. Modern techniques for precise manipulation and monitoring of neuronal activity in multiple brain regions are ideally suited for disentangling contributions of simple conditioning and occasion setting in associative learning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Superior ambiguous occasion setting with visual than temporal feature stimuli

Three experiments with rats compared the relative ease with which different sets of visual or temporal cues could participate in Pavlovian learning. In Experiment 1, 1 group was trained to discriminate between visual cues (Light vs. Dark), whereas the other group learned to discriminate between temporal cues (early [10 s] vs. late [90 s]). Both groups learned to distinguish food-paired from nonpaired periods equally well. In Experiment 2, 2 groups were trained on an ambiguous occasion setting task. For Group Visual, a 2-min Light period signaled that 1 10-s auditory conditioned stimulus, CS1, was reinforced with 1 unconditioned stimulus, US1, but that CS2 was not reinforced; whereas a 2-min dark period signaled that CS1 was not reinforced, but CS2 was reinforced with US2 (i.e., Light: CS1-US1, CS2-; Dark: CS1-, CS2-US2). For Group Temporal, early (10-s) or late (90-s) temporal cues within each of these Light and Dark periods were diagnostic of these contingencies (i.e., Early: CS1-US1, CS2-; Late: CS1-, CS2-US2). Group Visual learned the task, but Group Temporal did not. In Experiment 3 we demonstrated that animals could not solve a related temporal ambiguous occasion setting task in which 1 visual stimulus signaled that both CSs were reinforced early whereas the other visual stimulus signaled that the CSs were reinforced only late. Contrary to a currently popular information theory approach to timing in Pavlovian learning, these results suggest that overt nontemporal visual stimuli are better incorporated into conditional discrimination learning than are temporal stimuli. (PsycINFO Database Record

Assessing the blocking of occasion setting

An occasion setter (OS) is a stimulus or context with the capacity to disambiguate an ambiguous conditioned stimulus (CS). Previous research has shown that OSs share some features with regular Pavlovian CSs. Amongst them, research has shown that OSs are subject to blocking; that is, a new OS exerts reduced behavioral control after training in compound with a previously established OS. Of additional interest, in Pavlovian blocking, it has been reported that a blocked CS comes to elicit conditioned responding after the extinction of the blocking CS. This is an example of retrospective revaluation, a family of phenomena in which the response to a specific stimulus is modified by training a related cue. Here, three experiments sought to extend the analogies between OS and Pavlovian conditioning by examining the blocking of OSs and its retrospective revaluation. In all experiments, an OS was established by pairing a CS with food in the presence of the OS, but not in its absence (i.e., positive OS). Blocking was then trained by presenting the OS in compound with a novel OS. Experiment 1 showed blocking of the second OS, but direct exposure to the blocking OS did not enhance responding to the second OS. Experiment 2 replicated the blocking effect but subsequent training of the blocking OS with a reversed contingency showed no retrospective revaluation. Experiment 3 examined whether blocking of the OS occurred with a novel CS during the compound phase. In this experiment blocking was again observed, but only when subjects were tested with the original CS. These results are discussed focusing on the underlying links at work in occasion setting.

Elemental, configural, and occasion setting mechanisms in biconditional and patterning discriminations

Three experiments explored the utility of considering mechanisms of occasion setting for understanding patterning and biconditional discriminations - two more complex conditional discriminations in which the stimulus-outcome relations of occasion setting are embedded. In Experiment 1, rats were trained in an appetitive conditioning task with either a biconditional or a patterning discrimination using relatively brief CSs (10s) and differential outcomes as USs. In this study, rats learned the positive patterning task before they had learned negative patterning, and the biconditional task was the most difficult. However, a detailed examination of the results suggested that rats trained in the biconditional task responded to the stimulus compounds mainly on the basis of individual stimulus-outcome associations. Different conditioned response (CR) topographies as a function of reinforcer type complicated interpretation of these results. Experiment 2 confirmed that the biconditional task, with the parameters used here, was not learned, regardless of whether training involved differential or non-differential outcomes. In Experiment 3 the CS duration was increased to 30s and two different USs were used that each supported similar CR topographies. Under these conditions, we observed that whereas the positive patterning task was learned most rapidly, the biconditional discrimination was learned faster than the negative patterning task. Considered in relation to other findings on patterning and biconditional discriminations, the results suggest that elemental, configural, and/or modulatory occasion setting mechanisms may play different roles in these complex conditional discrimination tasks especially as a function of stimulus duration and differential outcome training.

Temporal Averaging Across Stimuli Signaling the Same or Different Reinforcing Outcomes in the Peak Procedure

The present study examined factors that affect temporal averaging in rats when discriminative stimuli are compounded following separate training indicating the availability of reward after different fixed intervals (FI) on a peak procedure. One group of rats, Group Differential, learned that a flashing light stimulus signaled that one type of food pellet reward could be earned for lever pressing after an FI 5 s interval and that a second type of food pellet reward could be earned after an FI 20 s interval in the presence of a tone stimulus. A second group of rats, Group Non-Differential, was similarly trained except that both types of rewards were scheduled across flash and tone trials. When given non-reinforced flash + tone compound test trials the interval containing the maximal response rate was no different than on flash alone test trials, although some responding also appeared near the long FI time. After these FI contingencies were reversed (flash signaled FI 20 s and tone signaled FI 5 s), however, further compound test trials more clearly revealed a temporal averaging pattern in both groups. The peak interval was shifted to the right of the FI 5 stimulus. Moreover, Group Differential rats acquired the reversed discrimination somewhat more rapidly than Group Non-Differential rats, and in a final selective satiation test Group Differential rats responded less in later intervals after they had been sated on the FI 20 s reward. These data suggest that temporal averaging in stimulus compound tests occurs even when the stimuli being combined signal qualitatively different rewards, but that decreasing the value of one of those rewards can shift responding away from the relevant time interval in a selective satiation test. However, when an especially salient stimulus (e.g., flashing light) signals a short FI, rats tend to process the compound stimulus more in terms of its individual elements.

Pavlovian contextual and instrumental biconditional discrimination learning in mice

Genetically-modified animal models are a powerful tool for investigating the link between neurological and behavioral changes and for the development of therapeutic interventions. Executive function deficits are symptomatic of many human clinical disorders but few tasks exist for studying executive functions in mice. To address this need, we describe procedures for establishing Pavlovian contextual and instrumental biconditional discriminations (BCDs) in C57BL/6J mice. In the first experiment, contextual cues disambiguated when two short duration stimulus targets would be followed by food pellets. In the second experiment, discrete visual cues signaled when lever press or nose poke responses would be continuously reinforced with food pellets. Mice learned both BCDs as evidenced by differential responding in each cue during training and, more critically, during extinction testing. The implications of these findings for using BCD tasks to analyze the neural substrates of executive processing in animal models are discussed.

Visual discrimination learning with an iPad-equipped apparatus

Visual discrimination tasks are commonly used to assess visual learning and memory in non-human animals. The current experiments explored the suitability of an iPad (Apple, Cupertino, California), as a low-cost alternative touchscreen for visual discrimination tasks. In Experiment 1, rats were trained with patterned black-and-white stimuli in a successive non-match to sample procedure. Rats successfully interacted with the iPad but failed to learn to withhold responding on trials in which the sample matched the comparison. Experiment 2 used the same patterned stimuli, but the procedure was simplified to a successive discrimination procedure and we explored the use of procedures known to facilitate discrimination learning. Rats that received training with differential outcomes and a differential reinforcement of other behavior schedule successfully acquired the task. In Experiment 3, the same rats were tested in a simultaneous discrimination task and we explored the use of a correction and non-correction method during acquisition. Rats that failed to learn the discrimination in the previous experiment, improved while trained with the correction method. These experiments support the use of the iPad in visual discrimination tasks and inform future studies investigating learning and memory within a touchscreen-equipped (iPad or other) apparatus.

On the nature of CS and US representations in Pavlovian learning

A significant problem in the study of Pavlovian conditioning is characterizing the nature of the representations of events that enter into learning. This issue has been explored extensively with regard to the question of what features of the unconditioned stimulus enter into learning, but considerably less work has been directed to the question of characterizing the nature of the conditioned stimulus. This article introduces a multilayered connectionist network approach to understanding how "perceptual" or "conceptual" representations of the conditioned stimulus might emerge from conditioning and participate in various learning phenomena. The model is applied to acquired equivalence/distinctiveness of cue effects, as well as a variety of conditional discrimination learning tasks (patterning, biconditional, ambiguous occasion setting, feature discriminations). In addition, studies that have examined what aspects of the unconditioned stimulus enter into learning are also reviewed. Ultimately, it is concluded that adopting a multilayered connectionist network perspective of Pavlovian learning provides us with a richer way in which to view basic learning processes, but a number of key theoretical problems remain to be solved, particularly as they relate to the integration of what we know about the nature of the representations of conditioned and unconditioned stimuli.

Pavlovian biconditional discrimination learning in the C57BL/6J mouse

Four experiments using mice examined acquisition of Pavlovian biconditional discriminations in which two stimulus compounds were paired with food (AX+ and BY+) and two were not (AY- and BX-). Temporally asynchronous compounds were generated by using contextual stimuli (Experiment 1) and 15-s discrete visual cues (Experiments 2A, 2B and 3) to disambiguate when embedded noise or tone stimuli would be paired with food. When food pellets followed both reinforced compounds, successful acquisition was obtained in Experiment 1 but not in Experiments 2A and 2B even though the order of trials was modeled after that used in Experiment 1. However, when differential outcomes followed the reinforced compounds in Experiment 3, acquisition was obtained with discrete cue stimulus compounds. The implications of these results for modulatory models of conditional discrimination learning in animals are discussed.

Issues in the extinction of specific stimulus-outcome associations in Pavlovian conditioning

This paper reviews a variety of studies designed to examine the effects of extinction upon control by specific stimulus-outcome (S-O) associations in Pavlovian conditioning. Studies conducted with rats in a magazine approach conditioning paradigm have shown that control by specific S-O associations is normally unaffected by extinction treatments, although other aspects of conditioned responding seem affected in a more enduring way. However, recent work suggests that extinction can undermine control by such associations if it is administered after the conditioned stimulus is weakly encoded. The results from these studies suggest that it may be important to consider multiple response systems in assessing the impact of extinction. Studies conducted with the flavor preference learning paradigm in rats also show that specific S-O associations can be undermined by procedures that involve presenting a flavor cue in the absence of its associated nutrient. These findings provide no support for the view that flavor preference learning necessarily entails some unique learning process that differs from more conventional processes. As in other situations, some of these effects likely involve a masking process, but the extent to which masking or true associative weakening occurs in extinction more generally is a topic that is not well understood. Finally, we present some data to suggest that extinction also involves conditional "occasion-setting" control by contextual cues. Special procedures are recommended in assessing such learning when the goal is to distinguish this form of learning from other more conventional mechanisms of extinction.