Warmup in avoidance as a function of time since prior training

Inhibited Personality Temperaments Translated Through Enhanced Avoidance and Associative Learning Increase Vulnerability for PTSD

Although many individuals who experience a trauma go on to develop post-traumatic stress disorder (PTSD), the rate of PTSD following trauma is only about 15-24%. There must be some pre-existing conditions that impart increased vulnerability to some individuals and not others. Diathesis models of PTSD theorize that pre-existing vulnerabilities interact with traumatic experiences to produce psychopathology. Recent work has indicated that personality factors such as behavioral inhibition (BI), harm avoidance (HA), and distressed (Type D) personality are vulnerability factors for the development of PTSD and anxiety disorders. These personality temperaments produce enhanced acquisition or maintenance of associations, especially avoidance, which is a criterion symptom of PTSD. In this review, we highlight the evidence for a relationship between these personality types and enhanced avoidance and associative learning, which may increase risk for the development of PTSD. First, we provide the evidence confirming a relationship among BI, HA, distressed (Type D) personality, and PTSD. Second, we present recent findings that BI is associated with enhanced avoidance learning in both humans and animal models. Third, we will review evidence that BI is also associated with enhanced eyeblink conditioning in both humans and animal models. Overall, data from both humans and animals suggest that these personality traits promote enhanced avoidance and associative learning, as well as slowing of extinction in some training protocols, which all support the learning diathesis model. These findings of enhanced learning in vulnerable individuals can be used to develop objective behavioral measures to pre-identify individuals who are more at risk for development of PTSD following traumatic events, allowing for early (possibly preventative) intervention, as well as suggesting possible therapies for PTSD targeted on remediating avoidance or associative learning. Future work should explore the neural substrates of enhanced avoidance and associative learning for behaviorally inhibited individuals in both the animal model and human participants.

Paired-housing selectively facilitates within-session extinction of avoidance behavior, and increases c-Fos expression in the medial prefrontal cortex, in anxiety vulnerable Wistar-Kyoto rats

OBJECTIVE

The perseveration of avoidance behavior, even in the absence of once threatening stimuli, is a key feature of anxiety and related psychiatric conditions. This phenomenon can be observed in the Wistar-Kyoto (WKY) rat which, in comparison to outbred controls, demonstrates impaired extinction of avoidance behavior. Also characteristic of the WKY rat is abnormalities of the neurocircuitry and neuroplasticity of the medial prefrontal cortex (mPFC). One means of reducing physiological responses to anxiety, and conditioned fear, in social species is the presence of a conspecific animal. The current study investigates whether or not pair-housed WKY rats would show facilitated extinction of avoidance in comparison to individual-housed WKY rats, and whether or not pair-housing influences mPFC activation during lever-press avoidance.

METHODS

Male WKY rats were assigned to individual-housed and pair-housed conditions. Rats were trained in lever-press avoidance. Each session of lever-press avoidance consisted of 20 trials, where pressing a lever in response to a warning tone prevented foot-shocks. Rats received 12 acquisition sessions over 4weeks; followed by 6 extinction sessions over 2weeks, where foot-shocks ceased to be delivered. Brains were harvested 90min after trials 1 and 10 of extinction sessions 1 and 6, and mPFC sections underwent c-Fos staining as a measure of activation.

RESULTS

Pair-housed rats showed facilitated lever-press avoidance extinction rates, but the main cause for this overall difference was a selective facilitation of within-session extinction. Similar to individual-housed rats, pair-housed rats continued to avoid during trial 1 of extinction even when the avoidance responding had been significantly reduced by the end of the previous session. Pair-housed rats sacrificed on trial 1 showed greater c-Fos expression in the anterior cingulate cortex and prelimbic cortex subregions of the mPFC compared individual-housed rats sacrificed on trial 1.

CONCLUSION

This data shows pair-housing to facilitate the extinction of avoidance, and to influence activity of the mPFC, in WKY rats. Despite this environmental manipulation, the pair-housed WKY rats continued to show avoidance responding on trial 1 of extinction sessions. This demonstrates that within-session extinction can be dissociated from between-session extinction-resistance in WKY rats. Furthermore, it suggests the individual-housing of WKY rats selectively slows within-session extinction, possibly by reducing neuronal activity of the mPFC during the testing situation.

Absence of "Warm-Up" during Active Avoidance Learning in a Rat Model of Anxiety Vulnerability: Insights from Computational Modeling

Avoidance behaviors, in which a learned response causes omission of an upcoming punisher, are a core feature of many psychiatric disorders. While reinforcement learning (RL) models have been widely used to study the development of appetitive behaviors, less attention has been paid to avoidance. Here, we present a RL model of lever-press avoidance learning in Sprague-Dawley (SD) rats and in the inbred Wistar Kyoto (WKY) rat, which has been proposed as a model of anxiety vulnerability. We focus on “warm-up,” transiently decreased avoidance responding at the start of a testing session, which is shown by SD but not WKY rats. We first show that a RL model can correctly simulate key aspects of acquisition, extinction, and warm-up in SD rats; we then show that WKY behavior can be simulated by altering three model parameters, which respectively govern the tendency to explore new behaviors vs. exploit previously reinforced ones, the tendency to repeat previous behaviors regardless of reinforcement, and the learning rate for predicting future outcomes. This suggests that several, dissociable mechanisms may contribute independently to strain differences in behavior. The model predicts that, if the “standard” inter-session interval is shortened from 48 to 24 h, SD rats (but not WKY) will continue to show warm-up; we confirm this prediction in an empirical study with SD and WKY rats. The model further predicts that SD rats will continue to show warm-up with inter-session intervals as short as a few minutes, while WKY rats will not show warm-up, even with inter-session intervals as long as a month. Together, the modeling and empirical data indicate that strain differences in warm-up are qualitative rather than just the result of differential sensitivity to task variables. Understanding the mechanisms that govern expression of warm-up behavior in avoidance may lead to better understanding of pathological avoidance, and potential pathways to modify these processes.

Activation of extracellular signal-regulated kinase (ERK) and ΔFosB in emotion-associated neural circuitry after asymptotic levels of active avoidance behavior are attained

Avoidance susceptibility may constitute a vulnerability to develop anxiety disorders, and Wistar-Kyoto (WKY) rats exhibit unique features in their acquisition of avoidance behavior that appear to promote susceptibility to this form of learning, namely the absence of the commonly observed "warm-up" effect. The present study sought to determine if strain differences in acquired avoidance behavior, between WKY and Sprague Dawley rats, could be attributed to differences in dopamine-related plasticity, represented by extracellular signal-regulated kinase (ERK) activity, and prolonged neuronal activation, represented by ΔFosB accumulation, in three key areas of the brain: the medial prefrontal cortex (mPFC), dorsal striatum (DS), and basolateral amygdala (BLA). Consistent with earlier work, WKY rats exhibited a higher level of asymptotic performance of avoidance behavior, which included an absence of warm-up in the first few trials of later training sessions, and they exhibited more non-reinforced anticipatory responses in the single minute prior to the initiation of the first warning signal presentation of each training session. In the brain, phosyphorylated ERK2 (pERK2) activation was higher in avoidance trained rats in both the mPFC and DS, although the difference in DS was mostly observed in WKY rats. Avoidance-training was associated with higher levels of ΔFosB expression in the mPFC of SD rats, but not WKY rats. The strain differences in pERK2 activation in the DS and ΔFosB levels in the mPFC may underlie the strain-specific differences observed in warm-up, the emission of non-reinforced anticipatory responses, and general differences in asymptotic performance of active avoidance behavior. The mPFC and DS require further study as potential neural targets for understanding avoidance susceptibility and, as a result, anxiety vulnerability.

Vulnerability factors in anxiety determined through differences in active-avoidance behavior

The risk for developing anxiety disorders is greater in females and those individuals exhibiting a behaviorally inhibited temperament. Growth of behavioral avoidance in people is a significant predictor of symptom severity in anxiety disorders, including post-traumatic stress disorder. Using an animal model, our lab is examining how the process of learning avoidant behavior may lead certain individuals to develop anxiety. Here we examined whether the known vulnerabilities of female sex and behaviorally inhibited temperament have individual or additive effects upon the acquisition of an active-avoidance response. A discrete trial lever-press escape-avoidance protocol was used to examine the acquisition of behavioral avoidance in male and female Sprague-Dawley (SD) rats and behaviorally inhibited inbred Wistar-Kyoto (WKY) rats. Overall, WKY rats of both sexes were indistinguishable in their behavior during the acquisition of an active-avoidance response, exhibiting quicker acquisition of reinforced responses both between and within session compared to SD rats. Further WKY rats emitted more non-reinforced responses than SD rats. Sex differences were evident in SD rats in both the acquisition of the reinforced response and the emission of non-reinforced responses, with SD females acquiring the response quicker and emitting more non-reinforced responses following lever presses that led to an escape from shock. As vulnerability factors, behavioral inhibition and female sex were each associated with more prevalent reinforced and non-reinforced avoidant behavior, but an additive effect of these 2 factors was not observed. These data illustrate the importance of genetics (both strain and sex) in the assessment and modeling of anxiety vulnerability through the acquisition of active-avoidance responses and the persistence of emitting those responses in periods of non-reinforcement.

Reduction of shock duration as negative reinforcement in free-operant avoidance

Rats were trained on a free-operant procedure in which shock duration was controlled by responses within a limited range of interresponse times. Shocks of 1.6-mA intensity occurred randomly with average density of 10 shocks per minute. As long as interresponse times were 15 seconds or less, any shocks received were at the briefer of two durations (.3 second). Whenever interresponse times exceeded 15 seconds, any shocks received were at the longer duration (1.0 second). For six of eight animals, avoidance responding developed quickly and reached levels of better than 90%. Four yoked animals stopped responding within the first few sessions. Shock duration reduction without change in shock probability or intensity was sufficient for the acquisition and maintenance of avoidance responding.

Warmup in free-operant avoidance as a function of the response-shock = shock-shock interval

Warmup effects, the repeated within-session transitions from ineffective to effective avoidance, were examined with rats on free-operant shock-delay procedures. The shock-shock and response-shock intervals were kept equal as they were varied. As measured by both response rates and shock rates, the magnitude of within-session change in performance was inversely related to the size of the manipulated intervals. The duration of warmup tended to decrease as the intervals were increased. This finding, that increased shock frequencies do not shorten the warmup, appears to be inconsistent with all interpretations of the warmup that have been offered to date. Late-session performances replicated general features of prior experiments, but differed with respect to details of secondary conclusions in previous reports. These differences may stem from the selection of especially proficient avoiders for previous experiments.

Duration-reduction of avoidance sessions as negative reinforcement

Five rats were exposed to a shock-postponement procedure in which responses on each of two levers initially had equivalent effects. After an initial training sequence that ensured at least some responding on each lever, an additional consequence was made conjointly operative on the previously less-preferred lever for each animal. Each response on this lever continued to postpone shock, but also reduced the session duration by one minute. The conjoint contingencies were operative until, through session-shortening responses and the passage of time, the session was scheduled to end in two minutes; during the final two minutes the session-shortening contingency was disabled while the shock-postponement contingency continued to be operative on both levers. When responding shifted to a predominance on the session-shortening lever, the conjoint contingency was shifted to the other lever; for four of the five rats this reversal was followed by two additional reversals. Two of the rats' responding showed clear, strong, and unambiguous sensitivity to the session-shortening contingency. The responding of two others was also systematically controlled by that contingency, but the effects were less clearcut. The fifth animal showed an initial shift when session-shortening was introduced, but its subsequent behavior proved insensitive to reversals of procedure. The results clearly indicate a sensitivity of behavior to events on a time scale quite distinct from that of immediate consequences. They also support an interpretation of avoidance sessions, considered in their entirety, as events whose contingent relationship to behavior can affect that behavior-even in the absence of stimuli that delineate those relationships. Finally, these results support an interpretation of aversively based conditioning within a broader context, analogous to the "open versus closed economy" interpretation of appetitively controlled behavior.

Remote effects of aversive contingencies: Disruption of appetitive behavior by adjacent avoidance sessions

Disruption of ongoing appetitive behavior before and after daily avoidance sessions was examined. After baselines of appetitive responding were established under a fixed-interval 180-s schedule of food presentation, 4 rats were exposed to 40-min sessions of the appetitive schedule just prior to 100-min sessions of electric shock postponement, while another 4 rats received the 40-min appetitive sessions just following daily sessions of shock postponement. In all 8 subjects, fixed-interval response rates decreased relative to baseline levels, the effect being somewhat more pronounced when the avoidance sessions immediately followed. The disruption of fixed-interval responding was only partially reversed when avoidance sessions were discontinued. During the initial exposure to the avoidance sessions, patterns of responding under the fixed-interval schedule were differentially sensitive to disruption, with high baseline response rates generally more disturbed than low rates. These disruptions were not systematically related to changes in reinforcement frequency, which remained fairly high and invariant across all conditions of the experiment; they were also not systematically related to the response rates or to the shock rates of the adjacent avoidance sessions. The results, while qualitatively resembling patterns of conditioned suppression as typically studied, occurred on a greatly expanded time scale. As disruption of behavior extending over time, the present data suggest that some forms of conditioned suppression are perhaps best viewed within a larger temporal context.

Posttraining epinephrine treatment reduces the need for extensive training

We studied whether posttraining epinephrine accelerates the process of acquisition, and whether this effect leads to an enduring high level of performance on a long-term memory test (20 days). Rats were submitted to daily massed (30 trials) two-way active avoidance conditioning sessions followed by systemic epinephrine (0.01 mg/kg) or vehicle, until achieving a learning criterion (80% avoidances in one single session, maximum 14 sessions). By the sixth session, 78% of the animals with posttraining epinephrine attained the learning criterion compared to only the 23% of the rats in the vehicle group. Considering the whole acquisition phase, all the epinephrine treated animals, but only 66.67% in the vehicle group, reached the criterion. On the retention test, both groups maintained the level of performance reached at the end of the acquisition phase. We conclude that epinephrine accelerates the process of acquisition by enhancing memory consolidation, and that this effect might be especially useful in high demanding learning tasks, or for animals with low learning capacities.

Improvement of shuttle-box performance by anterodorsal medial septal lesions in rats

Septal lesions impair a variety of tasks, including inhibitory avoidance and one way active avoidance. In contrast, these lesions improve two-way active avoidance, probably by reducing anxiety. The present work aimed to study whether anterodorsal medial septal lesion (a) improves performance of two-way active avoidance task (Experiment I), as it has been observed with wider septal lesion, and (b) affect anxiety and/or locomotor activity (Experiment II). This precise region was chosen because some evidences suggest that its lesion do not lead to a reduction of anxiety. Lesioned rats tended to make a higher, but statistically non-significant (P=0.074), number of avoidances regardless of the session, being this difference statistically significant on the retention session (RT). The same lesion did not appear to have an anxiolytic effect, and did not affected basal locomotor activity. Different possible explanations of our results are discussed.

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.

Responding changes systematically within sessions during conditioning procedures

When the procedure is held constant within an experimental session, responding often changes systematically within that session. Many of these within-session changes in responding cannot be dismissed as learning curves or by-products of satiation. They have been observed in studies of positive reinforcement, avoidance, punishment, extinction, discrimination, delayed matching to sample, concept formation, maze and alley running, and laboratory analogues of foraging, as well as in the unconditioned substrates of conditioned behavior. When aversive stimuli are used, responding usually increases early in the session. When positive reinforcers are used, responding changes in a variety of ways, including increasing, decreasing, and bitonic functions. Both strong and minimal reinforcement procedures produce within-session decreases in positively reinforced behavior. Within-session changes in responding have substantial theoretical and methodological implications for research in conditioning.

Patterns of responding within sessions

Rates of responding changed systematically across sessions for rats pressing levers and keys and for pigeons pressing treadles and pecking keys. A bitonic function in which response rates increased and then decreased across sessions was the most common finding, although an increase in responding also occurred alone. The change in response rate was usually large. The function relating responding to time in session had the following general characteristics: It appeared early in training, and further experience moved and reduced its peak; it was flatter for longer sessions; and it was flatter, more symmetrical, and peaked later for lower than for higher rates of reinforcement. Factors related to reinforcement exerted more control over the location of the peak rate of responding and the steepness of the decline in response rates than did factors related to responding. These within-session changes in response rates have fundamental theoretical and methodological implications.