Pavlovian Learning Processes in Pediatric Anxiety Disorders: A Critical Review

Deficits in associative and Pavlovian learning are thought to lie at the center of anxiety-related disorders. However, the majority of studies have been carried out in adult populations. The aim of this review was to critically examine the behavioral and neuroimaging literature on Pavlovian learning in pediatric anxiety disorders. We conclude that although there is evidence for deficits in Pavlovian processes (e.g., heightened reactivity to safety cues in anxious samples), the extant literature suffers from key methodological and theoretical issues. We conclude with theoretical and methodological recommendations for future research in order to further elucidate the role of Pavlovian learning in the etiology, maintenance, and treatment of pediatric anxiety disorders.

The Role of the Rodent Lateral Orbitofrontal Cortex in Simple Pavlovian Cue-Outcome Learning Depends on Training Experience

The orbitofrontal cortex (OFC) is a critical structure in the flexible control of value-based behaviors. OFC dysfunction is typically only detected when task or environmental contingencies change, against a backdrop of apparently intact initial acquisition and behavior. While intact acquisition following OFC lesions in simple Pavlovian cue-outcome conditioning is often predicted by models of OFC function, this predicted null effect has not been thoroughly investigated. Here, we test the effects of lesions and temporary muscimol inactivation of the rodent lateral OFC on the acquisition of a simple single cue-outcome relationship. Surprisingly, pretraining lesions significantly enhanced acquisition after overtraining, whereas post-training lesions and inactivation significantly impaired acquisition. This impaired acquisition to the cue reflects a disruption of behavioral control and not learning since the cue could also act as an effective blocking stimulus in an associative blocking procedure. These findings suggest that even simple cue-outcome representations acquired in the absence of OFC function are impoverished. Therefore, while OFC function is often associated with flexible behavioral control in complex environments, it is also involved in very simple Pavlovian acquisition where complex cue-outcome relationships are irrelevant to task performance.

Adaptive behaviour under conflict: Deconstructing extinction, reversal, and active avoidance learning

In complex environments, organisms must respond adaptively to situations despite conflicting information. Under natural (i.e. non-laboratory) circumstances, it is rare that cues or responses are consistently paired with a single outcome. Inconsistent pairings are more common, as are situations where cues and responses are associated with multiple outcomes. Such inconsistency creates conflict, and a response that is adaptive in one scenario may not be adaptive in another. Learning to adjust responses accordingly is important for species to survive and prosper. Here we review the behavioural and brain mechanisms of responding under conflict by focusing on three popular behavioural procedures: extinction, reversal learning, and active avoidance. Extinction involves adapting from reinforcement to non-reinforcement, reversal learning involves swapping the reinforcement of cues or responses, and active avoidance involves performing a response to avoid an aversive outcome, which may conflict with other defensive strategies. We note that each of these phenomena relies on somewhat overlapping neural circuits, suggesting that such circuits may be critical for the general ability to respond appropriately under conflict.

The fate of redundant cues in human predictive learning: The outcome ratio effect

In four experiments, participants were shown a sequence of pairs of pictures of food and asked to predict whether each pair signalled an allergic reaction in a hypothetical patient. The pairs of pictures were used to present two simple discriminations that differed in their outcome ratio. A rich discrimination, 3AX+ BX−, involved three trials in which the compound of two foods, AX, was followed by a reaction, for every trial in which the compound BX was not followed by the outcome. A lean discrimination, CY+ 3DY− was based on the opposite outcome ratio. Upon the completion of this training, participants were asked to rate how likely an individual food would be followed by the allergic reaction. In each experiment, the rating for X was stronger than for Y. This outcome ratio effect poses a challenge for theories of learning that assume changes in associative strength are governed by a common error term, based on the significance of all the cues present on a trial. Instead, the results are consistent with the assumption that changes in associative strength are governed by an individual error term, based on the significance of a single cue.

A model of amygdala function following plastic changes at specific synapses during extinction

The synaptic networks in the amygdala have been the subject of intense interest in recent times, primarily because of the role of this structure in emotion. Fear and its extinction depend on the workings of these networks, with particular interest in extinction because of its potential to ameliorate adverse symptoms associated with post-traumatic stress disorder. Here we place emphasis on the extinction networks revealed by recent techniques, and on the probable plasticity properties of their synaptic connections. We use modules of neurons representing each of the principal components identified as involved in extinction. Each of these modules consists of neural networks, containing specific ratios of excitatory and specialized inhibitory neurons as well as synaptic plasticity mechanisms appropriate for the component of the amygdala they represent. While these models can produce dynamic output, here we concentrate on the equilibrium outputs and do not model the details of the plasticity mechanisms. Pavlovian fear conditioning generates a fear memory in the lateral amygdala module that leads to activation of neurons in the basal nucleus fear module but not in the basal nucleus extinction module. Extinction protocols excite infralimbic medial prefrontal cortex neurons (IL) which in turn excite so-called extinction neurons in the amygdala, leading to the release of endocannabinoids from them and an increase in efficacy of synapses formed by lateral amygdala neurons on them. The model simulations show how such a mechanism could explain experimental observations involving the role of IL as well as endocannabinoids in different temporal phases of extinction.

Altered balance of excitatory and inhibitory learning in a genetically modified mouse model of glutamatergic dysfunction relevant to schizophrenia

The GluA1 AMPAR subunit (encoded by the Gria1 gene) has been implicated in schizophrenia. Gria1 knockout in mice results in recently experienced stimuli acquiring aberrantly high salience. This suggests that GluA1 may be important for learning that is sensitive to the temporal contiguity between events. To test this, mice were trained on a Pavlovian trace conditioning procedure in which the presentation of an auditory cue and food were separated by a temporal interval. Wild-type mice initially learnt, but with prolonged training came to withhold responding during the trace-conditioned cue, responding less than for another cue that was nonreinforced. Gria1 knockout mice, in contrast, showed sustained performance over training, responding more to the trace-conditioned cue than the nonreinforced cue. Therefore, the trace-conditioned cue acquired inhibitory properties (signalling the absence of food) in wild-type mice, but Gria1 deletion impaired the acquisition of inhibition, thus maintaining the stimulus as an excitatory predictor of food. Furthermore, when there was no trace both groups showed successful learning. These results suggest that cognitive abnormalities in disorders like schizophrenia in which gluatamatergic signalling is implicated may be caused by aberrant salience leading to a change in the nature of the information that is encoded.

Youth with substance abuse histories exhibit dysfunctional representation of expected value during a passive avoidance task

Individuals with substance abuse (SA) histories show impairment in the computations necessary for decision-making, including expected value (EV) and prediction error (PE). Neuroimaging findings, however, have been inconsistent. Sixteen youth with (SApositive) and 29 youth without (SAnegative) substance abuse histories completed a passive avoidance task while undergoing functional MRI. The groups did not significantly differ on age, gender composition or IQ. Behavioral results indicated that SApositive youth showed significantly less learning than SAnegative youth over the task. SApositive youth show problems representing EV information when attempting to avoid sub-optimal choices in bilateral inferior frontal gyrus and striatum. Furthermore, SApositive youth showed a significantly increased differential response to reward versus punishment feedback modulated by PE in posterior cingulate cortex relative to SAnegative youth. Disrupted decision-making is likely to exacerbate SA as a failure to represent EV during the avoidance of sub-optimal choices is likely to increase the likelihood of SA. With respect to the representation of PE, future work will be needed to clarify the impact of different substances on the neural systems underpinning PE representation. Moreover, interaction of age/development and substance abuse on PE signaling will need to be explored.

An appetitive conditioned stimulus enhances fear acquisition and impairs fear extinction

Four experiments used between- and within-subject designs to examine appetitive–aversive interactions in rats. Experiments 1 and 2 examined the effect of an excitatory appetitive conditioned stimulus (CS) on acquisition and extinction of conditioned fear. In Experiment 1, a CS shocked in a compound with an appetitive excitor (i.e., a stimulus previously paired with sucrose) underwent greater fear conditioning than a CS shocked in a compound with a neutral stimulus. Conversely, in Experiment 2, a CS extinguished in a compound with an appetitive excitor underwent less extinction than a CS extinguished in a compound with a neutral stimulus. Experiments 3 and 4 compared the amount of fear conditioning to an appetitive excitor and a familiar but neutral target CS when the compound of these stimuli was paired with shock. In each experiment, more fear accrued to the appetitive excitor than to the neutral CS. These results show that an appetitive excitor influences acquisition and extinction of conditioned fear to a neutral CS and itself undergoes a greater associative change than the neutral CS across compound conditioning. They are discussed with respect to the role of motivational information in regulating an associative change in appetitive–aversive interactions.

Attention-related Pearce-Kaye-Hall signals in basolateral amygdala require the midbrain dopaminergic system

BACKGROUND

Neural activity in basolateral amygdala has recently been shown to reflect surprise or attention as predicted by the Pearce-Kaye-Hall model (PKH)--an influential model of associative learning. Theoretically, a PKH attentional signal originates in prediction errors of the kind associated with phasic firing of dopamine neurons. This requirement for prediction errors, coupled with projections from the midbrain dopamine system into basolateral amygdala, suggests that the PKH signal in amygdala may depend on dopaminergic input.

METHODS

To test this, we recorded single unit activity in basolateral amygdala in rats with 6-hydroxydopamine or sham lesions of the ipsilateral midbrain region. Neurons were recorded as the rats performed a task previously used to demonstrate both dopaminergic reward prediction errors and attentional signals in basolateral amygdala neurons.

RESULTS

We found that neurons recorded in sham lesioned rats exhibited the same attention-related PKH signal observed in previous studies. By contrast, neurons recorded in rats with ipsilateral 6-hydroxydopamine lesions failed to show attentional signaling.

CONCLUSIONS

These results indicate a linkage between the neural instantiations of the basolateral complex of the amygdala attentional signal and dopaminergic prediction errors. Such a linkage would have important implications for understanding both normal and aberrant learning and behavior, particularly in diseases thought to have a primary effect on dopamine systems, such as addiction and schizophrenia.

Causal imprinting in causal structure learning

Suppose one observes a correlation between two events, B and C, and infers that B causes C. Later one discovers that event A explains away the correlation between B and C. Normatively, one should now dismiss or weaken the belief that B causes C. Nonetheless, participants in the current study who observed a positive contingency between B and C followed by evidence that B and C were independent given A, persisted in believing that B causes C. The authors term this difficulty in revising initially learned causal structures "causal imprinting." Throughout four experiments, causal imprinting was obtained using multiple dependent measures and control conditions. A Bayesian analysis showed that causal imprinting may be normative under some conditions, but causal imprinting also occurred in the current study when it was clearly non-normative. It is suggested that causal imprinting occurs due to the influence of prior knowledge on how reasoners interpret later evidence. Consistent with this view, when participants first viewed the evidence showing that B and C are independent given A, later evidence with only B and C did not lead to the belief that B causes C.

Increasing histone acetylation in the hippocampus-infralimbic network enhances fear extinction

BACKGROUND

A key finding from recent studies of epigenetic mechanisms of memory is that increasing histone acetylation after a learning experience enhances memory consolidation. This has been demonstrated in several preparations, but little is known about whether excitatory and inhibitory memories are equally sensitive to drugs that promote histone acetylation and how transcriptional changes in the hippocampal-medial prefrontal cortex network contribute to these drug effects.

METHODS

We compare the long-term behavioral consequences of systemic, intrahippocampal and intra-medial prefrontal cortex administration of the histone deacetylase inhibitor sodium butyrate (NaB) after contextual fear conditioning and extinction 1 and/or 14 days later in male c57BL/6J mice (n = 302). Levels of histone acetylation and expression of the product of the immediate-early gene c-Fos were assessed by immunohistochemistry following infusion of NaB into the hippocampus (n = 26).

RESULTS

Across a variety of conditions, the effects of NaB on extinction were larger and more persistent compared to the effects on initial memory formation. NaB administered following weak extinction induced behavioral extinction, infralimbic histone acetylation and c-Fos expression consistent with strong extinction. No similar effect was seen in the prelimbic cortex. The involvement of the infralimbic cortex was confirmed as infusions of NaB into the infralimbic, but not prelimbic cortex, induced extinction enhancements.

CONCLUSIONS

These studies show that the memory modulating ability of drugs that enhance acetylation is sensitive to a variety of behavioral and molecular conditions. We further identify transcriptional changes in the hippocampal-infralimbic circuit associated with extinction enhancements induced by the histone deacetylase inhibitor NaB.

Facets of Pavlovian and operant extinction

Research on extinction is of fundamental importance in both Pavlovian and operant approaches to the experimental analysis of learning. Although these approaches are often motivated by different empirical and theoretical questions, extinction has emerged as a research area in which common themes unite the two approaches. In this review, we focus on some common considerations in the analysis of Pavlovian and operant extinction. These include methodological challenges and interpretational issues in analyzing behavior during and after extinction. We consider the different roles that theory has played in the development of research on extinction in these preparations and conclude with some attention to applications of extinction.

Repeated extinction and reversal learning of an approach response supports an arousal-mediated learning model

We assessed the effects of repeated extinction and reversals of two conditional stimuli (CS+/CS-) on an appetitive conditioned approach response in rats. Three results were observed that could not be accounted for by a simple linear operator model such as the one proposed by Rescorla and Wagner (1972): (1) responding to a CS- declined faster when a CS+ was simultaneously extinguished; (2) reacquisition of pre-extinction performance recovered rapidly within one session; and (3) reversal of CS+/CS- contingencies resulted in a more rapid recovery to the current CS- (former CS+) than the current CS+, accompanied by a slower acquisition of performance to the current CS+. An arousal parameter that mediates learning was introduced to a linear operator model to account for these effects. The arousal-mediated learning model adequately fit the data and predicted data from a second experiment with different rats in which only repeated reversals of CS+/CS- were assessed. According to this arousal-mediated learning model, learning is accelerated by US-elicited arousal and it slows down in the absence of US. Because arousal varies faster than conditioning, the model accounts for the decline in responding during extinction mainly through a reduction in arousal, not a change in learning. By preserving learning during extinction, the model is able to account for relapse effects like rapid reacquisition, renewal, and reinstatement.

A Cognitive Model for Generalization during Sequential Learning

Traditional artificial neural network models of learning suffer fromcatastrophic interference. They are commonly trained to perform only one specific task, and, when trained on a new task, they forget the original task completely. It has been shown that the foundational neurocomputational principles embodied by the Leabra cognitive modeling framework, specifically fast lateral inhibition and a local synaptic plasticity model that incorporates both correlational and error-based components, are sufficient to largely overcome this limitation during the sequential learning of multiple motor skills. Evidence has also provided that Leabra is able to generalize the subsequences of motor skills, when doing so is appropriate. In this paper, we provide a detailed analysis of the extent of generalization possible with Leabra during sequential learning of multiple tasks. For comparison, we measure the generalization exhibited by the backpropagation of error learning algorithm. Furthermore, we demonstrate the applicability of sequential learning to a pair of movement tasks using a simulated robotic arm.

Direct comparisons of the size and persistence of anisomycin-induced consolidation and reconsolidation deficits

An issue of increasing theoretical interest in the study of learning is to compare the processes that follow an initial learning experience (such as learning an association between a context and a shock; memory consolidation processes) with those that follow retrieval of that learning experience (such as exposure to the context in the absence of shock; memory reconsolidation and extinction processes). Much of what is known about these processes comes from separate experiments examining one process or the other; there have been few attempts to compare these processes directly in a single experiment. A challenge in between-experiment comparisons of consolidation and reconsolidation deficits is that they frequently involve comparisons between groups that are not matched on factors that may influence the size and persistence of these deficits (e.g., prior learning experience, memory expression prior to deficit). The following experiments examined the size and persistence of these deficits after matching both the amount of experience with a context and the levels of performance in that context prior to delivery of the protein synthesis inhibitor anisomycin. We found that systemic or intrahippocampal administration of anisomycin caused a deficit in groups receiving context conditioning (consolidation groups) or reactivation (reconsolidation groups) immediately prior to the injections. With systemic injections, the deficit was larger and more persistent in consolidation groups; with intrahippocampal injections, the initial deficit was statistically identical, yet was more persistent in the consolidation group. These experiments showed that when experiences and performance are matched prior to anisomycin injections, consolidation deficits are generally larger and more persistent compared to reconsolidation deficits.

A biologically realistic network model of acquisition and extinction of conditioned fear associations in lateral amygdala neurons

The basolateral amygdala plays an important role in the acquisition and expression of both fear conditioning and fear extinction. To understand how a single structure could encode these "opposite" memories, we developed a biophysical network model of the lateral amygdala (LA) neurons during auditory fear conditioning and extinction. Membrane channel properties were selected to match waveforms and firing properties of pyramidal cells and interneurons in LA, from published in vitro studies. Hebbian plasticity was implemented in excitatory AMPA and inhibitory GABA(A) receptor-mediated synapses to model learning. The occurrence of synaptic potentiation versus depression was determined by intracellular calcium levels, according to the calcium control hypothesis. The model was able to replicate conditioning- and extinction-induced changes in tone responses of LA neurons in behaving rats. Our main finding is that LA activity during both acquisition and extinction can be controlled by a balance between pyramidal cell and interneuron activations. Extinction training depressed conditioned synapses and also potentiated local interneurons, thereby inhibiting the responses of pyramidal cells to auditory input. Both long-term depression and potentiation of inhibition were required to initiate and maintain extinction. The model provides insights into the sites of plasticity in conditioning and extinction, the mechanism of spontaneous recovery, and the role of amygdala NMDA receptors in extinction learning.

A modular theory of learning and performance

We describe a theory to account for the acquisition and extinction of response rate (conditioning) and pattern (timing). This modular theory is a development of packet theory (Kirkpatrick, 2002; Kirkpatrick & Church, 2003) that adds a distinction between pattern and strength memories, as well as contributing closed-form equations. We describe the theory using equations related to a flow diagram and illustrate it by an application to an experiment with repeated acquisitions and extinctions of a multiple-cued-interval procedure using rats. The parameter estimates for the theory were based on a calibration sample from the data, and the predictions for different measures of performance on a validation sample from the same data (cross-validation). The theory's predictions were similar to predictions based on the reliability of the behavior.

Model-based fMRI and its application to reward learning and decision making

In model-based functional magnetic resonance imaging (fMRI), signals derived from a computational model for a specific cognitive process are correlated against fMRI data from subjects performing a relevant task to determine brain regions showing a response profile consistent with that model. A key advantage of this technique over more conventional neuroimaging approaches is that model-based fMRI can provide insights into how a particular cognitive process is implemented in a specific brain area as opposed to merely identifying where a particular process is located. This review will briefly summarize the approach of model-based fMRI, with reference to the field of reward learning and decision making, where computational models have been used to probe the neural mechanisms underlying learning of reward associations, modifying action choice to obtain reward, as well as in encoding expected value signals that reflect the abstract structure of a decision problem. Finally, some of the limitations of this approach will be discussed.

Effects of repeated acquisitions and extinctions on response rate and pattern

The procedure developed by R. A. Rescorla (2002) was used to study the effects of repeated acquisitions and extinctions of head entry responses into a food cup by rats. In each of 4 20-session phases, food was delivered at the end of particular 30-s auditory and visual stimuli, but not at the end of different 30-s auditory and visual stimuli. Based on response rates to individual stimuli and compound stimuli, the increase in response rate in acquisition occurred more rapidly than the decrease in extinction. Acquisition, but not extinction, occurred faster after successive transitions between acquisition and extinction. Temporal gradients of responding developed during acquisition and remained during extinction. Conclusions based on mean response rate, temporal gradients, and transfer tests were consistent.

Generalization gradients for acquisition and extinction in human contingency learning

Two experiments investigated the perceptual generalization of acquisition and extinction in human contingency learning. In Experiment 1, the degree of perceptual similarity between the acquisition stimulus and the generalization stimulus was manipulated over five groups. This successfully generated a generalization gradient of acquisition. In the subsequent phase, the response to the generalization stimulus was extinguished in each group. Finally, the acquisition stimulus was presented again. The response recovered differently over groups, thereby establishing the generalization gradient of extinction. In Experiment 2, the acquisition stimulus itself was extinguished before the set of generalization stimuli was tested between groups. One group evidenced a response recovery at test, which suggests that the gradient of acquisition is somewhat broader than the gradient of extinction.

Elemental representations of stimuli in associative learning

This article reviews evidence and theories concerning the nature of stimulus representations in Pavlovian conditioning. It focuses on the elemental approach developed in stimulus sampling theory (R. C. Atkinson & W. K. Estes, 1963; R. R. Bush & F. Mosteller, 1951b) and extended by I. P. L. McLaren and N. J. Mackintosh (2000, 2002) and contrasts this with models that invoke notions of configural representations that uniquely code for different patterns of stimulus inputs (e.g., J. M. Pearce, 1987, 1994; R. A. Rescorla & A. R. Wagner, 1972; A. R. Wagner & S. E. Brandon, 2001). The article then presents a new elemental model that emphasizes interactions between stimulus elements. This model is shown to explain a range of behavioral findings, including those (e.g., negative patterning and biconditional discriminations) traditionally thought to be beyond the explanatory capabilities of elemental models. Moreover, the model offers a ready explanation for recent findings reported by R. A. Rescorla (2000, 2001, 2002b) concerning the way that stimuli with different conditioning histories acquire associative strength when conditioned in compound.

Effects of post-session injections of anisomycin on the extinction of a spatial preference and on the acquisition of a spatial reversal preference

Four experiments with C57BL/6 mice examined the effects of protein synthesis inhibition on extinction of spatial preferences in the Morris water maze. The protein synthesis inhibitor anisomycin was injected systemically immediately after spatial reversal sessions, which consisted of eight, two, or one reversal trials. Anisomycin had no effect on extinction of a spatial preference with eight reversal trials per session. When only two reversal trials were administered per session, anisomycin slowed extinction initially, although full extinction occurred with further training even in the presence of anisomycin. In both cases, anisomycin blocked acquisition of the reversal preference. When anisomycin injections followed a single simple extinction trial, performance was not disrupted. These findings suggest that acquisition and extinction may involve different molecular processes, and they do not support the idea that brief extinction trials induce a protein synthesis-dependent reconsolidation phase of spatial memory.

The learning curve: implications of a quantitative analysis

The negatively accelerated, gradually increasing learning curve is an artifact of group averaging in several commonly used basic learning paradigms (pigeon autoshaping, delay- and trace-eye-blink conditioning in the rabbit and rat, autoshaped hopper entry in the rat, plus maze performance in the rat, and water maze performance in the mouse). The learning curves for individual subjects show an abrupt, often step-like increase from the untrained level of responding to the level seen in the well trained subject. The rise is at least as abrupt as that commonly seen in psychometric functions in stimulus detection experiments. It may indicate that the appearance of conditioned behavior is mediated by an evidence-based decision process, as in stimulus detection experiments. If the appearance of conditioned behavior is taken instead to reflect the increase in an underlying associative strength, then a negligible portion of the function relating associative strength to amount of experience is behaviorally visible. Consequently, rate of learning cannot be estimated from the group-average curve; the best measure is latency to the onset of responding, determined for each subject individually.