Selective entorhinal and nonselective cortical-hippocampal region lesions, but not selective hippocampal lesions, disrupt learned irrelevance in rabbit eyeblink conditioning

Neurocomputational correlates of learned irrelevance in humans

Inappropriate behaviors may result from acquiring maladaptive associations between irrelevant information in the environment and important events, such as reward or punishment. Pre-exposure effects are believed to prevent the expression of irrelevant associations. For example, learned irrelevance delays the expression of associations between conditioned (CS) and unconditioned (US) stimuli following their uncorrelated presentation. The neuronal substrates of pre-exposure effects in humans are largely unknown because these effects rapidly attenuate when using traditional pre-exposure paradigms. The latter are therefore incompatible with neuroimaging approaches that require many trial repetitions. Moreover, large methodological differences between animal and human research on pre-exposure effects challenge the presumption of shared neurocognitive substrates, and question the prevalent use of pre-exposure effects in animals to model symptoms of human mental disorders. To overcome these limitations, we combined a novel learned irrelevance task with model-based fMRI. We report the results of a model that describes learned irrelevance as a dynamic process, which evolves across trials and integrates the weighting between two state-action values pertaining to 'CS-no US' associations (acquired during pre-exposure) and 'CS-US' associations (acquired during subsequent conditioning). This relative weighting correlated i) positively with the learned irrelevance effect observed in the behavioral task, ii) positively with activity in the entorhinal cortex, and iii) negatively with activity in the nucleus accumbens (NAcc). Furthermore, the model updates the relative weighting of the two state-action values via two separate prediction error (PE) signals that allow the dynamic accumulation of evidence for the CS to predict the 'US' or a 'no US' outcome. One PE signal, designed to increase the relative weight of 'CS-US' associations following 'US' outcomes, correlated with activity in the NAcc, while another PE signal, designed to increase the relative weight of 'CS-no US' associations following 'no US' outcomes, correlated with activity in the basolateral amygdala. By extending previous animal observations to humans, the present study provides a novel approach to foster translational research on pre-exposure effects.

US alone trials presented during acquisition do not disrupt classical eyeblink conditioning: Empirical and computational findings

Studies of partial reinforcement in eyeblink conditioning have typically shown slower learning of a CS-US association when paired CS-US trials are interleaved with CS-alone trials. However, recent work has shown that CS-US learning is not slowed by interleaved US-alone trials. This discrepancy is surprising since both partial reinforcement protocols reduce the total number of paired CS-US trials. Previously, Kimble et al. (1955) reported that inserting a block of US-alone trials during CS-US training did not disrupt eyeblink acquisition. Here, we sought to replicate and extend these findings by comparing interleaved vs. blocked US-alone trials during CS-US paired training. Ninety-seven undergraduates volunteered for this experiment for research credit. Participants received 60 acquisition trials, consisting of either 100% CS-US paired trials, 50% US-alone trials intermixed with CS-US paired trials, or a block of 20 US-alone trials inserted between blocks of 20 CS-US trials. We also utilized a previously published computational model of hippocampal and cerebellar learning to test the effects of these US-alone protocols. Both empirical and computational results supported the finding that US-alone trials, either intermixed or inserted as a block of trials, do not disrupt acquisition of conditioned eyeblinks. Possible neural substrates of these US-alone effects are discussed.

Complementary learning systems within the hippocampus: a neural network modelling approach to reconciling episodic memory with statistical learning

A growing literature suggests that the hippocampus is critical for the rapid extraction of regularities from the environment. Although this fits with the known role of the hippocampus in rapid learning, it seems at odds with the idea that the hippocampus specializes in memorizing individual episodes. In particular, the Complementary Learning Systems theory argues that there is a computational trade-off between learning the specifics of individual experiences and regularities that hold across those experiences. We asked whether it is possible for the hippocampus to handle both statistical learning and memorization of individual episodes. We exposed a neural network model that instantiates known properties of hippocampal projections and subfields to sequences of items with temporal regularities. We found that the monosynaptic pathway-the pathway connecting entorhinal cortex directly to region CA1-was able to support statistical learning, while the trisynaptic pathway-connecting entorhinal cortex to CA1 through dentate gyrus and CA3-learned individual episodes, with apparent representations of regularities resulting from associative reactivation through recurrence. Thus, in paradigms involving rapid learning, the computational trade-off between learning episodes and regularities may be handled by separate anatomical pathways within the hippocampus itself.This article is part of the themed issue 'New frontiers for statistical learning in the cognitive sciences'.

Enhanced Eyeblink Conditioning in Behaviorally Inhibited Individuals is Disrupted by Proactive Interference Following US Alone Pre-exposures

Anxiety vulnerable individuals exhibit enhanced acquisition of conditioned eyeblinks as well as enhanced proactive interference from conditioned stimulus (CS) or unconditioned stimulus (US) alone pre-exposures (Holloway et al., 2012). US alone pre-exposures disrupt subsequent conditioned response (CR) acquisition to CS-US paired trials as compared to context pre-exposure controls. While Holloway et al. (2012) reported enhanced acquisition in high trait anxiety individuals in the context condition, anxiety vulnerability effects were not reported for the US alone pre-exposure group. It appears from the published data that there were no differences between high and low anxiety individuals in the US alone condition. In the work reported here, we sought to extend the findings of enhanced proactive interference with US alone pre-exposures to determine if the enhanced conditioning was disrupted by proactive interference procedures. We also were interested in the spontaneous eyeblinks during the pre-exposure phase of training. We categorized individuals as anxiety vulnerability or non-vulnerable individuals based scores on the Adult Measure of Behavioral Inhibition (AMBI). Sixty-six participants received 60 trials consisting of 30 US alone or context alone pre-exposures followed by 30 CS-US trials. US alone pre-exposures not only disrupted CR acquisition overall, but behaviorally inhibited (BI) individuals exhibited enhanced proactive interference as compared to non-inhibited (NI) individuals. In addition, US alone pre-exposures disrupted the enhanced acquisition observed in BI individuals as compared to NI individuals following context alone pre-exposures. Differences were also found in rates of spontaneous eyeblinks between BI and NI individuals during context pre-exposure. Our findings will be discussed in the light of the neural substrates of eyeblink conditioning as well as possible factors such as hypervigilance in the amygdala and hippocampal systems, and possible learned helplessness. Applications of these findings of enhanced proactive interference in BI individuals to pre-exposure therapies to reduce anxiety disorders such as posttraumatic stress disorder (PTSD) will be discussed.

CS-US preexposure effects on trace eyeblink conditioning in young rats: Potential implications for functional brain development

Recent studies of delay eyeblink conditioning (EBC) in young rats have demonstrated different effects of various conditioned and unconditioned stimulus (CS-US) preexposure conditions on learning at different ages. The present study extends this research to trace EBC. Subjects experienced 1 of 3 preexposure conditions (paired CS-US, unpaired CS-US, or no stimuli) at either 20 or 24 days of age. Four days later, they were conditioned using either trace (Experiment 1) or delay (Experiment 2) EBC parameters. Results were similar at both ages tested. Paired preexposure facilitated acquisition of delay but not trace relative to context preexposure. Unpaired preexposure impaired acquisition of both delay and trace. These behavioral findings provide a foundation for hypotheses about the functional maturation of cerebellar, hippocampal, and entorhinal learning circuits.

Cortico-hippocampal interaction and adaptive stimulus representation: a neurocomputational theory of associative learning and memory

Computational models of the hippocampal region link psychological theories of associative learning with their underlying physiological and anatomical substrates. Our approach to theory development began with a broad description of the computations that depend on the hippocampal region in classical conditioning (Gluck and Myers, 1993 and Gluck and Myers, 2001). In this initial model, the hippocampal region was treated as an Information-processing system that transformed stimulus representations, compressing (making more similar) representations of inputs that co-occur or are otherwise redundant, while differentiating (or making less similar) representations of inputs that predict different future events. This model led to novel predictions for the behavioral consequences of hippocampal-region lesions in rodents and of brain damage in humans who have amnesia or are in the earliest stages of Alzheimer's disease. Many of these predictions have, since been confirmed by our lab and others. Functional brain imaging studies have provided further supporting evidence. In more recent computational modeling, we have shown how some aspects of this proposed information-processing function could emerge from known anatomical and physiological characteristics of the hippocampal region, including the entorhinal cortex and the septo-hippocampal cholinergic system. The modeling to date lays the groundwork for future directions that increase the depth of detail of the biological modeling, as well as the breadth of behavioral phenomena addressed. In particular, we are working now to reconcile these kinds of incremental associative learning models with other models of the hippocampal region that account for the rapid formation of declarative memories.

Disruption of learned irrelevance in acute schizophrenia in a novel continuous within-subject paradigm suitable for fMRI

Learned irrelevance (LIrr) is closely related to latent inhibition (LI). In LI a to-be-conditioned stimulus (CS) is prexposed alone prior to the opportunity to learn an association between the CS and an unconditioned stimulus (UCS). In LIrr preexposure consists of intermixed presentations of both CS and UCS in a random relationship to each other. In both paradigms preexposure leads in normal subjects to reduced or retarded learning of the CS-UCS association. Acute schizophrenics fail to show LI. LI is usually demonstrated as a one-off, between-groups difference in trials to learning, so posing problems for neuroimaging. We have developed a novel, continuous, within-subject paradigm in which normal subjects show robust and repeated LIrr. We show that this paradigm is suitable for functional magnetic resonance imaging (fMRI) and gives rise, in normal subjects, to activation in the hippocampal formation, consistent with data from animal experiments on LI. We also report, consistent with previous studies of LI, loss (indeed, significant reversal) of LIrr in acute (first 2 weeks of current psychotic episode) schizophrenics. Chronic schizophrenics failed to demonstrate learning, precluding measurement in this group of LIrr. These findings establish the likely value of the new paradigm for neuroimaging studies of attentional dysfunction in acute schizophrenia.

Integrating Incremental Learning and Episodic Memory Models of the Hippocampal Region

By integrating previous computational models of corticohippocampal function, the authors develop and test a unified theory of the neural substrates of familiarity, recollection, and classical conditioning. This approach integrates models from 2 traditions of hippocampal modeling, those of episodic memory and incremental learning, by drawing on an earlier mathematical model of conditioning, SOP (A. Wagner, 1981). The model describes how a familiarity signal may arise from parahippocampal cortices, giving a novel explanation for the finding that the neural response to a stimulus in these regions decreases with increasing stimulus familiarity. Recollection is ascribed to the hippocampus proper. It is shown how the properties of episodic representations in the neocortex, parahippocampal gyrus, and hippocampus proper may explain phenomena in classical conditioning. The model reproduces the effects of hippocampal, septal, and broad hippocampal region lesions on contextual modulation of classical conditioning, blocking, learned irrelevance, and latent inhibition.

Computational models of the hippocampal region: linking incremental learning and episodic memory

The hippocampal region, a group of brain structures important for learning and memory, has been the focus of a large number of computational models. These tend to fall into two groups: (1) models of the role of the hippocampal region in incremental learning, which focus on the development of new representations that are sensitive to stimulus regularities and environmental context; (2) models that focus on the role of the hippocampal region in the rapid storage and retrieval of episodic memories. Rather than being in conflict, it is becoming apparent that both approaches are partially correct and might reflect the different functions of substructures of the hippocampal region. Future computational models will help to elaborate how these different substructures interact.

Dissociating medial temporal and basal ganglia memory systems with a latent learning task

The medial temporal (MT) lobes and basal ganglia have both been implicated as brain substrates of associative learning. Here, we show a dissociation between medial temporal and basal ganglia damage using a latent learning task, in which prior exposure to cues, uncorrelated with each other, slows subsequent learning of an association between them. Consistent with prior work, we found a robust exposure effect in healthy controls, with exposed controls learning more slowly than non-exposed controls. This effect was abolished in medial temporal amnesia: both exposed and non-exposed amnesic patients learned at the same speed. A group of patients with basal ganglia damage due to Parkinson's disease showed a reversal of the effect: exposed subjects learned faster than non-exposed subjects. Our findings point to distinct and dissociable contributions of medial temporal lobe and basal ganglia structures to learning and memory.

Blocking in rabbit eyeblink conditioning is not due to learned inattention: indirect support for an error correction mechanism of blocking

Blocking is a classical conditioning task in which prior training to one cue such as a tone reduces learning about a second cue such as a light, when subsequently trained as a tone-light compound. Blocking has been theorized to come about through a US-modulated error correction mechanism by Rescorla & Wagner (1972) as well as through a mechanism of learned inattention as theorized by Mackintosh (1973). In the case of eyeblink conditioning, an error correction mechanism has been hypothesized to take place in the cerebellum while some form of inattention has been hypothesized to take place in the hippocampal region. The hypothesis we are testing is whether the mechanism of learned inattention is involved in blocking in rabbit eyeblink conditioning. If blocking in eyeblink conditioning is produced by a mechanism of learned inattention, then training to a previously blocked cue should be slower than training to that cue in a naïve animal. Rabbits that had received tone training followed by tone-light training exhibited blocking. Rabbits that had been previously blocked to the light acquired conditioned responses to the light at the same rate as naive rabbits. This finding failed to support the hypothesis that blocking in rabbit eyeblink conditioning is due to learned inattention, but does support the Rescorla-Wagner mechanism of error correction. The present finding along with previous work on error correction mechanism in the cerebellar-brainstem circuit (Kim et al., 1998) lend support to the theory that blocking, at least in rabbit eyeblink conditioning, seems to be due to an error correction mechanism rather than a learned inattention mechanism.

Parallel neural systems for classical conditioning: support from computational modeling

Classical conditioning has been explained by two main types of theories that postulate different learning mechanisms. Rescorla and Wagner (1972) put forth a theory in which conditioning is based on the ability of the US to drive learning through error correction. Alternatively, Mackintosh (1973) put forth a theory in which the ability of the CS to be associated with the unconditioned stimulus is modulated. We have proposed a reconciliation of these two mechanisms as working in parallel within different neural systems: a cerebellar system for US modulation and a hippocampal system for CS modulation. We developed a computational model of cerebellar function in eyeblink conditioning based on the error correction mechanism of the Rescorla-Wagner rule in which learning-related activity from the cerebellum inhibits the inferior olive, which is the US input pathway to the cerebellum (Gluck et al., 1994). We developed a computational model of the hippocampal region that forms altered representations of conditioned stimuli based on their behavioral outcomes (Gluck & Myers, 1993; Myers et al., 1995). Overall, computational modeling and empirical findings support the idea that, at least in the case of eyeblink conditioning, there may be two different neural systems: the cerebellum which mediates US-based error correction and hippocampus which alters representations of CSs.