Design of a neurally plausible model of fear learning

FraidyRat: A Virtual Module Examining the Neural Circuitry Underlying Fear Conditioning

FraidyRat is a teaching tool that allows students to investigate the neural basis of fear conditioning and extinction using a virtual rat with a virtual brain. FraidyRat models well-known phenomena at both a behavioral and neural level. Students use virtual versions of tract tracing, systemic and intracerebrally infused drugs, neural recording, and electrical stimulation to understand the neural substrates underlying the observed behavior. This module helps students develop critical thinking skills in order to deduce immediate cause and effect as well as inductive reasoning to grasp the broader scheme. This module utilizes scaffolded instruction and formative assessment to shape the thinking of students as they unfold and discover the neural mechanisms responsible for fear conditioning and extinction in FraidyRat, which largely reflect what is found in real rats. Experience with this three-week module resulted in students showing significant gains in content knowledge as well as a trend toward gains in critical thinking. An attitudinal questionnaire showed that students had an overall positive experience. This module can be replicated at any institution with just a computer. All materials are available at: https://mdcune.psych.ucla.edu/modules/fraidy-rat.

Neural Circuits Underlying Social Fear in Rodents: An Integrative Computational Model

Social avoidance in rodents arises from a complex interplay between the prefrontal cortex and subcortical structures, such as the ventromedial hypothalamus and the dorsal periaqueductal gray matter. Experimental studies are revealing the contribution of these areas, but an integrative view and model of how they interact to produce adaptive behavior are still lacking. Here, we present a computational model of social avoidance, proposing a set of integrated hypotheses on the possible macro organization of the brain system underlying this phenomenon. The model is validated by accounting for several different empirical findings and produces predictions to be tested in future experiments.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Extinction and discrimination in a Bayesian model of context fear conditioning (BaconX)

The extinction of contextual fear is commonly an essential requirement for successful exposure therapy for fear disorders. However, experimental work on extinction of contextual fear is limited, and there little or no directly relevant theoretical work. Here, we extend BACON, a neurocomputational model of context fear conditioning that provides plausible explanations for a number of aspects of context fear conditioning, to deal with extinction (calling the model BaconX). In this model, contextual representations are formed in the hippocampus and association of fear to them occurs in the amygdala. Representation creation, conditionability, and development of between‐session extinction are controlled by degree of confidence (assessed by the Bayesian weight of evidence) that an active contextual representation is in fact that of the current context (i.e., is “valid”). The model predicts that: (1) extinction which persists between sessions will occur only if at a sessions end there is high confidence that the active representation is valid. It follows that the shorter the context placement‐to‐US (shock) interval (“PSI”) and the less is therefore learned about context, the longer extinction sessions must be for enduring extinction to occur, while too short PSIs will preclude successful extinction. (2) Short‐PSI deficits can be rescued by contextual exposure even after conditioning has occurred. (3) Learning to discriminate well between a conditioned and similar safe context requires representations of each to form, which may not occur if PSI was too short. (4) Extinction‐causing inhibition must be applied downstream of the conditioning locus for reasonable generalization properties to be generated. (5) Context change tends to cause return of extinguished contextual fear. (6). Extinction carried out in the conditioning context generalizes better than extinction executed in contexts to which fear has generalized (as done in exposure therapy). (7) BaconX suggests novel approaches to exposure therapy.

Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives

A fascinating challenge in the field of human-robot interaction is the possibility to endow robots with emotional intelligence in order to make the interaction more intuitive, genuine, and natural. To achieve this, a critical point is the capability of the robot to infer and interpret human emotions. Emotion recognition has been widely explored in the broader fields of human-machine interaction and affective computing. Here, we report recent advances in emotion recognition, with particular regard to the human-robot interaction context. Our aim is to review the state of the art of currently adopted emotional models, interaction modalities, and classification strategies and offer our point of view on future developments and critical issues. We focus on facial expressions, body poses and kinematics, voice, brain activity, and peripheral physiological responses, also providing a list of available datasets containing data from these modalities.

A Computational Model Integrating Multiple Phenomena on Cued Fear Conditioning, Extinction, and Reinstatement

Conditioning, extinction, and reinstatement are fundamental learning processes of animal adaptation, also strongly involved in human pathologies such as post-traumatic stress disorder, anxiety, depression, and dependencies. Cued fear conditioning, extinction, restatement, and systematic manipulations of the underlying brain amygdala and medial prefrontal cortex, represent key experimental paradigms to study such processes. Numerous empirical studies have revealed several aspects and the neural systems and plasticity underlying them, but at the moment we lack a comprehensive view. Here we propose a computational model based on firing rate leaky units that contributes to such integration by accounting for 25 different experiments on fear conditioning, extinction, and restatement, on the basis of a single neural architecture having a structure and plasticity grounded in known brain biology. This allows the model to furnish three novel contributions to understand these open issues: (a) the functioning of the central and lateral amygdala system supporting conditioning; (b) the role played by the endocannabinoids system in within- and between-session extinction; (c) the formation of three important types of neurons underlying fear processing, namely fear, extinction, and persistent neurons. The model integration of the results on fear conditioning goes substantially beyond what was done in previous models.

A systems-neuroscience model of phasic dopamine

We describe a neurobiologically informed computational model of phasic dopamine signaling to account for a wide range of findings, including many considered inconsistent with the simple reward prediction error (RPE) formalism. The central feature of this PVLV framework is a distinction between a primary value (PV) system for anticipating primary rewards (Unconditioned Stimuli [USs]), and a learned value (LV) system for learning about stimuli associated with such rewards (CSs). The LV system represents the amygdala, which drives phasic bursting in midbrain dopamine areas, while the PV system represents the ventral striatum, which drives shunting inhibition of dopamine for expected USs (via direct inhibitory projections) and phasic pausing for expected USs (via the lateral habenula). Our model accounts for data supporting the separability of these systems, including individual differences in CS-based (sign-tracking) versus US-based learning (goal-tracking). Both systems use competing opponent-processing pathways representing evidence for and against specific USs, which can explain data dissociating the processes involved in acquisition versus extinction conditioning. Further, opponent processing proved critical in accounting for the full range of conditioned inhibition phenomena, and the closely related paradigm of second-order conditioning. Finally, we show how additional separable pathways representing aversive USs, largely mirroring those for appetitive USs, also have important differences from the positive valence case, allowing the model to account for several important phenomena in aversive conditioning. Overall, accounting for all of these phenomena strongly constrains the model, thus providing a well-validated framework for understanding phasic dopamine signaling. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Hippocampal subfield volumes in abstinent men and women with a history of alcohol use disorder

Alcohol Use Disorder (AUD) has been associated with abnormalities in hippocampal volumes, but these relationships have not been fully explored with respect to sub-regional volumes, nor in association with individual characteristics such as age, gender differences, drinking history, and memory. The present study examined the impact of those variables in relation to hippocampal subfield volumes in abstinent men and women with a history of AUD. Using Magnetic Resonance Imaging at 3 Tesla, we obtained brain images from 67 participants with AUD (31 women) and 64 nonalcoholic control (NC) participants (31 women). The average duration of the most recent period of sobriety for AUD participants was 7.1 years. We used Freesurfer 6.0 to segment the hippocampus into 12 regions. These were imputed into statistical models to examine the relationships of brain volume with AUD group, age, gender, memory, and drinking history. Interactions with gender and age were of particular interest. Compared to the NC group, the AUD group had approximately 5% smaller subiculum, CA1, molecular layer, and hippocampal tail regions. Age was negatively associated with volumes for the AUD group in the subiculum and the hippocampal tail, but no significant interactions with gender were identified. The relationships for delayed and immediate memory with hippocampal tail volume differed for AUD and NC groups: Higher scores on tests of immediate and delayed memory were associated with smaller volumes in the AUD group, but larger volumes in the NC group. Length of sobriety was associated with decreasing CA1 volume in women (0.19% per year) and increasing volume size in men (0.38% per year). The course of abstinence on CA1 volume differed for men and women, and the differential relationships of subfield volumes to age and memory could indicate a distinction in the impact of AUD on functions of the hippocampal tail. These findings confirm and extend evidence that AUD, age, gender, memory, and abstinence differentially impact volumes of component parts of the hippocampus.

Beyond the "Conceptual Nervous System": Can computational cognitive neuroscience transform learning theory?

In the last century, learning theory has been dominated by an approach assuming that associations between hypothetical representational nodes can support the acquisition of knowledge about the environment. The similarities between this approach and connectionism did not go unnoticed to learning theorists, with many of them explicitly adopting a neural network approach in the modeling of learning phenomena. Skinner famously criticized such use of hypothetical neural structures for the explanation of behavior (the "Conceptual Nervous System"), and one aspect of his criticism has proven to be correct: theory underdetermination is a pervasive problem in cognitive modeling in general, and in associationist and connectionist models in particular. That is, models implementing two very different cognitive processes often make the exact same behavioral predictions, meaning that important theoretical questions posed by contrasting the two models remain unanswered. We show through several examples that theory underdetermination is common in the learning theory literature, affecting the solvability of some of the most important theoretical problems that have been posed in the last decades. Computational cognitive neuroscience (CCN) offers a solution to this problem, by including neurobiological constraints in computational models of behavior and cognition. Rather than simply being inspired by neural computation, CCN models are built to reflect as much as possible about the actual neural structures thought to underlie a particular behavior. They go beyond the "Conceptual Nervous System" and offer a true integration of behavioral and neural levels of analysis.

Human Pavlovian fear conditioning conforms to probabilistic learning

Learning to predict threat from environmental cues is a fundamental skill in changing environments. This aversive learning process is exemplified by Pavlovian threat conditioning. Despite a plethora of studies on the neural mechanisms supporting the formation of associations between neutral and aversive events, our computational understanding of this process is fragmented. Importantly, different computational models give rise to different and partly opposing predictions for the trial-by-trial dynamics of learning, for example expressed in the activity of the autonomic nervous system (ANS). Here, we investigate human ANS responses to conditioned stimuli during Pavlovian fear conditioning. To obtain precise, trial-by-trial, single-subject estimates of ANS responses, we build on a statistical framework for psychophysiological modelling. We then consider previously proposed non-probabilistic models, a simple probabilistic model, and non-learning models, as well as different observation functions to link learning models with ANS activity. Across three experiments, and both for skin conductance (SCR) and pupil size responses (PSR), a probabilistic learning model best explains ANS responses. Notably, SCR and PSR reflect different quantities of the same model: SCR track a mixture of expected outcome and uncertainty, while PSR track expected outcome alone. In summary, by combining psychophysiological modelling with computational learning theory, we provide systematic evidence that the formation and maintenance of Pavlovian threat predictions in humans may rely on probabilistic inference and includes estimation of uncertainty. This could inform theories of neural implementation of aversive learning.

Dentate Gyrus Contributes to Retrieval as well as Encoding: Evidence from Context Fear Conditioning, Recall, and Extinction

Dentate gyrus (DG) is widely thought to provide a teaching signal that enables hippocampal encoding of memories, but its role during retrieval is poorly understood. Some data and models suggest that DG plays no role in retrieval; others encourage the opposite conclusion. To resolve this controversy, we evaluated the effects of optogenetic inhibition of dorsal DG during context fear conditioning, recall, generalization, and extinction in male mice. We found that (1) inhibition during training impaired context fear acquisition; (2) inhibition during recall did not impair fear expression in the training context, unless mice had to distinguish between similar feared and neutral contexts; (3) inhibition increased generalization of fear to an unfamiliar context that was similar to a feared one and impaired fear expression in the conditioned context when it was similar to a neutral one; and (4) inhibition impaired fear extinction. These effects, as well as several seemingly contradictory published findings, could be reproduced by BACON (Bayesian Context Fear Algorithm), a physiologically realistic hippocampal model positing that acquisition and retrieval both involve coordinated activity in DG and CA3. Our findings thus suggest that DG contributes to retrieval and extinction, as well as to the initial establishment of context fear.SIGNIFICANCE STATEMENT Despite abundant evidence that the hippocampal dentate gyrus (DG) plays a critical role in memory, it remains unclear whether the role of DG relates to memory acquisition or retrieval. Using contextual fear conditioning and optogenetic inhibition, we show that DG contributes to both of these processes. Using computational simulations, we identify specific mechanisms through which the suppression of DG affects memory performance. Finally, we show that DG contributes to fear extinction learning, a process in which learned fear is attenuated through exposures to a fearful context in the absence of threat. Our data resolve a long-standing question about the role of DG in memory and provide insight into how disorders affecting DG, including aging, stress, and depression, influence cognitive processes.

Biologically based neural circuit modelling for the study of fear learning and extinction

The neuronal systems that promote protective defensive behaviours have been studied extensively using Pavlovian conditioning. In this paradigm, an initially neutral-conditioned stimulus is paired with an aversive unconditioned stimulus leading the subjects to display behavioural signs of fear. Decades of research into the neural bases of this simple behavioural paradigm uncovered that the amygdala, a complex structure comprised of several interconnected nuclei, is an essential part of the neural circuits required for the acquisition, consolidation and expression of fear memory. However, emerging evidence from the confluence of electrophysiological, tract tracing, imaging, molecular, optogenetic and chemogenetic methodologies, reveals that fear learning is mediated by multiple connections between several amygdala nuclei and their distributed targets, dynamical changes in plasticity in local circuit elements as well as neuromodulatory mechanisms that promote synaptic plasticity. To uncover these complex relations and analyse multi-modal data sets acquired from these studies, we argue that biologically realistic computational modelling, in conjunction with experiments, offers an opportunity to advance our understanding of the neural circuit mechanisms of fear learning and to address how their dysfunction may lead to maladaptive fear responses in mental disorders.

Synaptic competition in the lateral amygdala and the stimulus specificity of conditioned fear: a biophysical modeling study

Competitive synaptic interactions between principal neurons (PNs) with differing intrinsic excitability were recently shown to determine which dorsal lateral amygdala (LAd) neurons are recruited into a fear memory trace. Here, we explored the contribution of these competitive interactions in determining the stimulus specificity of conditioned fear associations. To this end, we used a realistic biophysical computational model of LAd that included multi-compartment conductance-based models of 800 PNs and 200 interneurons. To reproduce the continuum of spike frequency adaptation displayed by PNs, the model included three subtypes of PNs with high, intermediate, and low spike frequency adaptation. In addition, the model network integrated spatially differentiated patterns of excitatory and inhibitory connections within LA, dopaminergic and noradrenergic inputs, extrinsic thalamic and cortical tone afferents to simulate conditioned stimuli as well as shock inputs for the unconditioned stimulus. Last, glutamatergic synapses in the model could undergo activity-dependent plasticity. Our results suggest that plasticity at both excitatory (PN-PN) and di-synaptic inhibitory (PN-ITN and, particularly, ITN-PN) connections are major determinants of the synaptic competition governing the assignment of PNs to the memory trace. The model also revealed that training-induced potentiation of PN-PN synapses promotes, whereas that of ITN-PN synapses opposes, stimulus generalization. Indeed, suppressing plasticity of PN-PN synapses increased, whereas preventing plasticity of interneuronal synapses decreased the CS specificity of PN recruitment. Overall, our results indicate that the plasticity configuration imprinted in the network by synaptic competition ensures memory specificity. Given that anxiety disorders are characterized by tendency to generalize learned fear to safe stimuli or situations, understanding how plasticity of intrinsic LAd synapses regulates the specificity of learned fear is an important challenge for future experimental studies.

Mechanisms underlying the formation of the amygdalar fear memory trace: A computational perspective

Recent experimental and modeling studies on the lateral amygdala (LA) have implicated intrinsic excitability and competitive synaptic interactions among principal neurons (PNs) in the formation of auditory fear memories. The present modeling studies, conducted over an expanded range of intrinsic excitability in the network, revealed that only excitable PNs that received tone inputs participate in the competition. Strikingly, the number of model PNs integrated into the fear memory trace remained constant despite the much larger range considered, and model runs highlighted several conditioning-induced tone responsive characteristics of the various PN populations. Furthermore, these studies showed that although excitation was important, disynaptic inhibition among PNs is the dominant mechanism that keeps the number of plastic PNs stable despite large variations in the network's excitability. Finally, we found that the overall level of inhibition in the model network determines the number of projection cells integrated into the fear memory trace.

Developmental emergence of fear/threat learning: neurobiology, associations and timing

Pavlovian fear or threat conditioning, where a neutral stimulus takes on aversive properties through pairing with an aversive stimulus, has been an important tool for exploring the neurobiology of learning. In the past decades, this neurobehavioral approach has been expanded to include the developing infant. Indeed, protracted postnatal brain development permits the exploration of how incorporating the amygdala, prefrontal cortex and hippocampus into this learning system impacts the acquisition and expression of aversive conditioning. Here, we review the developmental trajectory of these key brain areas involved in aversive conditioning and relate it to pups' transition to independence through weaning. Overall, the data suggests that adult-like features of threat learning emerge as the relevant brain areas become incorporated into this learning. Specifically, the developmental emergence of the amygdala permits cue learning and the emergence of the hippocampus permits context learning. We also describe unique features of learning in early life that block threat learning and enhance interaction with the mother or exploration of the environment. Finally, we describe the development of a sense of time within this learning and its involvement in creating associations. Together these data suggest that the development of threat learning is a useful tool for dissecting adult-like functioning of brain circuits, as well as providing unique insights into ecologically relevant developmental changes.

Severe learning deficits of IRSp53 mutant mice are caused by altered NMDA receptor-dependent signal transduction

Learning and memory is dependent on postsynaptic architecture and signaling processes in forebrain regions. The insulin receptor substrate protein of 53 kDa (IRSp53, also known as Baiap2) is a signaling and adapter protein in forebrain excitatory synapses. Mice deficient in IRSp53 display enhanced levels of postsynaptic N-methyl-D-aspartate receptors (NMDARs) and long-term potentiation (LTP) associated with severe learning deficits. In humans, reduced IRSp53/Baiap2 expression is associated with a variety of neurological disorders including autism, schizophrenia, and Alzheimer's disease. Here, we analyzed mice lacking one copy of the gene coding for IRSp53 using behavioral tests including contextual fear conditioning and the puzzle box. We show that a 50% reduction in IRSp53 levels strongly affects the performance in fear-evoking learning paradigms. This correlates with increased targeting of NMDARs to the postsynaptic density (PSD) in hippocampi of both heterozygous and knock out (ko) mice at the expense of extrasynaptic NMDARs. As hippocampal NMDAR-dependent LTP is enhanced in IRSp53-deficient mice, we investigated signaling cascades important for the formation of fear-evoked memories. Here, we observed a dramatic increase in cAMP response element-binding protein-dependent signaling in heterozygous and IRSp53-deficient mice, necessary for the transcriptional dependent phase of LTP. In contrast, activation of the MAPK and Akt kinase pathways required for translation-dependent phase of LTP are reduced. Our data suggest that loss or even the reduction in IRSp53 increases NMDAR-dependent cAMP responsive element-binding protein activation in the hippocampus, and interferes with the ability of mice to learn upon anxiety-related stimuli. We show here that a moderate reduction in the postsynaptic protein IRSp53 in mice leads to an increase in postsynaptic NMDA receptors. Both in heterozygous and IRSp53 deficient mice, this is associated with altered postsynaptic signal transduction, and poor performance of mice in fear-associated learning paradigms, indicating that precise control of postsynaptic NMDA receptor density is essential for memory formation.

The Origins and Organization of Vertebrate Pavlovian Conditioning

Pavlovian conditioning is the process by which we learn relationships between stimuli and thus constitutes a basic building block for how the brain constructs representations of the world. We first review the major concepts of Pavlovian conditioning and point out many of the pervasive misunderstandings about just what conditioning is. This brings us to a modern redefinition of conditioning as the process whereby experience with a conditional relationship between stimuli bestows these stimuli with the ability to promote adaptive behavior patterns that did not occur before the experience. Working from this framework, we provide an in-depth analysis of two examples, fear conditioning and food-based appetitive conditioning, which include a description of the only partially overlapping neural circuitry of each. We also describe how these circuits promote the basic characteristics that define Pavlovian conditioning, such as error-correction-driven regulation of learning.

Out with the old and in with the new: Synaptic mechanisms of extinction in the amygdala

Considerable research indicates that long-term synaptic plasticity in the amygdala underlies the acquisition of emotional memories, including those learned during Pavlovian fear conditioning. Much less is known about the synaptic mechanisms involved in other forms of associative learning, including extinction, that update fear memories. Extinction learning might reverse conditioning-related changes (e.g., depotentiation) or induce plasticity at inhibitory synapses (e.g., long-term potentiation) to suppress conditioned fear responses. Either mechanism must account for fear recovery phenomena after extinction, as well as savings of extinction after fear recovery. This article is part of a Special Issue entitled SI: Brain and Memory.

A Bayesian context fear learning algorithm/automaton

Contextual fear conditioning is thought to involve the synaptic plasticity-dependent establishment in hippocampus of representations of to-be-conditioned contexts which can then become associated with USs in the amygdala. A conceptual and computational model of this process is proposed in which contextual attributes are assumed to be sampled serially and randomly during contextual exposures. Given this assumption, moment-to-moment information about such attributes will often be quite different from one exposure to another and, in particular, between exposures during which representations are created, exposures during which conditioning occurs, and during recall sessions. This presents challenges to current conceptual models of hippocampal function. In order to meet these challenges, our model's hippocampus was made to operate in different modes during representation creation and recall, and non-hippocampal machinery was constructed that controlled these hippocampal modes. This machinery uses a comparison between contextual information currently observed and information associated with existing hippocampal representations of familiar contexts to compute the Bayesian Weight of Evidence that the current context is (or is not) a known one, and it uses this value to assess the appropriateness of creation or recall modes. The model predicts a number of known phenomena such as the immediate shock deficit, spurious fear conditioning to contexts that are absent but similar to actually present ones, and modulation of conditioning by pre-familiarization with contexts. It also predicts a number of as yet unknown phenomena.

Hebbian and neuromodulatory mechanisms interact to trigger associative memory formation

A long-standing hypothesis termed "Hebbian plasticity" suggests that memories are formed through strengthening of synaptic connections between neurons with correlated activity. In contrast, other theories propose that coactivation of Hebbian and neuromodulatory processes produce the synaptic strengthening that underlies memory formation. Using optogenetics we directly tested whether Hebbian plasticity alone is both necessary and sufficient to produce physiological changes mediating actual memory formation in behaving animals. Our previous work with this method suggested that Hebbian mechanisms are sufficient to produce aversive associative learning under artificial conditions involving strong, iterative training. Here we systematically tested whether Hebbian mechanisms are necessary and sufficient to produce associative learning under more moderate training conditions that are similar to those that occur in daily life. We measured neural plasticity in the lateral amygdala, a brain region important for associative memory storage about danger. Our findings provide evidence that Hebbian mechanisms are necessary to produce neural plasticity in the lateral amygdala and behavioral memory formation. However, under these conditions Hebbian mechanisms alone were not sufficient to produce these physiological and behavioral effects unless neuromodulatory systems were coactivated. These results provide insight into how aversive experiences trigger memories and suggest that combined Hebbian and neuromodulatory processes interact to engage associative aversive learning.

Representation of aversive prediction errors in the human periaqueductal gray

Pain is a primary driver of learning and motivated action. It is also a target of learning, as nociceptive brain responses are shaped by learning processes. We combined an instrumental pain avoidance task with an axiomatic approach to assessing fMRI signals related to prediction errors (PEs), which drive reinforcement-based learning. We found that pain PEs were encoded in the periaqueductal gray (PAG), a structure important for pain control and learning in animal models. Axiomatic tests combined with dynamic causal modeling suggested that ventromedial prefrontal cortex, supported by putamen, provides an expected value-related input to the PAG, which then conveys PE signals to prefrontal regions important for behavioral regulation, including orbitofrontal, anterior mid-cingulate and dorsomedial prefrontal cortices. Thus, pain-related learning involves distinct neural circuitry, with implications for behavior and pain dynamics.

Computational search for hypotheses concerning the endocannabinoid contribution to the extinction of fear conditioning

Fear conditioning, in which a cue is conditioned to elicit a fear response, and extinction, in which a previously conditioned cue no longer elicits a fear response, depend on neural plasticity occurring within the amygdala. Projection neurons in the basolateral amygdala (BLA) learn to respond to the cue during fear conditioning, and they mediate fear responding by transferring cue signals to the output stage of the amygdala. Some BLA projection neurons retain their cue responses after extinction. Recent work shows that activation of the endocannabinoid system is necessary for extinction, and it leads to long-term depression (LTD) of the GABAergic synapses that inhibitory interneurons make onto BLA projection neurons. Such GABAergic LTD would enhance the responses of the BLA projection neurons that mediate fear responding, so it would seem to oppose, rather than promote, extinction. To address this paradox, a computational analysis of two well-known conceptual models of amygdaloid plasticity was undertaken. The analysis employed exhaustive state-space search conducted within a declarative programming environment. The analysis reveals that GABAergic LTD actually increases the number of synaptic strength configurations that achieve extinction while preserving the cue responses of some BLA projection neurons in both models. The results suggest that GABAergic LTD helps the amygdala retain cue memory during extinction even as the amygdala learns to suppress the previously conditioned response. The analysis also reveals which features of both models are essential for their ability to achieve extinction with some cue memory preservation, and suggests experimental tests of those features.

The contextual brain: implications for fear conditioning, extinction and psychopathology

Contexts surround and imbue meaning to events; they are essential for recollecting the past, interpreting the present and anticipating the future. Indeed, the brain's capacity to contextualize information permits enormous cognitive and behavioural flexibility. Studies of Pavlovian fear conditioning and extinction in rodents and humans suggest that a neural circuit including the hippocampus, amygdala and medial prefrontal cortex is involved in the learning and memory processes that enable context-dependent behaviour. Dysfunction in this network may be involved in several forms of psychopathology, including post-traumatic stress disorder, schizophrenia and substance abuse disorders.

Anatomy and computational modeling of networks underlying cognitive-emotional interaction

The classical dichotomy between cognition and emotion equated the first with rationality or logic and the second with irrational behaviors. The idea that cognition and emotion are separable, antagonistic forces competing for dominance of mind has been hard to displace despite abundant evidence to the contrary. For instance, it is now known that a pathological absence of emotion leads to profound impairment of decision making. Behavioral observations of this kind are corroborated at the mechanistic level: neuroanatomical studies reveal that brain areas typically described as underlying either cognitive or emotional processes are linked in ways that imply complex interactions that do not resemble a simple mutual antagonism. Instead, physiological studies and network simulations suggest that top-down signals from prefrontal cortex realize "cognitive control" in part by either suppressing or promoting emotional responses controlled by the amygdala, in a way that facilitates adaptation to changing task demands. Behavioral, anatomical, and physiological data suggest that emotion and cognition are equal partners in enabling a continuum or matrix of flexible behaviors that are subserved by multiple brain regions acting in concert. Here we focus on neuroanatomical data that highlight circuitry that structures cognitive-emotional interactions by directly or indirectly linking prefrontal areas with the amygdala. We also present an initial computational circuit model, based on anatomical, physiological, and behavioral data to explicitly frame the learning and performance mechanisms by which cognition and emotion interact to achieve flexible behavior.

A model of amygdala-hippocampal-prefrontal interaction in fear conditioning and extinction in animals

Empirical research has shown that the amygdala, hippocampus, and ventromedial prefrontal cortex (vmPFC) are involved in fear conditioning. However, the functional contribution of each brain area and the nature of their interactions are not clearly understood. Here, we extend existing neural network models of the functional roles of the hippocampus in classical conditioning to include interactions with the amygdala and prefrontal cortex. We apply the model to fear conditioning, in which animals learn physiological (e.g. heart rate) and behavioral (e.g. freezing) responses to stimuli that have been paired with a highly aversive event (e.g. electrical shock). The key feature of our model is that learning of these conditioned responses in the central nucleus of the amygdala is modulated by two separate processes, one from basolateral amygdala and signaling a positive prediction error, and one from the vmPFC, via the intercalated cells of the amygdala, and signaling a negative prediction error. In addition, we propose that hippocampal input to both vmPFC and basolateral amygdala is essential for contextual modulation of fear acquisition and extinction. The model is sufficient to account for a body of data from various animal fear conditioning paradigms, including acquisition, extinction, reacquisition, and context specificity effects. Consistent with studies on lesioned animals, our model shows that damage to the vmPFC impairs extinction, while damage to the hippocampus impairs extinction in a different context (e.g., a different conditioning chamber from that used in initial training in animal experiments). We also discuss model limitations and predictions, including the effects of number of training trials on fear conditioning.

Towards a unified model of pavlovian conditioning: short review of trace conditioning models

There are three basic paradigms of classical conditioning: delay, trace and context conditioning where presentation of a conditioned stimulus (CS) or a context typically predicts an unconditioned stimulus (US). In delay conditioning CS and US normally coterminate, whereas in trace conditioning an interval of time exists between CS termination and US onset. The modeling of trace conditioning is a rather difficult computational problem and is a challenge to the behavior and connectionist approaches mainly due to a time gap between CS and US. To account for trace conditioning, Pavlov (Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex, Oxford University Press, London, 1927) postulated the existence of a stimulus "trace" in the nervous system. Meanwhile, there exist many other options for solving this association problem. There are several excellent reviews of computational models of classical conditioning but none has thus far been devoted to trace conditioning. Eight representative models of trace conditioning aimed at building a prospective model are being reviewed below in a brief form. As a result, one of them, comprising the most important features of its predecessors, can be suggested as a real candidate for a unified model of trace conditioning.