Integration of Parallel Opposing Memories Underlies Memory Extinction

Neuroscience: Tantalized Flies Are Primed for Satiety

How animals maintain and switch between distinct motivational states is an important question in neuroscience. New work in Drosophila identifies an excitatory neuronal circuit that builds up mating drive while priming itself for satiety.

A plea for more attention to mental representations

In this invited essay I plead for a renewed attention to mental representations in research into psychopathology and its treatment. With the cognitive revolution in the second half of the previous century most psychological models of psychopathology were based on people's capacity to form mental representations of the world. However, much research effort was put into investigating cognitive biases instead of investigating mental representations, and attention shifted towards such biases, including claims that they are causal in psychopathology. Similar claims were made about emotion regulation difficulties. Moreover, in many treatment models that were developed the focus was more on cognitive biases and emotion regulation, than on underlying representations. In this essay I argue that the causal status of cognitive biases, emotion regulation, and similar phenomena is overestimated, and that in clinical reality such phenomena can be best conceptualized as output of activated mental representations. Moreover, I argue that the disappointing effectiveness and protection to relapse of many current psychological treatments are related to not addressing the underlying mental representation. Next I discuss aspects of mental representations that are important for understanding and treating psychopathology, after which I discuss the clinical implications. I sketch how better understanding the specificities of mental representations can help us to improve psychological treatments, and I make some suggestions for future research.

Memory engrams: Recalling the past and imagining the future

In 1904, Richard Semon introduced the term "engram" to describe the neural substrate for storing memories. An experience, Semon proposed, activates a subset of cells that undergo off-line, persistent chemical and/or physical changes to become an engram. Subsequent reactivation of this engram induces memory retrieval. Although Semon's contributions were largely ignored in his lifetime, new technologies that allow researchers to image and manipulate the brain at the level of individual neurons has reinvigorated engram research. We review recent progress in studying engrams, including an evaluation of evidence for the existence of engrams, the importance of intrinsic excitability and synaptic plasticity in engrams, and the lifetime of an engram. Together, these findings are beginning to define an engram as the basic unit of memory.

The natverse, a versatile toolbox for combining and analysing neuroanatomical data

To analyse neuron data at scale, neuroscientists expend substantial effort reading documentation, installing dependencies and moving between analysis and visualisation environments. To facilitate this, we have developed a suite of interoperable open-source R packages called the natverse. The natverse allows users to read local and remote data, perform popular analyses including visualisation and clustering and graph-theoretic analysis of neuronal branching. Unlike most tools, the natverse enables comparison across many neurons of morphology and connectivity after imaging or co-registration within a common template space. The natverse also enables transformations between different template spaces and imaging modalities. We demonstrate tools that integrate the vast majority of Drosophila neuroanatomical light microscopy and electron microscopy connectomic datasets. The natverse is an easy-to-use environment for neuroscientists to solve complex, large-scale analysis challenges as well as an open platform to create new code and packages to share with the community.

The Drosophila Mushroom Body: From Architecture to Algorithm in a Learning Circuit

The Drosophila brain contains a relatively simple circuit for forming Pavlovian associations, yet it achieves many operations common across memory systems. Recent advances have established a clear framework for Drosophila learning and revealed the following key operations: a) pattern separation, whereby dense combinatorial representations of odors are preprocessed to generate highly specific, nonoverlapping odor patterns used for learning; b) convergence, in which sensory information is funneled to a small set of output neurons that guide behavioral actions; c) plasticity, where changing the mapping of sensory input to behavioral output requires a strong reinforcement signal, which is also modulated by internal state and environmental context; and d) modularization, in which a memory consists of multiple parallel traces, which are distinct in stability and flexibility and exist in anatomically well-defined modules within the network. Cross-module interactions allow for higher-order effects where past experience influences future learning. Many of these operations have parallels with processes of memory formation and action selection in more complex brains.

Differential Role for a Defined Lateral Horn Neuron Subset in Naïve Odor Valence in Drosophila

Value coding of external stimuli in general, and odor valence in particular, is crucial for survival. In flies, odor valence is thought to be coded by two types of neurons: mushroom body output neurons (MBONs) and lateral horn (LH) neurons. MBONs are classified as neurons that promote either attraction or aversion, but not both, and they are dynamically activated by upstream neurons. This dynamic activation updates the valence values. In contrast, LH neurons receive scaled, but non-dynamic, input from their upstream neurons. It remains unclear how such a non-dynamic system generates differential valence values. Recently, PD2a1/b1 LH neurons were demonstrated to promote approach behavior at low odor concentration in starved flies. Here, we demonstrate that at high odor concentrations, these same neurons contribute to avoidance in satiated flies. The contribution of PD2a1/b1 LH neurons to aversion is context dependent. It is diminished in starved flies, although PD2a1/b1 neural activity remains unchanged, and at lower odor concentration. In addition, PD2a1/b1 aversive effect develops over time. Thus, our results indicate that, even though PD2a1/b1 LH neurons transmit hard-wired output, their effect on valence can change. Taken together, we suggest that the valence model described for MBONs does not hold for LH neurons.

Recurrent architecture for adaptive regulation of learning in the insect brain

Dopaminergic neurons (DANs) drive learning across the animal kingdom, but the upstream circuits that regulate their activity and thereby learning remain poorly understood. We provide a synaptic-resolution connectome of the circuitry upstream of all DANs in a learning center, the mushroom body of Drosophila larva. We discover afferent sensory pathways and a large population of neurons that provide feedback from mushroom body output neurons and link distinct memory systems (aversive and appetitive). We combine this with functional studies of DANs and their presynaptic partners and with comprehensive circuit modeling. We find that DANs compare convergent feedback from aversive and appetitive systems, which enables the computation of integrated predictions that may improve future learning. Computational modeling reveals that the discovered feedback motifs increase model flexibility and performance on learning tasks. Our study provides the most detailed view to date of biological circuit motifs that support associative learning.

Cellular and circuit mechanisms of olfactory associative learning in Drosophila

Recent years have witnessed significant progress in understanding how memories are encoded, from the molecular to the cellular and the circuit/systems levels. With a good compromise between brain complexity and behavioral sophistication, the fruit fly Drosophila melanogaster is one of the preeminent animal models of learning and memory. Here we review how memories are encoded in Drosophila, with a focus on short-term memory and an eye toward future directions. Forward genetic screens have revealed a large number of genes and transcripts necessary for learning and memory, some acting cell-autonomously. Further, the relative numerical simplicity of the fly brain has enabled the reverse engineering of learning circuits with remarkable precision, in some cases ascribing behavioral phenotypes to single neurons. Functional imaging and physiological studies have localized and parsed the plasticity that occurs during learning at some of the major loci. Connectomics projects are significantly expanding anatomical knowledge of the nervous system, filling out the roadmap for ongoing functional/physiological and behavioral studies, which are being accelerated by simultaneous tool development. These developments have provided unprecedented insight into the fundamental neural principles of learning, and lay the groundwork for deep understanding in the near future.

Opponent processes in visual memories: A model of attraction and repulsion in navigating insects' mushroom bodies

Solitary foraging insects display stunning navigational behaviours in visually complex natural environments. Current literature assumes that these insects are mostly driven by attractive visual memories, which are learnt when the insect's gaze is precisely oriented toward the goal direction, typically along its familiar route or towards its nest. That way, an insect could return home by simply moving in the direction that appears most familiar. Here we show using virtual reconstructions of natural environments that this principle suffers from fundamental drawbacks, notably, a given view of the world does not provide information about whether the agent should turn or not to reach its goal. We propose a simple model where the agent continuously compares its current view with both goal and anti-goal visual memories, which are treated as attractive and repulsive respectively. We show that this strategy effectively results in an opponent process, albeit not at the perceptual level-such as those proposed for colour vision or polarisation detection-but at the level of the environmental space. This opponent process results in a signal that strongly correlates with the angular error of the current body orientation so that a single view of the world now suffices to indicate whether the agent should turn or not. By incorporating this principle into a simple agent navigating in reconstructed natural environments, we show that it overcomes the usual shortcomings and produces a step-increase in navigation effectiveness and robustness. Our findings provide a functional explanation to recent behavioural observations in ants and why and how so-called aversive and appetitive memories must be combined. We propose a likely neural implementation based on insects' mushroom bodies' circuitry that produces behavioural and neural predictions contrasting with previous models.

Dopaminergic neurons can influence heat-box place learning in Drosophila

Dopamine provides crucial neuromodulatory functions in several insect and rodent learning and memory paradigms. However, an early study suggested that dopamine may be dispensable for aversive place memory in Drosophila. Here we tested the involvement of particular dopaminergic neurons in place learning and memory. We used the thermogenetic tool Gr28bD to activate protocerebral anterior medial (PAM) cluster and non-PAM dopaminergic neurons in an operant way in heat-box place learning. We show that activation of PAM neurons influences performance during place learning, but not during memory testing. These findings provide a gateway to explore how dopamine influences place learning.

Toward nanoscale localization of memory engrams in Drosophila

The Mushroom Body (MB) is the primary location of stored associative memories in the Drosophila brain. We discuss recent advances in understanding the MB's neuronal circuits made using advanced light microscopic methods and cell-type-specific genetic tools. We also review how the compartmentalized nature of the MB's organization allows this brain area to form and store memories with widely different dynamics.

Towards a functional connectome in Drosophila

The full functionality of the brain is determined by its molecular, cellular and circuit structure. Modern neuroscience now prioritizes the mapping of whole brain connectomes by detecting all direct neuron to neuron synaptic connections, a feat first accomplished for C. elegans, a full reconstruction of a 302-neuron nervous system. Efforts at Janelia Research Campus will soon reconstruct the whole brain connectomes of a larval and an adult Drosophila. These connectomes will provide a framework for incorporating detailed neural circuit information that Drosophila neuroscientists have gathered over decades. But when viewed in the context of a whole brain, it becomes difficult to isolate the contributions of distinct circuits, whether sensory systems or higher brain regions. The complete wiring diagram tells us that sensory information is not only processed in separate channels, but that even the earliest sensory layers are strongly synaptically interconnected. In the higher brain, long-range projections densely interconnect major brain regions and convergence centers that integrate input from different sensory systems. Furthermore, we also need to understand the impact of neuronal communication beyond direct synaptic modulation. Nevertheless, all of this can be pursued with Drosophila, combining connectomics with a diverse array of genetic tools and behavioral paradigms that provide effective approaches to entire brain function.

Sex-dependent opposite effects of a tropomyosin-related kinase B receptor (TrkB) agonist 7,8-dihydroxyflavone on cued fear extinction in mice

Tropomyosin-related kinase B receptor (TrkB) is one of the new candidate receptors for drugs targeting psychiatric and neurodegenerative disorders. Recently, 7,8-dihydroxyflavone (7,8-DHF) has been identified as a selective TrkB agonist that crosses the blood-brain barrier after oral or intraperitoneal administration, and it enhances cued fear extinction in male rodents. However, its effects on females remain unclear. Preclinical research including both sexes is important for the development of treatment, particularly, for stress-related disorders such as post-traumatic stress disorder because such disorders are more prevalent in women. Therefore, we investigated the effects of 7,8-DHF on cued and contextual fear extinction in both male and female mice. Here we demonstrated that the administration of 7,8-DHF before each extinction session attenuated cued fear extinction in females; conversely, it enhanced cued fear extinction in males. However, administration of 7,8-DHF immediately after each extinction session did not affect cued fear extinction in either sex. Moreover, in contextual fear extinction, administration of 7,8-DHF before each extinction session did not affect fear extinction in either sex. Thus, 7,8-DHF showed sex-dependent opposite effects on cued fear extinction in mice when administered before but not immediately after each extinction session. Our results could contribute to the development of pharmacotherapy involving 7,8-DHF, particularly for stress-related disorders.

A neural mechanism for deprivation state-specific expression of relevant memories in Drosophila

Motivational states modulate how animals value sensory stimuli and engage in goal-directed behaviors. The motivational states of thirst and hunger are represented in the brain by shared and unique neuromodulatory systems. However, it is unclear how such systems interact to coordinate the expression of appropriate state-specific behavior. We show that the activity of two brain neurons expressing leucokinin neuropeptide is elevated in thirsty and hungry flies, and that leucokinin release is necessary for state-dependent expression of water- and sugar-seeking memories. Leucokinin inhibits two types of mushroom-body-innervating dopaminergic neurons (DANs) to promote thirst-specific water memory expression, whereas it activates other mushroom-body-innervating DANs to facilitate hunger-dependent sugar memory expression. Selection of hunger- or thirst-appropriate memory emerges from competition between leucokinin and other neuromodulatory hunger signals at the level of the DANs. Therefore, coordinated modulation of the dopaminergic system allows flies to prioritize the expression of the relevant state-dependent motivated behavior.

Neuronal mechanisms underlying innate and learned olfactory processing in Drosophila

Olfaction allows animals to adapt their behavior in response to different chemical cues in their environment. How does the brain efficiently discriminate different odors to drive appropriate behavior, and how does it flexibly assign value to odors to adjust behavior according to experience? This review traces neuronal mechanisms underlying these processes in adult Drosophila melanogaster from olfactory receptors to higher brain centers. We highlight neural circuit principles such as lateral inhibition, segregation and integration of olfactory channels, temporal accumulation of sensory evidence, and compartmentalized synaptic plasticity underlying associative memory.

Reversal learning in Drosophila larvae

Adjusting behavior to changed environmental contingencies is critical for survival, and reversal learning provides an experimental handle on such cognitive flexibility. Here, we investigate reversal learning in larval Drosophila Using odor-taste associations, we establish olfactory reversal learning in the appetitive and the aversive domain, using either fructose as a reward or high-concentration sodium chloride as a punishment, respectively. Reversal learning is demonstrated both in differential and in absolute conditioning, in either valence domain. In differential conditioning, the animals are first trained such that an odor A is paired, for example, with the reward whereas odor B is not (A+/B); this is followed by a second training phase with reversed contingencies (A/B+). In absolute conditioning, odor B is omitted, such that the animals are first trained with paired presentations of A and reward, followed by unpaired training in the second training phase. Our results reveal "true" reversal learning in that the opposite associative effects of both the first and the second training phase are detectable after reversed-contingency training. In what is a surprisingly quick, one-trial contingency adjustment in the Drosophila larva, the present study establishes a simple and genetically easy accessible study case of cognitive flexibility.

A Neural Circuit Arbitrates between Persistence and Withdrawal in Hungry Drosophila

In pursuit of food, hungry animals mobilize significant energy resources and overcome exhaustion and fear. How need and motivation control the decision to continue or change behavior is not understood. Using a single fly treadmill, we show that hungry flies persistently track a food odor and increase their effort over repeated trials in the absence of reward suggesting that need dominates negative experience. We further show that odor tracking is regulated by two mushroom body output neurons (MBONs) connecting the MB to the lateral horn. These MBONs, together with dopaminergic neurons and Dop1R2 signaling, control behavioral persistence. Conversely, an octopaminergic neuron, VPM4, which directly innervates one of the MBONs, acts as a brake on odor tracking by connecting feeding and olfaction. Together, our data suggest a function for the MB in internal state-dependent expression of behavior that can be suppressed by external inputs conveying a competing behavioral drive.

•

Hunger motivates persistent food odor tracking even without reward

•

Two synaptically connected MBONs, -γ1pedc>αβ and -α2sc, regulate odor tracking

•

Octopamine neurons connect feeding and counteract MBON and odor tracking

•

Dopaminergic neurons and Dop1R2 signaling promote persistent tracking

An optogenetic analogue of second-order reinforcement in Drosophila

In insects, odours are coded by the combinatorial activation of ascending pathways, including their third-order representation in mushroom body Kenyon cells. Kenyon cells also receive intersecting input from ascending and mostly dopaminergic reinforcement pathways. Indeed, in Drosophila, presenting an odour together with activation of the dopaminergic mushroom body input neuron PPL1-01 leads to a weakening of the synapse between Kenyon cells and the approach-promoting mushroom body output neuron MBON-11. As a result of such weakened approach tendencies, flies avoid the shock-predicting odour in a subsequent choice test. Thus, increased activity in PPL1-01 stands for punishment, whereas reduced activity in MBON-11 stands for predicted punishment. Given that punishment-predictors can themselves serve as punishments of second order, we tested whether presenting an odour together with the optogenetic silencing of MBON-11 would lead to learned odour avoidance, and found this to be the case. In turn, the optogenetic activation of MBON-11 together with odour presentation led to learned odour approach. Thus, manipulating activity in MBON-11 can be an analogue of predicted, second-order reinforcement.

Distinct Dopamine Receptor Pathways Underlie the Temporal Sensitivity of Associative Learning

Animals rely on the relative timing of events in their environment to form and update predictive associations, but the molecular and circuit mechanisms for this temporal sensitivity remain incompletely understood. Here, we show that olfactory associations in Drosophila can be written and reversed on a trial-by-trial basis depending on the temporal relationship between an odor cue and dopaminergic reinforcement. Through the synchronous recording of neural activity and behavior, we show that reversals in learned odor attraction correlate with bidirectional neural plasticity in the mushroom body, the associative olfactory center of the fly. Two dopamine receptors, DopR1 and DopR2, contribute to this temporal sensitivity by coupling to distinct second messengers and directing either synaptic depression or potentiation. Our results reveal how dopamine-receptor signaling pathways can detect the order of events to instruct opposing forms of synaptic and behavioral plasticity, allowing animals to flexibly update their associations in a dynamic environment.

Proactive and retroactive interference with associative memory consolidation in the snail Lymnaea is time and circuit dependent

Interference-based forgetting occurs when new information acquired either before or after a learning event attenuates memory expression (proactive and retroactive interference, respectively). Multiple learning events often occur in rapid succession, leading to competition between consolidating memories. However, it is unknown what factors determine which memory is remembered or forgotten. Here, we challenge the snail, Lymnaea, to acquire two consecutive similar or different memories and identify learning-induced changes in neurons of its well-characterized motor circuits. We show that when new learning takes place during a stable period of the original memory, proactive interference only occurs if the two consolidating memories engage the same circuit mechanisms. If different circuits are used, both memories survive. However, any new learning during a labile period of consolidation promotes retroactive interference and the acquisition of the new memory. Therefore, the effect of interference depends both on the timing of new learning and the underlying neuronal mechanisms.

A Dopaminergic Basis for Fear Extinction

It is a joyous relief when an event we dread fails to materialize. In fear extinction, the appetitive nature of an omitted aversive event is not a mere epiphenomenon but drives the reduction of fear responses and the formation of long-term extinction memories. Dopamine emerges as key neurobiological mediator of these related processes.

Kinase and Phosphatase Engagement Is Dissociated Between Memory Formation and Extinction

Associative long-term memories (LTMs) support long-lasting behavioral changes resulting from sensory experiences. Retrieval of a stable LTM by means of a large number of conditioned stimulus (CS) alone presentations produces inhibition of the original memory through extinction. Currently, there are two opposing hypotheses to account for the neural mechanisms supporting extinction. The unlearning hypothesis posits that extinction affects the original memory trace by reverting the synaptic changes supporting LTM. On the contrary, the new learning hypothesis proposes that extinction is simply the formation of a new associative memory that inhibits the expression of the original one. We propose that detailed analysis of extinction-associated molecular mechanisms could help distinguish between these hypotheses. Here we will review experimental evidence regarding the role of protein kinases and phosphatases (K&P) on LTM formation and extinction. Even though K&P regulate both memory processes, their participation appears to be dissociated. LTM formation recruits kinases, but is constrained by phosphatases. Memory extinction presents a more diverse molecular landscape, requiring phosphatases and some kinases, but also being constrained by kinase activity. Based on the available evidence, we propose a new theoretical model for memory extinction: a neuronal segregation of K&P supports a combination of time-dependent reversible inhibition of the original memory [CS-unconditioned stimulus (US)], with establishment of a new associative memory trace (CS-noUS).

Visual Contrast Modulates Operant Learning Responses in Larval Zebrafish

The larval zebrafish is a promising vertebrate model organism to study neural mechanisms underlying learning and memory due to its small brain and rich behavioral repertoire. Here, we report on a high-throughput operant conditioning system for zebrafish larvae, which can simultaneously train 12 fish to associate a visual conditioned pattern with electroshocks. We find that the learning responses can be enhanced by the visual contrast, not the spatial features of the conditioned patterns, highlighted by several behavioral metrics. By further characterizing the learning curves as well as memory extinction, we demonstrate that the percentage of learners and the memory length increase as the conditioned pattern becomes darker. Finally, little difference in operant learning responses was found between AB wild-type fish and elavl3:H2B-GCaMP6f transgenic fish.

The role of attractive and repellent scene memories in ant homing (Myrmecia croslandi)

Solitary foraging ants rely on vision when travelling along routes and when pinpointing their nest. We tethered foragers of Myrmecia croslandi on a trackball and recorded their intended movements when the trackball was located on their normal foraging corridor (on-route), above their nest and at a location several meters away where they have never been before (off-route). We find that at on- and off-route locations, most ants walk in the nest or foraging direction and continue to do so for tens of metres in a straight line. In contrast, above the nest, ants walk in random directions and change walking direction frequently. In addition, the walking direction of ants above the nest oscillates at a fine scale, reflecting search movements that are absent from the paths of ants at the other locations. An agent-based simulation shows that the behaviour of ants at all three locations can be explained by the integration of attractive and repellent views directed towards or away from the nest, respectively. Ants are likely to acquire such views via systematic scanning movements during their learning walks. The model predicts that ants placed in a completely unfamiliar environment should behave as if at the nest, which our subsequent experiments confirmed. We conclude first, that the ants’ behaviour at release sites is exclusively driven by what they currently see and not by information on expected outcomes of their behaviour. Second, that navigating ants might continuously integrate attractive and repellent visual memories. We discuss the benefits of such a procedure.

Dopamine neurons drive fear extinction learning by signaling the omission of expected aversive outcomes

Extinction of fear responses is critical for adaptive behavior and deficits in this form of safety learning are hallmark of anxiety disorders. However, the neuronal mechanisms that initiate extinction learning are largely unknown. Here we show, using single-unit electrophysiology and cell-type specific fiber photometry, that dopamine neurons in the ventral tegmental area (VTA) are activated by the omission of the aversive unconditioned stimulus (US) during fear extinction. This dopamine signal occurred specifically during the beginning of extinction when the US omission is unexpected, and correlated strongly with extinction learning. Furthermore, temporally-specific optogenetic inhibition or excitation of dopamine neurons at the time of the US omission revealed that this dopamine signal is both necessary for, and sufficient to accelerate, normal fear extinction learning. These results identify a prediction error-like neuronal signal that is necessary to initiate fear extinction and reveal a crucial role of DA neurons in this form of safety learning.

To survive, animals must identify and react to stimuli in their environment that signal danger. But they must also adapt their behavior when those stimuli no longer signal danger – hiding whenever you hear a loud noise might keep you safe, but it also prevents you from searching for food. In the laboratory, we can study this form of learning using procedures called fear conditioning and extinction.

During fear conditioning, animals learn that a stimulus, such as a tone, signals that an unpleasant event is about to occur. That event might involve receiving a mild shock to the foot, for example. After experiencing the tone and shock paired together multiple times, animals will initially show signs of fear – such as freezing – when they hear the tone. But if later the tone occurs without being followed by the shock, these fear responses fade. This fading process is called extinction. Extinction does not involve erasing the old memory about the tone-shock relationship. That is, it is not a form of forgetting. Instead, the animals learn that the tone no longer signals an impending shock.

By monitoring brain activity in mice trained to associate a shock with a tone, Salinas-Hernández et al. reveal how the brain begins to learn that the shock no longer follows the tone. When the mice do not receive the anticipated shock to the foot, a group of brain cells that produce the chemical dopamine increase their activity. These neurons also fire whenever animals receive a reward, particularly one that exceeds their expectations. The more the dopamine neurons fire, the faster the mice reduce their fear responses to the tone. Preventing the neurons from increasing their activity prevents the mice from extinguishing their fear memory. By contrast, activating the neurons speeds up the extinction process.

Understanding how the brain extinguishes learned fear responses has therapeutic implications. Many anxiety disorders, such as post-traumatic stress disorder, involve impaired fear extinction learning. Indeed, exposure therapy – used to treat anxiety disorders such as phobias – is a form of fear extinction. Manipulating the activity of dopamine neurons during extinction could therefore help to treat anxiety disorders.