Heterosynaptic Plasticity Underlies Aversive Olfactory Learning in Drosophila

Reinforcement learning when your life depends on it: A neuro-economic theory of learning

Synaptic plasticity enables animals to adapt to their environment, but memory formation can require a substantial amount of metabolic energy, potentially impairing survival. Hence, a neuro-economic dilemma arises whether learning is a profitable investment or not, and the brain must therefore judiciously regulate learning. Indeed, in experiments it was observed that during starvation, Drosophila suppress formation of energy-intensive aversive memories. Here we include energy considerations in a reinforcement learning framework. Simulated flies learned to avoid noxious stimuli through synaptic plasticity in either the energy expensive long-term memory (LTM) pathway, or the decaying anesthesia-resistant memory (ARM) pathway. The objective of the flies is to maximize their lifespan, which is calculated with a hazard function. We find that strategies that switch between the LTM and ARM pathways, based on energy reserve and reward prediction error, prolong lifespan. Our study highlights the significance of energy-regulation of memory pathways and dopaminergic control for adaptive learning and survival. It might also benefit engineering applications of reinforcement learning under resources constraints.

Dopamine-mediated interactions between short- and long-term memory dynamics

In dynamic environments, animals make behavioural decisions on the basis of the innate valences of sensory cues and information learnt about these cues across multiple timescales1-3. However, it remains unclear how the innate valence of a sensory stimulus affects the acquisition of learnt valence information and subsequent memory dynamics. Here we show that in the Drosophila brain, interconnected short- and long-term memory units of the mushroom body jointly regulate memory through dopamine signals that encode innate and learnt sensory valences. By performing time-lapse in vivo voltage-imaging studies of neural spiking in more than 500 flies undergoing olfactory associative conditioning, we found that protocerebral posterior lateral 1 dopamine neurons (PPL1-DANs)4 heterogeneously and bidirectionally encode innate and learnt valences of punishment, reward and odour cues. During learning, these valence signals regulate memory storage and extinction in mushroom body output neurons (MBONs)5. During initial conditioning bouts, PPL1-γ1pedc and PPL1-γ2α'1 neurons control short-term memory formation, which weakens inhibitory feedback from MBON-γ1pedc>α/β to PPL1-α'2α2 and PPL1-α3. During further conditioning, this diminished feedback allows these two PPL1-DANs to encode the net innate plus learnt valence of the conditioned odour cue, which gates long-term memory formation. A computational model constrained by the fly connectome6,7 and our spiking data explains how dopamine signals mediate the circuit interactions between short- and long-term memory traces, yielding predictions that our experiments confirmed. Overall, the mushroom body achieves flexible learning through the integration of innate and learnt valences in parallel learning units sharing feedback interconnections. This hybrid physiological-anatomical mechanism may be a general means by which dopamine regulates memory dynamics in other species and brain structures, including the vertebrate basal ganglia.

Bio-Plausible Multimodal Learning with Emerging Neuromorphic Devices

Multimodal machine learning, as a prospective advancement in artificial intelligence, endeavors to emulate the brain's multimodal learning abilities with the objective to enhance interactions with humans. However, this approach requires simultaneous processing of diverse types of data, leading to increased model complexity, longer training times, and higher energy consumption. Multimodal neuromorphic devices have the capability to preprocess spatio-temporal information from various physical signals into unified electrical signals with high information density, thereby enabling more biologically plausible multimodal learning with low complexity and high energy-efficiency. Here, this work conducts a comparison between the expression of multimodal machine learning and multimodal neuromorphic computing, followed by an overview of the key characteristics associated with multimodal neuromorphic devices. The bio-plausible operational principles and the multimodal learning abilities of emerging devices are examined, which are classified into heterogeneous and homogeneous multimodal neuromorphic devices. Subsequently, this work provides a detailed description of the multimodal learning capabilities demonstrated by neuromorphic circuits and their respective applications. Finally, this work highlights the limitations and challenges of multimodal neuromorphic computing in order to hopefully provide insight into potential future research directions.

Analysis of fast calcium dynamics of honey bee olfactory coding

Odour processing exhibits multiple parallels between vertebrate and invertebrate olfactory systems. Insects, in particular, have emerged as relevant models for olfactory studies because of the tractability of their olfactory circuits. Here, we used fast calcium imaging to track the activity of projection neurons in the honey bee antennal lobe (AL) during olfactory stimulation at high temporal resolution. We observed a heterogeneity of response profiles and an abundance of inhibitory activities, resulting in various response latencies and stimulus-specific post-odour neural signatures. Recorded calcium signals were fed to a mushroom body (MB) model constructed implementing the fundamental features of connectivity between olfactory projection neurons, Kenyon cells (KC), and MB output neurons (MBON). The model accounts for the increase of odorant discrimination in the MB compared to the AL and reveals the recruitment of two distinct KC populations that represent odorants and their aftersmell as two separate but temporally coherent neural objects. Finally, we showed that the learning-induced modulation of KC-to-MBON synapses can explain both the variations in associative learning scores across different conditioning protocols used in bees and the bees' response latency. Thus, it provides a simple explanation of how the time contingency between the stimulus and the reward can be encoded without the need for time tracking. This study broadens our understanding of olfactory coding and learning in honey bees. It demonstrates that a model based on simple MB connectivity rules and fed with real physiological data can explain fundamental aspects of odour processing and associative learning.

Compensatory enhancement of input maintains aversive dopaminergic reinforcement in hungry Drosophila

Hungry animals need compensatory mechanisms to maintain flexible brain function, while modulation reconfigures circuits to prioritize resource seeking. In Drosophila, hunger inhibits aversively reinforcing dopaminergic neurons (DANs) to permit the expression of food-seeking memories. Multitasking the reinforcement system for motivation potentially undermines aversive learning. We find that chronic hunger mildly enhances aversive learning and that satiated-baseline and hunger-enhanced learning require endocrine adipokinetic hormone (AKH) signaling. Circulating AKH influences aversive learning via its receptor in four neurons in the ventral brain, two of which are octopaminergic. Connectomics revealed AKH receptor-expressing neurons to be upstream of several classes of ascending neurons, many of which are presynaptic to aversively reinforcing DANs. Octopaminergic modulation of and output from at least one of these ascending pathways is required for shock- and bitter-taste-reinforced aversive learning. We propose that coordinated enhancement of input compensates for hunger-directed inhibition of aversive DANs to preserve reinforcement when required.

Diversity of visual inputs to Kenyon cells of the Drosophila mushroom body

The arthropod mushroom body is well-studied as an expansion layer representing olfactory stimuli and linking them to contingent events. However, 8% of mushroom body Kenyon cells in Drosophila melanogaster receive predominantly visual input, and their function remains unclear. Here, we identify inputs to visual Kenyon cells using the FlyWire adult whole-brain connectome. Input repertoires are similar across hemispheres and connectomes with certain inputs highly overrepresented. Many visual neurons presynaptic to Kenyon cells have large receptive fields, while interneuron inputs receive spatially restricted signals that may be tuned to specific visual features. Individual visual Kenyon cells randomly sample sparse inputs from combinations of visual channels, including multiple optic lobe neuropils. These connectivity patterns suggest that visual coding in the mushroom body, like olfactory coding, is sparse, distributed, and combinatorial. However, the specific input repertoire to the smaller population of visual Kenyon cells suggests a constrained encoding of visual stimuli.

Common principles for odour coding across vertebrates and invertebrates

The olfactory system is an ideal and tractable system for exploring how the brain transforms sensory inputs into behaviour. The basic tasks of any olfactory system include odour detection, discrimination and categorization. The challenge for the olfactory system is to transform the high-dimensional space of olfactory stimuli into the much smaller space of perceived objects and valence that endows odours with meaning. Our current understanding of how neural circuits address this challenge has come primarily from observations of the mechanisms of the brain for processing other sensory modalities, such as vision and hearing, in which optimized deep hierarchical circuits are used to extract sensory features that vary along continuous physical dimensions. The olfactory system, by contrast, contends with an ill-defined, high-dimensional stimulus space and discrete stimuli using a circuit architecture that is shallow and parallelized. Here, we present recent observations in vertebrate and invertebrate systems that relate the statistical structure and state-dependent modulation of olfactory codes to mechanisms of perception and odour-guided behaviour.

A lightweight data-driven spiking neuronal network model of Drosophila olfactory nervous system with dedicated hardware support

Data-driven spiking neuronal network (SNN) models enable in-silico analysis of the nervous system at the cellular and synaptic level. Therefore, they are a key tool for elucidating the information processing principles of the brain. While extensive research has focused on developing data-driven SNN models for mammalian brains, their complexity poses challenges in achieving precision. Network topology often relies on statistical inference, and the functions of specific brain regions and supporting neuronal activities remain unclear. Additionally, these models demand huge computing facilities and their simulation speed is considerably slower than real-time. Here, we propose a lightweight data-driven SNN model that strikes a balance between simplicity and reproducibility. The model is built using a qualitative modeling approach that can reproduce key dynamics of neuronal activity. We target the Drosophila olfactory nervous system, extracting its network topology from connectome data. The model was successfully implemented on a small entry-level field-programmable gate array and simulated the activity of a network in real-time. In addition, the model reproduced olfactory associative learning, the primary function of the olfactory system, and characteristic spiking activities of different neuron types. In sum, this paper propose a method for building data-driven SNN models from biological data. Our approach reproduces the function and neuronal activities of the nervous system and is lightweight, acceleratable with dedicated hardware, making it scalable to large-scale networks. Therefore, our approach is expected to play an important role in elucidating the brain's information processing at the cellular and synaptic level through an analysis-by-construction approach. In addition, it may be applicable to edge artificial intelligence systems in the future.

Functional Imaging of Learning-Induced Plasticity in the Central Nervous System with Genetically Encoded Reporters in Drosophila

Learning and memory allow animals to adjust their behavior based on the predictive value of their past experiences. Memories often exist in complex representations, spread across numerous cells and synapses in the brain. Studying relatively simple forms of memory provides insights into the fundamental processes that underlie multiple forms of memory. Associative learning occurs when an animal learns the relationship between two previously unrelated sensory stimuli, such as when a hungry animal learns that a particular odor is followed by a tasty reward. Drosophila is a particularly powerful model to study how this type of memory works. The fundamental principles are widely shared among animals, and there is a wide range of genetic tools available to study circuit function in flies. In addition, the olfactory structures that mediate associative learning in flies, such as the mushroom body and its associated neurons, are anatomically organized, relatively well-characterized, and readily accessible to imaging. Here, we review the olfactory anatomy and physiology of the olfactory system, describe how plasticity in the olfactory pathway mediates learning and memory, and explain the general principles underlying calcium imaging approaches.

Investigating visual navigation using spiking neural network models of the insect mushroom bodies

Ants are capable of learning long visually guided foraging routes with limited neural resources. The visual scene memory needed for this behaviour is mediated by the mushroom bodies; an insect brain region important for learning and memory. In a visual navigation context, the mushroom bodies are theorised to act as familiarity detectors, guiding ants to views that are similar to those previously learned when first travelling along a foraging route. Evidence from behavioural experiments, computational studies and brain lesions all support this idea. Here we further investigate the role of mushroom bodies in visual navigation with a spiking neural network model learning complex natural scenes. By implementing these networks in GeNN-a library for building GPU accelerated spiking neural networks-we were able to test these models offline on an image database representing navigation through a complex outdoor natural environment, and also online embodied on a robot. The mushroom body model successfully learnt a large series of visual scenes (400 scenes corresponding to a 27 m route) and used these memories to choose accurate heading directions during route recapitulation in both complex environments. Through analysing our model's Kenyon cell (KC) activity, we were able to demonstrate that KC activity is directly related to the respective novelty of input images. Through conducting a parameter search we found that there is a non-linear dependence between optimal KC to visual projection neuron (VPN) connection sparsity and the length of time the model is presented with an image stimulus. The parameter search also showed training the model on lower proportions of a route generally produced better accuracy when testing on the entire route. We embodied the mushroom body model and comparator visual navigation algorithms on a Quanser Q-car robot with all processing running on an Nvidia Jetson TX2. On a 6.5 m route, the mushroom body model had a mean distance to training route (error) of 0.144 ± 0.088 m over 5 trials, which was performance comparable to standard visual-only navigation algorithms. Thus, we have demonstrated that a biologically plausible model of the ant mushroom body can navigate complex environments both in simulation and the real world. Understanding the neural basis of this behaviour will provide insight into how neural circuits are tuned to rapidly learn behaviourally relevant information from complex environments and provide inspiration for creating bio-mimetic computer/robotic systems that can learn rapidly with low energy requirements.

Spaced training activates Miro/Milton-dependent mitochondrial dynamics in neuronal axons to sustain long-term memory

Neurons have differential and fluctuating energy needs across distinct cellular compartments, shaped by brain electrochemical activity associated with cognition. In vitro studies show that mitochondria transport from soma to axons is key to maintaining neuronal energy homeostasis. Nevertheless, whether the spatial distribution of neuronal mitochondria is dynamically adjusted in vivo in an experience-dependent manner remains unknown. In Drosophila, associative long-term memory (LTM) formation is initiated by an early and persistent upregulation of mitochondrial pyruvate flux in the axonal compartment of neurons in the mushroom body (MB). Through behavior experiments, super-resolution analysis of mitochondria morphology in the neuronal soma and in vivo mitochondrial fluorescence recovery after photobleaching (FRAP) measurements in the axons, we show that LTM induction, contrary to shorter-lived memories, is sustained by the departure of some mitochondria from MB neuronal soma and increased mitochondrial dynamics in the axonal compartment. Accordingly, impairing mitochondrial dynamics abolished the increased pyruvate consumption, specifically after spaced training and in the MB axonal compartment, thereby preventing LTM formation. Our results thus promote reorganization of the mitochondrial network in neurons as an integral step in elaborating high-order cognitive processes.

Diverse memory paradigms in Drosophila reveal diverse neural mechanisms

In this review, we aggregated the different types of learning and memory paradigms developed in adult Drosophila and attempted to assess the similarities and differences in the neural mechanisms supporting diverse types of memory. The simplest association memory assays are conditioning paradigms (olfactory, visual, and gustatory). A great deal of work has been done on these memories, revealing hundreds of genes and neural circuits supporting this memory. Variations of conditioning assays (reversal learning, trace conditioning, latent inhibition, and extinction) also reveal interesting memory mechanisms, whereas mechanisms supporting spatial memory (thermal maze, orientation memory, and heat box) and the conditioned suppression of innate behaviors (phototaxis, negative geotaxis, anemotaxis, and locomotion) remain largely unexplored. In recent years, there has been an increased interest in multisensory and multicomponent memories (context-dependent and cross-modal memory) and higher-order memory (sensory preconditioning and second-order conditioning). Some of this work has revealed how the intricate mushroom body (MB) neural circuitry can support more complex memories. Finally, the most complex memories are arguably those involving social memory: courtship conditioning and social learning (mate-copying and egg-laying behaviors). Currently, very little is known about the mechanisms supporting social memories. Overall, the MBs are important for association memories of multiple sensory modalities and multisensory integration, whereas the central complex is important for place, orientation, and navigation memories. Interestingly, several different types of memory appear to use similar or variants of the olfactory conditioning neural circuitry, which are repurposed in different ways.

Presynaptic regulators in memory formation

The intricate molecular and structural sequences guiding the formation and consolidation of memories within neuronal circuits remain largely elusive. In this study, we investigate the roles of two pivotal presynaptic regulators, the small GTPase Rab3, enriched at synaptic vesicles, and the cell adhesion protein Neurexin-1, in the formation of distinct memory phases within the Drosophila mushroom body Kenyon cells. Our findings suggest that both proteins play crucial roles in memory-supporting processes within the presynaptic terminal, operating within distinct plasticity modules. These modules likely encompass remodeling and maturation of existing active zones (AZs), as well as the formation of new AZs.

Skewing information flow through pre- and postsynaptic plasticity in the mushroom bodies of Drosophila

Animal brains need to store information to construct a representation of their environment. Knowledge of what happened in the past allows both vertebrates and invertebrates to predict future outcomes by recalling previous experience. Although invertebrate and vertebrate brains share common principles at the molecular, cellular, and circuit-architectural levels, there are also obvious differences as exemplified by the use of acetylcholine versus glutamate as the considered main excitatory neurotransmitters in the respective central nervous systems. Nonetheless, across central nervous systems, synaptic plasticity is thought to be a main substrate for memory storage. Therefore, how brain circuits and synaptic contacts change following learning is of fundamental interest for understanding brain computations tied to behavior in any animal. Recent progress has been made in understanding such plastic changes following olfactory associative learning in the mushroom bodies (MBs) of Drosophila A current framework of memory-guided behavioral selection is based on the MB skew model, in which antagonistic synaptic pathways are selectively changed in strength. Here, we review insights into plasticity at dedicated Drosophila MB output pathways and update what is known about the plasticity of both pre- and postsynaptic compartments of Drosophila MB neurons.

The caloric value of food intake structurally adjusts a neuronal mushroom body circuit mediating olfactory learning in Drosophila

Associative learning enables the adaptive adjustment of behavioral decisions based on acquired, predicted outcomes. The valence of what is learned is influenced not only by the learned stimuli and their temporal relations, but also by prior experiences and internal states. In this study, we used the fruit fly Drosophila melanogaster to demonstrate that neuronal circuits involved in associative olfactory learning undergo restructuring during extended periods of low-caloric food intake. Specifically, we observed a decrease in the connections between specific dopaminergic neurons (DANs) and Kenyon cells at distinct compartments of the mushroom body. This structural synaptic plasticity was contingent upon the presence of allatostatin A receptors in specific DANs and could be mimicked optogenetically by expressing a light-activated adenylate cyclase in exactly these DANs. Importantly, we found that this rearrangement in synaptic connections influenced aversive, punishment-induced olfactory learning but did not impact appetitive, reward-based learning. Whether induced by prolonged low-caloric conditions or optogenetic manipulation of cAMP levels, this synaptic rearrangement resulted in a reduction of aversive associative learning. Consequently, the balance between positive and negative reinforcing signals shifted, diminishing the ability to learn to avoid odor cues signaling negative outcomes. These results exemplify how a neuronal circuit required for learning and memory undergoes structural plasticity dependent on prior experiences of the nutritional value of food.

Modeling neurodegenerative and neurodevelopmental disorders in the Drosophila mushroom body

The common fruit fly Drosophila melanogaster provides a powerful platform to investigate the genetic, molecular, cellular, and neural circuit mechanisms of behavior. Research in this model system has shed light on multiple aspects of brain physiology and behavior, from fundamental neuronal function to complex behaviors. A major anatomical region that modulates complex behaviors is the mushroom body (MB). The MB integrates multimodal sensory information and is involved in behaviors ranging from sensory processing/responses to learning and memory. Many genes that underlie brain disorders are conserved, from flies to humans, and studies in Drosophila have contributed significantly to our understanding of the mechanisms of brain disorders. Genetic mutations that mimic human diseases-such as Fragile X syndrome, neurofibromatosis type 1, Parkinson's disease, and Alzheimer's disease-affect MB structure and function, altering behavior. Studies dissecting the effects of disease-causing mutations in the MB have identified key pathological mechanisms, and the development of a complete connectome promises to add a comprehensive anatomical framework for disease modeling. Here, we review Drosophila models of human neurodevelopmental and neurodegenerative disorders via the effects of their underlying mutations on MB structure, function, and the resulting behavioral alterations.

An integrative sensor of body states: how the mushroom body modulates behavior depending on physiological context

The brain constantly compares past and present experiences to predict the future, thereby enabling instantaneous and future behavioral adjustments. Integration of external information with the animal's current internal needs and behavioral state represents a key challenge of the nervous system. Recent advancements in dissecting the function of the Drosophila mushroom body (MB) at the single-cell level have uncovered its three-layered logic and parallel systems conveying positive and negative values during associative learning. This review explores a lesser-known role of the MB in detecting and integrating body states such as hunger, thirst, and sleep, ultimately modulating motivation and sensory-driven decisions based on the physiological state of the fly. State-dependent signals predominantly affect the activity of modulatory MB input neurons (dopaminergic, serotoninergic, and octopaminergic), but also induce plastic changes directly at the level of the MB intrinsic and output neurons. Thus, the MB emerges as a tightly regulated relay station in the insect brain, orchestrating neuroadaptations due to current internal and behavioral states leading to short- but also long-lasting changes in behavior. While these adaptations are crucial to ensure fitness and survival, recent findings also underscore how circuit motifs in the MB may reflect fundamental design principles that contribute to maladaptive behaviors such as addiction or depression-like symptoms.

An octopamine-specific GRAB sensor reveals a monoamine relay circuitry that boosts aversive learning

Octopamine (OA), analogous to norepinephrine in vertebrates, is an essential monoamine neurotransmitter in invertebrates that plays a significant role in various biological functions, including olfactory associative learning. However, the spatial and temporal dynamics of OA in vivo remain poorly understood due to limitations associated with the currently available methods used to detect it. To overcome these limitations, we developed a genetically encoded GPCR  activation-based (GRAB) OA sensor called GRABOA1.0. This sensor is highly selective for OA and exhibits a robust and rapid increase in fluorescence in response to extracellular OA. Using GRABOA1.0, we monitored OA release in the Drosophila mushroom body (MB), the fly's learning center, and found that OA is released in response to both odor and shock stimuli in an aversive learning model. This OA release requires acetylcholine (ACh) released from Kenyon cells, signaling via nicotinic ACh receptors. Finally, we discovered that OA amplifies aversive learning behavior by augmenting dopamine-mediated punishment signals via Octβ1R in dopaminergic neurons, leading to alterations in synaptic plasticity within the MB. Thus, our new GRABOA1.0 sensor can be used to monitor OA release in real time under physiological conditions, providing valuable insights into the cellular and circuit mechanisms that underlie OA signaling.

Beyond prediction error: 25 years of modeling the associations formed in the insect mushroom body

The insect mushroom body has gained increasing attention as a system in which the computational basis of neural learning circuits can be unraveled. We now understand in detail the key locations in this circuit where synaptic associations are formed between sensory patterns and values leading to actions. However, the actual learning rule (or rules) implemented by neural activity and leading to synaptic change is still an open question. Here, I survey the diversity of answers that have been offered in computational models of this system over the past decades, including the recurring assumption-in line with top-down theories of associative learning-that the core function is to reduce prediction error. However, I will argue, a more bottom-up approach may ultimately reveal a richer algorithmic capacity in this still enigmatic brain neuropil.

Sensory encoding and memory in the mushroom body: signals, noise, and variability

To survive in changing environments, animals need to learn to associate specific sensory stimuli with positive or negative valence. How do they form stimulus-specific memories to distinguish between positively/negatively associated stimuli and other irrelevant stimuli? Solving this task is one of the functions of the mushroom body, the associative memory center in insect brains. Here we summarize recent work on sensory encoding and memory in the Drosophila mushroom body, highlighting general principles such as pattern separation, sparse coding, noise and variability, coincidence detection, and spatially localized neuromodulation, and placing the mushroom body in comparative perspective with mammalian memory systems.

Cyclic nucleotide-induced bidirectional long-term synaptic plasticity in Drosophila mushroom body

Activation of the cAMP pathway is one of the common mechanisms underlying long-term potentiation (LTP). In the Drosophila mushroom body, simultaneous activation of odour-coding Kenyon cells (KCs) and reinforcement-coding dopaminergic neurons activates adenylyl cyclase in KC presynaptic terminals, which is believed to trigger synaptic plasticity underlying olfactory associative learning. However, learning induces long-term depression (LTD) at these synapses, contradicting the universal role of cAMP as a facilitator of transmission. Here, we developed a system to electrophysiologically monitor both short-term and long-term synaptic plasticity at KC output synapses and demonstrated that they are indeed an exception in which activation of the cAMP-protein kinase A pathway induces LTD. Contrary to the prevailing model, our cAMP imaging found no evidence for synergistic action of dopamine and KC activity on cAMP synthesis. Furthermore, we found that forskolin-induced cAMP increase alone was insufficient for plasticity induction; it additionally required simultaneous KC activation to replicate the presynaptic LTD induced by pairing with dopamine. On the other hand, activation of the cGMP pathway paired with KC activation induced slowly developing LTP, proving antagonistic actions of the two second-messenger pathways predicted by behavioural study. Finally, KC subtype-specific interrogation of synapses revealed that different KC subtypes exhibit distinct plasticity duration even among synapses on the same postsynaptic neuron. Thus, our work not only revises the role of cAMP in synaptic plasticity by uncovering the unexpected convergence point of the cAMP pathway and neuronal activity, but also establishes the methods to address physiological mechanisms of synaptic plasticity in this important model. KEY POINTS: Although presynaptic cAMP increase generally facilitates synapses, olfactory associative learning in Drosophila, which depends on dopamine and cAMP signalling genes, induces long-term depression (LTD) at the mushroom body output synapses. By combining electrophysiology, pharmacology and optogenetics, we directly demonstrate that these synapses are an exception where activation of the cAMP-protein kinase A pathway leads to presynaptic LTD. Dopamine- or forskolin-induced cAMP increase alone is not sufficient for LTD induction; neuronal activity, which has been believed to trigger cAMP synthesis in synergy with dopamine input, is required in the downstream pathway of cAMP. In contrast to cAMP, activation of the cGMP pathway paired with neuronal activity induces presynaptic long-term potentiation, which explains behaviourally observed opposing actions of transmitters co-released by dopaminergic neurons. Our work not only revises the role of cAMP in synaptic plasticity, but also provides essential methods to address physiological mechanisms of synaptic plasticity in this important model system.

Roles of feedback and feed-forward networks of dopamine subsystems: insights from Drosophila studies

Across animal species, dopamine-operated memory systems comprise anatomically segregated, functionally diverse subsystems. Although individual subsystems could operate independently to support distinct types of memory, the logical interplay between subsystems is expected to enable more complex memory processing by allowing existing memory to influence future learning. Recent comprehensive ultrastructural analysis of the Drosophila mushroom body revealed intricate networks interconnecting the dopamine subsystems-the mushroom body compartments. Here, we review the functions of some of these connections that are beginning to be understood. Memory consolidation is mediated by two different forms of network: A recurrent feedback loop within a compartment maintains sustained dopamine activity required for consolidation, whereas feed-forward connections across compartments allow short-term memory formation in one compartment to open the gate for long-term memory formation in another compartment. Extinction and reversal of aversive memory rely on a similar feed-forward circuit motif that signals omission of punishment as a reward, which triggers plasticity that counteracts the original aversive memory trace. Finally, indirect feed-forward connections from a long-term memory compartment to short-term memory compartments mediate higher-order conditioning. Collectively, these emerging studies indicate that feedback control and hierarchical connectivity allow the dopamine subsystems to work cooperatively to support diverse and complex forms of learning.

Use-Dependent, Untapped Dual Kinase Signaling Localized in Brain Learning Circuitry

Imaging brain learning and memory circuit kinase signaling is a monumental challenge. The separation of phases-based activity reporter of kinase (SPARK) biosensors allow circuit-localized studies of multiple interactive kinases in vivo, including protein kinase A (PKA) and extracellular signal-regulated kinase (ERK) signaling. In the precisely-mapped Drosophila brain learning/memory circuit, we find PKA and ERK signaling differentially enriched in distinct Kenyon cell connectivity nodes. We discover that potentiating normal circuit activity induces circuit-localized PKA and ERK signaling, expanding kinase function within new presynaptic and postsynaptic domains. Activity-induced PKA signaling shows extensive overlap with previously selective ERK signaling nodes, while activity-induced ERK signaling arises in new connectivity nodes. We find targeted synaptic transmission blockade in Kenyon cells elevates circuit-localized ERK induction in Kenyon cells with normally high baseline ERK signaling, suggesting lateral and feedback inhibition. We discover overexpression of the pathway-linking Meng-Po (human SBK1) serine/threonine kinase to improve learning acquisition and memory consolidation results in dramatically heightened PKA and ERK signaling in separable Kenyon cell circuit connectivity nodes, revealing both synchronized and untapped signaling potential. Finally, we find that a mechanically-induced epileptic seizure model (easily shocked "bang-sensitive" mutants) has strongly elevated, circuit-localized PKA and ERK signaling. Both sexes were used in all experiments, except for the hemizygous male-only seizure model. Hyperexcitable, learning-enhanced, and epileptic seizure models have comparably elevated interactive kinase signaling, suggesting a common basis of use-dependent induction. We conclude that PKA and ERK signaling modulation is locally coordinated in use-dependent spatial circuit dynamics underlying seizure susceptibility linked to learning/memory potential.

An octopamine-specific GRAB sensor reveals a monoamine relay circuitry that boosts aversive learning

Octopamine (OA), analogous to norepinephrine in vertebrates, is an essential monoamine neurotransmitter in invertebrates that plays a significant role in various biological functions, including olfactory associative learning. However, the spatial and temporal dynamics of OA in vivo remain poorly understood due to limitations associated with the currently available methods used to detect it. To overcome these limitations, we developed a genetically encoded GPCR activation-based (GRAB) OA sensor called GRABOA1.0. This sensor is highly selective for OA and exhibits a robust and rapid increase in fluorescence in response to extracellular OA. Using GRABOA1.0, we monitored OA release in the Drosophila mushroom body (MB), the fly's learning center, and found that OA is released in response to both odor and shock stimuli in an aversive learning model. This OA release requires acetylcholine (ACh) released from Kenyon cells, signaling via nicotinic ACh receptors. Finally, we discovered that OA amplifies aversive learning behavior by augmenting dopamine-mediated punishment signals via Octβ1R in dopaminergic neurons, leading to alterations in synaptic plasticity within the MB. Thus, our new GRABOA1.0 sensor can be used to monitor OA release in real-time under physiological conditions, providing valuable insights into the cellular and circuit mechanisms that underlie OA signaling.

Asymmetric neurons are necessary for olfactory learning in the Drosophila brain

Animals have complementary parallel memory systems that process signals from various sensory modalities. In the brain of the fruit fly Drosophila melanogaster, mushroom body (MB) circuitry is the primary associative neuropil, critical for all stages of olfactory memory. Here, our findings suggest that active signaling from specific asymmetric body (AB) neurons is also crucial for this process. These AB neurons respond to odors and electric shock separately and exhibit timing-sensitive neuronal activity in response to paired stimulation while leaving a decreased memory trace during retrieval. Our experiments also show that rutabaga-encoded adenylate cyclase, which mediates coincidence detection, is required for learning and short-term memory in both AB and MB. We observed additive effects when manipulating rutabaga co-expression in both structures. Together, these results implicate the AB in playing a critical role in associative olfactory learning and short-term memory.

Prediction error drives associative learning and conditioned behavior in a spiking model of Drosophila larva

Predicting reinforcement from sensory cues is beneficial for goal-directed behavior. In insect brains, underlying associations between cues and reinforcement, encoded by dopaminergic neurons, are formed in the mushroom body. We propose a spiking model of the Drosophila larva mushroom body. It includes a feedback motif conveying learned reinforcement expectation to dopaminergic neurons, which can compute prediction error as the difference between expected and present reinforcement. We demonstrate that this can serve as a driving force in learning. When combined with synaptic homeostasis, our model accounts for theoretically derived features of acquisition and loss of associations that depend on the intensity of the reinforcement and its temporal proximity to the cue. From modeling olfactory learning over the time course of behavioral experiments and simulating the locomotion of individual larvae toward or away from odor sources in a virtual environment, we conclude that learning driven by prediction errors can explain larval behavior.

Minimal circuit motifs for second-order conditioning in the insect mushroom body

In well-established first-order conditioning experiments, the concurrence of a sensory cue with reinforcement forms an association, allowing the cue to predict future reinforcement. In the insect mushroom body, a brain region central to learning and memory, such associations are encoded in the synapses between its intrinsic and output neurons. This process is mediated by the activity of dopaminergic neurons that encode reinforcement signals. In second-order conditioning, a new sensory cue is paired with an already established one that presumably activates dopaminergic neurons due to its predictive power of the reinforcement. We explored minimal circuit motifs in the mushroom body for their ability to support second-order conditioning using mechanistic models. We found that dopaminergic neurons can either be activated directly by the mushroom body's intrinsic neurons or via feedback from the output neurons via several pathways. We demonstrated that the circuit motifs differ in their computational efficiency and robustness. Beyond previous research, we suggest an additional motif that relies on feedforward input of the mushroom body intrinsic neurons to dopaminergic neurons as a promising candidate for experimental evaluation. It differentiates well between trained and novel stimuli, demonstrating robust performance across a range of model parameters.

Cyclic nucleotide-induced bidirectional long-term synaptic plasticity in Drosophila mushroom body

Activation of the cAMP pathway is one of the common mechanisms underlying long-term potentiation (LTP). In the Drosophila mushroom body, simultaneous activation of odor-coding Kenyon cells (KCs) and reinforcement-coding dopaminergic neurons activates adenylyl cyclase in KC presynaptic terminals, which is believed to trigger synaptic plasticity underlying olfactory associative learning. However, learning induces long-term depression (LTD) at these synapses, contradicting the universal role of cAMP as a facilitator of transmission. Here, we develop a system to electrophysiologically monitor both short-term and long-term synaptic plasticity at KC output synapses and demonstrate that they are indeed an exception where activation of the cAMP/protein kinase A pathway induces LTD. Contrary to the prevailing model, our cAMP imaging finds no evidence for synergistic action of dopamine and KC activity on cAMP synthesis. Furthermore, we find that forskolin-induced cAMP increase alone is insufficient for plasticity induction; it additionally requires simultaneous KC activation to replicate the presynaptic LTD induced by pairing with dopamine. On the other hand, activation of the cGMP pathway paired with KC activation induces slowly developing LTP, proving antagonistic actions of the two second-messenger pathways predicted by behavioral study. Finally, KC subtype-specific interrogation of synapses reveals that different KC subtypes exhibit distinct plasticity duration even among synapses on the same postsynaptic neuron. Thus, our work not only revises the role of cAMP in synaptic plasticity by uncovering the unexpected convergence point of the cAMP pathway and neuronal activity, but also establishes the methods to address physiological mechanisms of synaptic plasticity in this important model.

Combinatory Actions of Co-transmitters in Dopaminergic Systems Modulate Drosophila Olfactory Memories

Dopamine neurons (DANs) are extensively studied in the context of associative learning, in both vertebrates and invertebrates. In the acquisition of male and female Drosophila olfactory memory, the PAM cluster of DANs provides the reward signal, and the PPL1 cluster of DANs sends the punishment signal to the Kenyon cells (KCs) of mushroom bodies, the center for memory formation. However, thermo-genetical activation of the PPL1 DANs after memory acquisition impaired aversive memory, and that of the PAM DANs impaired appetitive memory. We demonstrate that the knockdown of glutamate decarboxylase, which catalyzes glutamate conversion to GABA in PAM DANs, potentiated the appetitive memory. In addition, the knockdown of glutamate transporter in PPL1 DANs potentiated aversive memory, suggesting that GABA and glutamate co-transmitters act in an inhibitory manner in olfactory memory formation. We also found that, in γKCs, the Rdl receptor for GABA and the mGluR DmGluRA mediate the inhibition. Although multiple-spaced training is required to form long-term aversive memory, a single cycle of training was sufficient to develop long-term memory when the glutamate transporter was knocked down, in even a single subset of PPL1 DANs. Our results suggest that the mGluR signaling pathway may set a threshold for memory acquisition to allow the organisms' behaviors to adapt to changing physiological conditions and environments.SIGNIFICANCE STATEMENT In the acquisition of olfactory memory in Drosophila, the PAM cluster of dopamine neurons (DANs) mediates the reward signal, while the PPL1 cluster of DANs conveys the punishment signal to the Kenyon cells of the mushroom bodies, which serve as the center for memory formation. We found that GABA co-transmitters in the PAM DANs and glutamate co-transmitters in the PPL1 DANs inhibit olfactory memory formation. Our findings demonstrate that long-term memory acquisition, which typically necessitates multiple-spaced training sessions to establish aversive memory, can be triggered with a single training cycle in cases where the glutamate co-transmission is inhibited, even within a single subset of PPL1 DANs, suggesting that the glutamate co-transmission may modulate the threshold for memory acquisition.

Dopaminergic neurons dynamically update sensory values during olfactory maneuver

Dopaminergic neurons (DANs) drive associative learning to update the value of sensory cues, but their contribution to the assessment of sensory values outside the context of association remains largely unexplored. Here, we show in Drosophila that DANs in the mushroom body encode the innate value of odors and constantly update the current value by inducing plasticity during olfactory maneuver. Our connectome-based network model linking all the way from the olfactory neurons to DANs reproduces the characteristics of DAN responses, proposing a concrete circuit mechanism for computation. Downstream of DANs, odors alone induce value- and dopamine-dependent changes in the activity of mushroom body output neurons, which store the current value of odors. Consistent with this neural plasticity, specific sets of DANs bidirectionally modulate flies' steering in a virtual olfactory environment. Thus, the DAN circuit known for discrete, associative learning also continuously updates odor values in a nonassociative manner.

Diversity of visual inputs to Kenyon cells of the Drosophila mushroom body

The arthropod mushroom body is well-studied as an expansion layer that represents olfactory stimuli and links them to contingent events. However, 8% of mushroom body Kenyon cells in Drosophila melanogaster receive predominantly visual input, and their tuning and function are poorly understood. Here, we use the FlyWire adult whole-brain connectome to identify inputs to visual Kenyon cells. The types of visual neurons we identify are similar across hemispheres and connectomes with certain inputs highly overrepresented. Many visual projection neurons presynaptic to Kenyon cells receive input from large swathes of visual space, while local visual interneurons, providing smaller fractions of input, receive more spatially restricted signals that may be tuned to specific features of the visual scene. Like olfactory Kenyon cells, visual Kenyon cells receive sparse inputs from different combinations of visual channels, including inputs from multiple optic lobe neuropils. The sets of inputs to individual visual Kenyon cells are consistent with random sampling of available inputs. These connectivity patterns suggest that visual coding in the mushroom body, like olfactory coding, is sparse, distributed, and combinatorial. However, the expansion coding properties appear different, with a specific repertoire of visual inputs projecting onto a relatively small number of visual Kenyon cells.

Reducing Catastrophic Forgetting With Associative Learning: A Lesson From Fruit Flies

Catastrophic forgetting remains an outstanding challenge in continual learning. Recently, methods inspired by the brain, such as continual representation learning and memory replay, have been used to combat catastrophic forgetting. Associative learning (retaining associations between inputs and outputs, even after good representations are learned) plays an important function in the brain; however, its role in continual learning has not been carefully studied. Here, we identified a two-layer neural circuit in the fruit fly olfactory system that performs continual associative learning between odors and their associated valences. In the first layer, inputs (odors) are encoded using sparse, high-dimensional representations, which reduces memory interference by activating nonoverlapping populations of neurons for different odors. In the second layer, only the synapses between odor-activated neurons and the odor's associated output neuron are modified during learning; the rest of the weights are frozen to prevent unrelated memories from being overwritten. We prove theoretically that these two perceptron-like layers help reduce catastrophic forgetting compared to the original perceptron algorithm, under continual learning. We then show empirically on benchmark data sets that this simple and lightweight architecture outperforms other popular neural-inspired algorithms when also using a two-layer feedforward architecture. Overall, fruit flies evolved an efficient continual associative learning algorithm, and circuit mechanisms from neuroscience can be translated to improve machine computation.

Effects of stochastic coding on olfactory discrimination in flies and mice

Sparse coding can improve discrimination of sensory stimuli by reducing overlap between their representations. Two factors, however, can offset sparse coding's benefits: similar sensory stimuli have significant overlap and responses vary across trials. To elucidate the effects of these 2 factors, we analyzed odor responses in the fly and mouse olfactory regions implicated in learning and discrimination-the mushroom body (MB) and the piriform cortex (PCx). We found that neuronal responses fall along a continuum from extremely reliable across trials to extremely variable or stochastic. Computationally, we show that the observed variability arises from noise within central circuits rather than sensory noise. We propose this coding scheme to be advantageous for coarse- and fine-odor discrimination. More reliable cells enable quick discrimination between dissimilar odors. For similar odors, however, these cells overlap and do not provide distinguishing information. By contrast, more unreliable cells are decorrelated for similar odors, providing distinguishing information, though these benefits only accrue with extended training with more trials. Overall, we have uncovered a conserved, stochastic coding scheme in vertebrates and invertebrates, and we identify a candidate mechanism, based on variability in a winner-take-all (WTA) inhibitory circuit, that improves discrimination with training.

Dopamine-Dependent Plasticity Is Heterogeneously Expressed by Presynaptic Calcium Activity across Individual Boutons of the Drosophila Mushroom Body

The Drosophila mushroom body (MB) is an important model system for studying the synaptic mechanisms of associative learning. In this system, coincidence of odor-evoked calcium influx and dopaminergic input in the presynaptic terminals of Kenyon cells (KCs), the principal neurons of the MB, triggers long-term depression (LTD), which plays a critical role in olfactory learning. However, it is controversial whether such synaptic plasticity is accompanied by a corresponding decrease in odor-evoked calcium activity in the KC presynaptic terminals. Here, we address this question by inducing LTD by pairing odor presentation with optogenetic activation of dopaminergic neurons (DANs). This allows us to rigorously compare the changes at the presynaptic and postsynaptic sites in the same conditions. By imaging presynaptic acetylcholine release in the condition where LTD is reliably observed in the postsynaptic calcium signals, we show that neurotransmitter release from KCs is depressed selectively in the MB compartments innervated by activated DANs, demonstrating the presynaptic nature of LTD. However, total odor-evoked calcium activity of the KC axon bundles does not show concurrent depression. We further conduct calcium imaging in individual presynaptic boutons and uncover the highly heterogeneous nature of calcium plasticity. Namely, only a subset of boutons, which are strongly activated by associated odors, undergo calcium activity depression, while weakly responding boutons show potentiation. Thus, our results suggest an unexpected nonlinear relationship between presynaptic calcium influx and the results of plasticity, challenging the simple view of cooperative actions of presynaptic calcium and dopaminergic input.

Independent insulin signaling modulators govern hot avoidance under different feeding states

Thermosensation is critical for the survival of animals. However, mechanisms through which nutritional status modulates thermosensation remain unclear. Herein, we showed that hungry Drosophila exhibit a strong hot avoidance behavior (HAB) compared to food-sated flies. We identified that hot stimulus increases the activity of α'β' mushroom body neurons (MBns), with weak activity in the sated state and strong activity in the hungry state. Furthermore, we showed that α'β' MBn receives the same level of hot input from the mALT projection neurons via cholinergic transmission in sated and hungry states. Differences in α'β' MBn activity between food-sated and hungry flies following heat stimuli are regulated by distinct Drosophila insulin-like peptides (Dilps). Dilp2 is secreted by insulin-producing cells (IPCs) and regulates HAB during satiety, whereas Dilp6 is secreted by the fat body and regulates HAB during the hungry state. We observed that Dilp2 induces PI3K/AKT signaling, whereas Dilp6 induces Ras/ERK signaling in α'β' MBn to regulate HAB in different feeding conditions. Finally, we showed that the 2 α'β'-related MB output neurons (MBONs), MBON-α'3 and MBON-β'1, are necessary for the output of integrated hot avoidance information from α'β' MBn. Our results demonstrate the presence of dual insulin modulation pathways in α'β' MBn, which are important for suitable behavioral responses in Drosophila during thermoregulation under different feeding states.

An antagonism between Spinophilin and Syd-1 operates upstream of memory-promoting presynaptic long-term plasticity

We still face fundamental gaps in understanding how molecular plastic changes of synapses intersect with circuit operation to define behavioral states. Here, we show that an antagonism between two conserved regulatory proteins, Spinophilin (Spn) and Syd-1, controls presynaptic long-term plasticity and the maintenance of olfactory memories in Drosophila. While Spn mutants could not trigger nanoscopic active zone remodeling under homeostatic challenge and failed to stably potentiate neurotransmitter release, concomitant reduction of Syd-1 rescued all these deficits. The Spn/Syd-1 antagonism converged on active zone close F-actin, and genetic or acute pharmacological depolymerization of F-actin rescued the Spn deficits by allowing access to synaptic vesicle release sites. Within the intrinsic mushroom body neurons, the Spn/Syd-1 antagonism specifically controlled olfactory memory stabilization but not initial learning. Thus, this evolutionarily conserved protein complex controls behaviorally relevant presynaptic long-term plasticity, also observed in the mammalian brain but still enigmatic concerning its molecular mechanisms and behavioral relevance.

Optogenetic induction of appetitive and aversive taste memories in Drosophila

Tastes typically evoke innate behavioral responses that can be broadly categorized as acceptance or rejection. However, research in Drosophila melanogaster indicates that taste responses also exhibit plasticity through experience-dependent changes in mushroom body circuits. In this study, we develop a novel taste learning paradigm using closed-loop optogenetics. We find that appetitive and aversive taste memories can be formed by pairing gustatory stimuli with optogenetic activation of sensory neurons or dopaminergic neurons encoding reward or punishment. As with olfactory memories, distinct dopaminergic subpopulations drive the parallel formation of short- and long-term appetitive memories. Long-term memories are protein synthesis-dependent and have energetic requirements that are satisfied by a variety of caloric food sources or by direct stimulation of MB-MP1 dopaminergic neurons. Our paradigm affords new opportunities to probe plasticity mechanisms within the taste system and understand the extent to which taste responses depend on experience.

Reward expectations direct learning and drive operant matching in Drosophila

Foraging animals must use decision-making strategies that dynamically adapt to the changing availability of rewards in the environment. A wide diversity of animals do this by distributing their choices in proportion to the rewards received from each option, Herrnstein's operant matching law. Theoretical work suggests an elegant mechanistic explanation for this ubiquitous behavior, as operant matching follows automatically from simple synaptic plasticity rules acting within behaviorally relevant neural circuits. However, no past work has mapped operant matching onto plasticity mechanisms in the brain, leaving the biological relevance of the theory unclear. Here, we discovered operant matching in Drosophila and showed that it requires synaptic plasticity that acts in the mushroom body and incorporates the expectation of reward. We began by developing a dynamic foraging paradigm to measure choices from individual flies as they learn to associate odor cues with probabilistic rewards. We then built a model of the fly mushroom body to explain each fly's sequential choice behavior using a family of biologically realistic synaptic plasticity rules. As predicted by past theoretical work, we found that synaptic plasticity rules could explain fly matching behavior by incorporating stimulus expectations, reward expectations, or both. However, by optogenetically bypassing the representation of reward expectation, we abolished matching behavior and showed that the plasticity rule must specifically incorporate reward expectations. Altogether, these results reveal the first synapse-level mechanisms of operant matching and provide compelling evidence for the role of reward expectation signals in the fly brain.

Neural circuit mechanisms for transforming learned olfactory valences into wind-oriented movement

How memories are used by the brain to guide future action is poorly understood. In olfactory associative learning in Drosophila, multiple compartments of the mushroom body act in parallel to assign a valence to a stimulus. Here, we show that appetitive memories stored in different compartments induce different levels of upwind locomotion. Using a photoactivation screen of a new collection of split-GAL4 drivers and EM connectomics, we identified a cluster of neurons postsynaptic to the mushroom body output neurons (MBONs) that can trigger robust upwind steering. These UpWind Neurons (UpWiNs) integrate inhibitory and excitatory synaptic inputs from MBONs of appetitive and aversive memory compartments, respectively. After formation of appetitive memory, UpWiNs acquire enhanced response to reward-predicting odors as the response of the inhibitory presynaptic MBON undergoes depression. Blocking UpWiNs impaired appetitive memory and reduced upwind locomotion during retrieval. Photoactivation of UpWiNs also increased the chance of returning to a location where activation was terminated, suggesting an additional role in olfactory navigation. Thus, our results provide insight into how learned abstract valences are gradually transformed into concrete memory-driven actions through divergent and convergent networks, a neuronal architecture that is commonly found in the vertebrate and invertebrate brains.

Differential second messenger signaling via dopamine neurons bidirectionally regulates memory retention

Memory formation and forgetting unnecessary memory must be balanced for adaptive animal behavior. While cyclic AMP (cAMP) signaling via dopamine neurons induces memory formation, here we report that cyclic guanine monophosphate (cGMP) signaling via dopamine neurons launches forgetting of unconsolidated memory in Drosophila. Genetic screening and proteomic analyses showed that neural activation induces the complex formation of a histone H3K9 demethylase, Kdm4B, and a GMP synthetase, Bur, which is necessary and sufficient for forgetting unconsolidated memory. Kdm4B/Bur is activated by phosphorylation through NO-dependent cGMP signaling via dopamine neurons, inducing gene expression, including kek2 encoding a presynaptic protein. Accordingly, Kdm4B/Bur activation induced presynaptic changes. Our data demonstrate a link between cGMP signaling and synapses via gene expression in forgetting, suggesting that the opposing functions of memory are orchestrated by distinct signaling via dopamine neurons, which affects synaptic integrity and thus balances animal behavior.

Learning and memory using Drosophila melanogaster: a focus on advances made in the fifth decade of research

In the last decade, researchers using Drosophila melanogaster have made extraordinary progress in uncovering the mysteries underlying learning and memory. This progress has been propelled by the amazing toolkit available that affords combined behavioral, molecular, electrophysiological, and systems neuroscience approaches. The arduous reconstruction of electron microscopic images resulted in a first-generation connectome of the adult and larval brain, revealing complex structural interconnections between memory-related neurons. This serves as substrate for future investigations on these connections and for building complete circuits from sensory cue detection to changes in motor behavior. Mushroom body output neurons (MBOn) were discovered, which individually forward information from discrete and non-overlapping compartments of the axons of mushroom body neurons (MBn). These neurons mirror the previously discovered tiling of mushroom body axons by inputs from dopamine neurons and have led to a model that ascribes the valence of the learning event, either appetitive or aversive, to the activity of different populations of dopamine neurons and the balance of MBOn activity in promoting avoidance or approach behavior. Studies of the calyx, which houses the MBn dendrites, have revealed a beautiful microglomeruluar organization and structural changes of synapses that occur with long-term memory (LTM) formation. Larval learning has advanced, positioning it to possibly lead in producing new conceptual insights due to its markedly simpler structure over the adult brain. Advances were made in how cAMP response element-binding protein interacts with protein kinases and other transcription factors to promote the formation of LTM. New insights were made on Orb2, a prion-like protein that forms oligomers to enhance synaptic protein synthesis required for LTM formation. Finally, Drosophila research has pioneered our understanding of the mechanisms that mediate permanent and transient active forgetting, an important function of the brain along with acquisition, consolidation, and retrieval. This was catalyzed partly by the identification of memory suppressor genes-genes whose normal function is to limit memory formation.

Olfactory navigation in arthropods

Using odors to find food and mates is one of the most ancient and highly conserved behaviors. Arthropods from flies to moths to crabs use broadly similar strategies to navigate toward odor sources-such as integrating flow information with odor information, comparing odor concentration across sensors, and integrating odor information over time. Because arthropods share many homologous brain structures-antennal lobes for processing olfactory information, mechanosensors for processing flow, mushroom bodies (or hemi-ellipsoid bodies) for associative learning, and central complexes for navigation, it is likely that these closely related behaviors are mediated by conserved neural circuits. However, differences in the types of odors they seek, the physics of odor dispersal, and the physics of locomotion in water, air, and on substrates mean that these circuits must have adapted to generate a wide diversity of odor-seeking behaviors. In this review, we discuss common strategies and specializations observed in olfactory navigation behavior across arthropods, and review our current knowledge about the neural circuits subserving this behavior. We propose that a comparative study of arthropod nervous systems may provide insight into how a set of basic circuit structures has diversified to generate behavior adapted to different environments.

Neural circuit mechanisms linking courtship and reward in Drosophila males

Courtship has evolved to achieve reproductive success in animal species. However, whether courtship itself has a positive value remains unclear. In the present work, we report that courtship is innately rewarding and can induce the expression of appetitive short-term memory (STM) and long-term memory (LTM) in Drosophila melanogaster males. Activation of male-specific P1 neurons is sufficient to mimic courtship-induced preference and memory performance. Surprisingly, P1 neurons functionally connect to a large proportion of dopaminergic neurons (DANs) in the protocerebral anterior medial (PAM) cluster. The acquisition of STM and LTM depends on two distinct subsets of PAM DANs that convey the courtship-reward signal to the restricted regions of the mushroom body (MB) γ and α/β lobes through two dopamine receptors, D1-like Dop1R1 and D2-like Dop2R. Furthermore, the retrieval of STM stored in the MB α'/β' lobes and LTM stored in the MB α/β lobe relies on two distinct MB output neurons. Finally, LTM consolidation requires two subsets of PAM DANs projecting to the MB α/β lobe and corresponding MB output neurons. Taken together, our findings demonstrate that courtship is a potent rewarding stimulus and reveal the underlying neural circuit mechanisms linking courtship and reward in Drosophila males.

Multisensory learning binds neurons into a cross-modal memory engram

Associating multiple sensory cues with objects and experience is a fundamental brain process that improves object recognition and memory performance. However, neural mechanisms that bind sensory features during learning and augment memory expression are unknown. Here we demonstrate multisensory appetitive and aversive memory in Drosophila. Combining colours and odours improved memory performance, even when each sensory modality was tested alone. Temporal control of neuronal function revealed visually selective mushroom body Kenyon cells (KCs) to be required for enhancement of both visual and olfactory memory after multisensory training. Voltage imaging in head-fixed flies showed that multisensory learning binds activity between streams of modality-specific KCs so that unimodal sensory input generates a multimodal neuronal response. Binding occurs between regions of the olfactory and visual KC axons, which receive valence-relevant dopaminergic reinforcement, and is propagated downstream. Dopamine locally releases GABAergic inhibition to permit specific microcircuits within KC-spanning serotonergic neurons to function as an excitatory bridge between the previously 'modality-selective' KC streams. Cross-modal binding thereby expands the KCs representing the memory engram for each modality into those representing the other. This broadening of the engram improves memory performance after multisensory learning and permits a single sensory feature to retrieve the memory of the multimodal experience.

A cognitive account of trace conditioning in insects

Trace conditioning is a form of Pavlovian learning in which the conditioned stimulus (CS) and the unconditioned stimulus (US) are separated by a temporal gap. Insects learn trace associations of variable nature (appetitive, aversive) and involving CSs of different sensory modalities (olfactory, visual). The accessibility of the insect neural system in behaving animals allowed identifying neural processes driving trace conditioning: the existence of prolonged neural responses to the CS after stimulus offset and the anticipation of US responses during the free-stimulus gap. Specific brain structures, such as the mushroom bodies seem to be allocated to this learning form. Here, we posit that a further component facilitating trace conditioning in insects relates to neuromodulatory mechanisms underlying enhanced attention. We thus propose a model based on different types of mushroom-body neurons, which provides a cognitive account of trace conditioning in insects.

Local 5-HT signaling bi-directionally regulates the coincidence time window for associative learning

The coincidence between conditioned stimulus (CS) and unconditioned stimulus (US) is essential for associative learning; however, the mechanism regulating the duration of this temporal window remains unclear. Here, we found that serotonin (5-HT) bi-directionally regulates the coincidence time window of olfactory learning in Drosophila and affects synaptic plasticity of Kenyon cells (KCs) in the mushroom body (MB). Utilizing GPCR-activation-based (GRAB) neurotransmitter sensors, we found that KC-released acetylcholine (ACh) activates a serotonergic dorsal paired medial (DPM) neuron, which in turn provides inhibitory feedback to KCs. Physiological stimuli induce spatially heterogeneous 5-HT signals, which proportionally gate the intrinsic coincidence time windows of different MB compartments. Artificially reducing or increasing the DPM neuron-released 5-HT shortens or prolongs the coincidence window, respectively. In a sequential trace conditioning paradigm, this serotonergic neuromodulation helps to bridge the CS-US temporal gap. Altogether, we report a model circuitry for perceiving the temporal coincidence and determining the causal relationship between environmental events.

Modulation of aversive value coding in the vertebrate and invertebrate brain

Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.

The cellular architecture of memory modules in Drosophila supports stochastic input integration

The ability to associate neutral stimuli with valence information and to store these associations as memories forms the basis for decision making. To determine the underlying computational principles, we build a realistic computational model of a central decision module within the Drosophila mushroom body (MB), the fly's center for learning and memory. Our model combines the electron microscopy-based architecture of one MB output neuron (MBON-α3), the synaptic connectivity of its 948 presynaptic Kenyon cells (KCs), and its membrane properties obtained from patch-clamp recordings. We show that this neuron is electrotonically compact and that synaptic input corresponding to simulated odor input robustly drives its spiking behavior. Therefore, sparse innervation by KCs can efficiently control and modulate MBON activity in response to learning with minimal requirements on the specificity of synaptic localization. This architecture allows efficient storage of large numbers of memories using the flexible stochastic connectivity of the circuit.

Spontaneous recovery of reward memory through active forgetting of extinction memory

Learned behavior can be suppressed by the extinction procedure. Such extinguished memory often returns spontaneously over time, making it difficult to treat diseases such as addiction. However, the biological mechanisms underlying such spontaneous recovery remain unclear. Here, we report that the extinguished reward memory in Drosophila recovers spontaneously because extinction training forms an aversive memory that can be actively forgotten via the Rac1/Dia pathway. Manipulating Rac1 activity does not affect sugar-reward memory and its immediate extinction effect but bidirectionally regulates spontaneous recovery-the decay process of extinction. Experiments using thermogenetic inhibition and functional imaging support that such extinction appears to be coded as an aversive experience. Genetic and pharmacological inhibition of formin Dia, a downstream effector of Rac1, specifically prevents spontaneous recovery after extinction in both behavioral performance and corresponding physiological traces. Together, our data suggest that spontaneous recovery is caused by active forgetting of the opposing extinction memory.

The connectome of an insect brain

Brains contain networks of interconnected neurons and so knowing the network architecture is essential for understanding brain function. We therefore mapped the synaptic-resolution connectome of an entire insect brain (Drosophila larva) with rich behavior, including learning, value computation, and action selection, comprising 3016 neurons and 548,000 synapses. We characterized neuron types, hubs, feedforward and feedback pathways, as well as cross-hemisphere and brain-nerve cord interactions. We found pervasive multisensory and interhemispheric integration, highly recurrent architecture, abundant feedback from descending neurons, and multiple novel circuit motifs. The brain's most recurrent circuits comprised the input and output neurons of the learning center. Some structural features, including multilayer shortcuts and nested recurrent loops, resembled state-of-the-art deep learning architectures. The identified brain architecture provides a basis for future experimental and theoretical studies of neural circuits.