Differential coding of absolute and relative aversive value in the Drosophila brain

Dopamine-mediated interactions between short- and long-term memory dynamics

In dynamic environments, animals make behavioural decisions on the basis of the innate valences of sensory cues and information learnt about these cues across multiple timescales1-3. However, it remains unclear how the innate valence of a sensory stimulus affects the acquisition of learnt valence information and subsequent memory dynamics. Here we show that in the Drosophila brain, interconnected short- and long-term memory units of the mushroom body jointly regulate memory through dopamine signals that encode innate and learnt sensory valences. By performing time-lapse in vivo voltage-imaging studies of neural spiking in more than 500 flies undergoing olfactory associative conditioning, we found that protocerebral posterior lateral 1 dopamine neurons (PPL1-DANs)4 heterogeneously and bidirectionally encode innate and learnt valences of punishment, reward and odour cues. During learning, these valence signals regulate memory storage and extinction in mushroom body output neurons (MBONs)5. During initial conditioning bouts, PPL1-γ1pedc and PPL1-γ2α'1 neurons control short-term memory formation, which weakens inhibitory feedback from MBON-γ1pedc>α/β to PPL1-α'2α2 and PPL1-α3. During further conditioning, this diminished feedback allows these two PPL1-DANs to encode the net innate plus learnt valence of the conditioned odour cue, which gates long-term memory formation. A computational model constrained by the fly connectome6,7 and our spiking data explains how dopamine signals mediate the circuit interactions between short- and long-term memory traces, yielding predictions that our experiments confirmed. Overall, the mushroom body achieves flexible learning through the integration of innate and learnt valences in parallel learning units sharing feedback interconnections. This hybrid physiological-anatomical mechanism may be a general means by which dopamine regulates memory dynamics in other species and brain structures, including the vertebrate basal ganglia.

Compensatory enhancement of input maintains aversive dopaminergic reinforcement in hungry Drosophila

Hungry animals need compensatory mechanisms to maintain flexible brain function, while modulation reconfigures circuits to prioritize resource seeking. In Drosophila, hunger inhibits aversively reinforcing dopaminergic neurons (DANs) to permit the expression of food-seeking memories. Multitasking the reinforcement system for motivation potentially undermines aversive learning. We find that chronic hunger mildly enhances aversive learning and that satiated-baseline and hunger-enhanced learning require endocrine adipokinetic hormone (AKH) signaling. Circulating AKH influences aversive learning via its receptor in four neurons in the ventral brain, two of which are octopaminergic. Connectomics revealed AKH receptor-expressing neurons to be upstream of several classes of ascending neurons, many of which are presynaptic to aversively reinforcing DANs. Octopaminergic modulation of and output from at least one of these ascending pathways is required for shock- and bitter-taste-reinforced aversive learning. We propose that coordinated enhancement of input compensates for hunger-directed inhibition of aversive DANs to preserve reinforcement when required.

Common principles for odour coding across vertebrates and invertebrates

The olfactory system is an ideal and tractable system for exploring how the brain transforms sensory inputs into behaviour. The basic tasks of any olfactory system include odour detection, discrimination and categorization. The challenge for the olfactory system is to transform the high-dimensional space of olfactory stimuli into the much smaller space of perceived objects and valence that endows odours with meaning. Our current understanding of how neural circuits address this challenge has come primarily from observations of the mechanisms of the brain for processing other sensory modalities, such as vision and hearing, in which optimized deep hierarchical circuits are used to extract sensory features that vary along continuous physical dimensions. The olfactory system, by contrast, contends with an ill-defined, high-dimensional stimulus space and discrete stimuli using a circuit architecture that is shallow and parallelized. Here, we present recent observations in vertebrate and invertebrate systems that relate the statistical structure and state-dependent modulation of olfactory codes to mechanisms of perception and odour-guided behaviour.

Skewing information flow through pre- and postsynaptic plasticity in the mushroom bodies of Drosophila

Animal brains need to store information to construct a representation of their environment. Knowledge of what happened in the past allows both vertebrates and invertebrates to predict future outcomes by recalling previous experience. Although invertebrate and vertebrate brains share common principles at the molecular, cellular, and circuit-architectural levels, there are also obvious differences as exemplified by the use of acetylcholine versus glutamate as the considered main excitatory neurotransmitters in the respective central nervous systems. Nonetheless, across central nervous systems, synaptic plasticity is thought to be a main substrate for memory storage. Therefore, how brain circuits and synaptic contacts change following learning is of fundamental interest for understanding brain computations tied to behavior in any animal. Recent progress has been made in understanding such plastic changes following olfactory associative learning in the mushroom bodies (MBs) of Drosophila A current framework of memory-guided behavioral selection is based on the MB skew model, in which antagonistic synaptic pathways are selectively changed in strength. Here, we review insights into plasticity at dedicated Drosophila MB output pathways and update what is known about the plasticity of both pre- and postsynaptic compartments of Drosophila MB neurons.

An integrative sensor of body states: how the mushroom body modulates behavior depending on physiological context

The brain constantly compares past and present experiences to predict the future, thereby enabling instantaneous and future behavioral adjustments. Integration of external information with the animal's current internal needs and behavioral state represents a key challenge of the nervous system. Recent advancements in dissecting the function of the Drosophila mushroom body (MB) at the single-cell level have uncovered its three-layered logic and parallel systems conveying positive and negative values during associative learning. This review explores a lesser-known role of the MB in detecting and integrating body states such as hunger, thirst, and sleep, ultimately modulating motivation and sensory-driven decisions based on the physiological state of the fly. State-dependent signals predominantly affect the activity of modulatory MB input neurons (dopaminergic, serotoninergic, and octopaminergic), but also induce plastic changes directly at the level of the MB intrinsic and output neurons. Thus, the MB emerges as a tightly regulated relay station in the insect brain, orchestrating neuroadaptations due to current internal and behavioral states leading to short- but also long-lasting changes in behavior. While these adaptations are crucial to ensure fitness and survival, recent findings also underscore how circuit motifs in the MB may reflect fundamental design principles that contribute to maladaptive behaviors such as addiction or depression-like symptoms.

Sensory encoding and memory in the mushroom body: signals, noise, and variability

To survive in changing environments, animals need to learn to associate specific sensory stimuli with positive or negative valence. How do they form stimulus-specific memories to distinguish between positively/negatively associated stimuli and other irrelevant stimuli? Solving this task is one of the functions of the mushroom body, the associative memory center in insect brains. Here we summarize recent work on sensory encoding and memory in the Drosophila mushroom body, highlighting general principles such as pattern separation, sparse coding, noise and variability, coincidence detection, and spatially localized neuromodulation, and placing the mushroom body in comparative perspective with mammalian memory systems.

Roles of feedback and feed-forward networks of dopamine subsystems: insights from Drosophila studies

Across animal species, dopamine-operated memory systems comprise anatomically segregated, functionally diverse subsystems. Although individual subsystems could operate independently to support distinct types of memory, the logical interplay between subsystems is expected to enable more complex memory processing by allowing existing memory to influence future learning. Recent comprehensive ultrastructural analysis of the Drosophila mushroom body revealed intricate networks interconnecting the dopamine subsystems-the mushroom body compartments. Here, we review the functions of some of these connections that are beginning to be understood. Memory consolidation is mediated by two different forms of network: A recurrent feedback loop within a compartment maintains sustained dopamine activity required for consolidation, whereas feed-forward connections across compartments allow short-term memory formation in one compartment to open the gate for long-term memory formation in another compartment. Extinction and reversal of aversive memory rely on a similar feed-forward circuit motif that signals omission of punishment as a reward, which triggers plasticity that counteracts the original aversive memory trace. Finally, indirect feed-forward connections from a long-term memory compartment to short-term memory compartments mediate higher-order conditioning. Collectively, these emerging studies indicate that feedback control and hierarchical connectivity allow the dopamine subsystems to work cooperatively to support diverse and complex forms of learning.

Prediction error drives associative learning and conditioned behavior in a spiking model of Drosophila larva

Predicting reinforcement from sensory cues is beneficial for goal-directed behavior. In insect brains, underlying associations between cues and reinforcement, encoded by dopaminergic neurons, are formed in the mushroom body. We propose a spiking model of the Drosophila larva mushroom body. It includes a feedback motif conveying learned reinforcement expectation to dopaminergic neurons, which can compute prediction error as the difference between expected and present reinforcement. We demonstrate that this can serve as a driving force in learning. When combined with synaptic homeostasis, our model accounts for theoretically derived features of acquisition and loss of associations that depend on the intensity of the reinforcement and its temporal proximity to the cue. From modeling olfactory learning over the time course of behavioral experiments and simulating the locomotion of individual larvae toward or away from odor sources in a virtual environment, we conclude that learning driven by prediction errors can explain larval behavior.

Probiotic Lactobacillus spp. improves Drosophila memory by increasing lactate dehydrogenase levels in the brain mushroom body neurons

Probiotics are live microorganisms that offer potential benefits to their hosts and can occasionally influence behavioral responses. However, the detailed mechanisms by which probiotics affect the behavior of their hosts and the underlying biogenic effects remain unclear. Lactic acid bacteria, specifically Lactobacillus spp. are known probiotics. Drosophila melanogaster, commonly known as the fruit fly, is a well-established model organism for investigating the interaction between the host and gut microbiota in translational research. Herein, we showed that 5-day administration of Lactobacillus acidophilus (termed GMNL-185) or Lacticaseibacillus rhamnosus (termed GMNL-680) enhances olfactory-associative memory in Drosophila. Moreover, a combined diet of GMNL-185 and GMNL-680 demonstrated synergistic effects on memory functions. Live brain imaging revealed a significant increase in calcium responses to the training odor in the mushroom body β and γ lobes of flies that underwent mixed feeding with GMNL-185 and GMNL-680. Quantitative reverse transcription polymerase chain reaction (qRT-PCR) and whole-mount brain immunohistochemistry revealed significant upregulation of lactate dehydrogenase (LDH) expression in the fly brain following the mixed feeding. Notably, the genetic knockdown of Ldh in neurons, specifically in mushroom body, ameliorated the beneficial effects of mixed feeding with GMNL-185 and GMNL-680 on memory improvement. Altogether, our results demonstrate that supplementation with L. acidophilus and L. rhamnosus enhances memory functions in flies by increasing brain LDH levels.

Dopaminergic systems create reward seeking despite adverse consequences

Resource-seeking behaviours are ordinarily constrained by physiological needs and threats of danger, and the loss of these controls is associated with pathological reward seeking1. Although dysfunction of the dopaminergic valuation system of the brain is known to contribute towards unconstrained reward seeking2,3, the underlying reasons for this behaviour are unclear. Here we describe dopaminergic neural mechanisms that produce reward seeking despite adverse consequences in Drosophila melanogaster. Odours paired with optogenetic activation of a defined subset of reward-encoding dopaminergic neurons become cues that starved flies seek while neglecting food and enduring electric shock punishment. Unconstrained seeking of reward is not observed after learning with sugar or synthetic engagement of other dopaminergic neuron populations. Antagonism between reward-encoding and punishment-encoding dopaminergic neurons accounts for the perseverance of reward seeking despite punishment, whereas synthetic engagement of the reward-encoding dopaminergic neurons also impairs the ordinary need-dependent dopaminergic valuation of available food. Connectome analyses reveal that the population of reward-encoding dopaminergic neurons receives highly heterogeneous input, consistent with parallel representation of diverse rewards, and recordings demonstrate state-specific gating and satiety-related signals. We propose that a similar dopaminergic valuation system dysfunction is likely to contribute to maladaptive seeking of rewards by mammals.

Dopaminergic neurons dynamically update sensory values during olfactory maneuver

Dopaminergic neurons (DANs) drive associative learning to update the value of sensory cues, but their contribution to the assessment of sensory values outside the context of association remains largely unexplored. Here, we show in Drosophila that DANs in the mushroom body encode the innate value of odors and constantly update the current value by inducing plasticity during olfactory maneuver. Our connectome-based network model linking all the way from the olfactory neurons to DANs reproduces the characteristics of DAN responses, proposing a concrete circuit mechanism for computation. Downstream of DANs, odors alone induce value- and dopamine-dependent changes in the activity of mushroom body output neurons, which store the current value of odors. Consistent with this neural plasticity, specific sets of DANs bidirectionally modulate flies' steering in a virtual olfactory environment. Thus, the DAN circuit known for discrete, associative learning also continuously updates odor values in a nonassociative manner.

Dopamine-Dependent Plasticity Is Heterogeneously Expressed by Presynaptic Calcium Activity across Individual Boutons of the Drosophila Mushroom Body

The Drosophila mushroom body (MB) is an important model system for studying the synaptic mechanisms of associative learning. In this system, coincidence of odor-evoked calcium influx and dopaminergic input in the presynaptic terminals of Kenyon cells (KCs), the principal neurons of the MB, triggers long-term depression (LTD), which plays a critical role in olfactory learning. However, it is controversial whether such synaptic plasticity is accompanied by a corresponding decrease in odor-evoked calcium activity in the KC presynaptic terminals. Here, we address this question by inducing LTD by pairing odor presentation with optogenetic activation of dopaminergic neurons (DANs). This allows us to rigorously compare the changes at the presynaptic and postsynaptic sites in the same conditions. By imaging presynaptic acetylcholine release in the condition where LTD is reliably observed in the postsynaptic calcium signals, we show that neurotransmitter release from KCs is depressed selectively in the MB compartments innervated by activated DANs, demonstrating the presynaptic nature of LTD. However, total odor-evoked calcium activity of the KC axon bundles does not show concurrent depression. We further conduct calcium imaging in individual presynaptic boutons and uncover the highly heterogeneous nature of calcium plasticity. Namely, only a subset of boutons, which are strongly activated by associated odors, undergo calcium activity depression, while weakly responding boutons show potentiation. Thus, our results suggest an unexpected nonlinear relationship between presynaptic calcium influx and the results of plasticity, challenging the simple view of cooperative actions of presynaptic calcium and dopaminergic input.

Reward expectations direct learning and drive operant matching in Drosophila

Foraging animals must use decision-making strategies that dynamically adapt to the changing availability of rewards in the environment. A wide diversity of animals do this by distributing their choices in proportion to the rewards received from each option, Herrnstein's operant matching law. Theoretical work suggests an elegant mechanistic explanation for this ubiquitous behavior, as operant matching follows automatically from simple synaptic plasticity rules acting within behaviorally relevant neural circuits. However, no past work has mapped operant matching onto plasticity mechanisms in the brain, leaving the biological relevance of the theory unclear. Here, we discovered operant matching in Drosophila and showed that it requires synaptic plasticity that acts in the mushroom body and incorporates the expectation of reward. We began by developing a dynamic foraging paradigm to measure choices from individual flies as they learn to associate odor cues with probabilistic rewards. We then built a model of the fly mushroom body to explain each fly's sequential choice behavior using a family of biologically realistic synaptic plasticity rules. As predicted by past theoretical work, we found that synaptic plasticity rules could explain fly matching behavior by incorporating stimulus expectations, reward expectations, or both. However, by optogenetically bypassing the representation of reward expectation, we abolished matching behavior and showed that the plasticity rule must specifically incorporate reward expectations. Altogether, these results reveal the first synapse-level mechanisms of operant matching and provide compelling evidence for the role of reward expectation signals in the fly brain.

Modulation of aversive value coding in the vertebrate and invertebrate brain

Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.