Identification of Dopaminergic Neurons That Can Both Establish Associative Memory and Acutely Terminate Its Behavioral Expression

Beyond prediction error: 25 years of modeling the associations formed in the insect mushroom body

The insect mushroom body has gained increasing attention as a system in which the computational basis of neural learning circuits can be unraveled. We now understand in detail the key locations in this circuit where synaptic associations are formed between sensory patterns and values leading to actions. However, the actual learning rule (or rules) implemented by neural activity and leading to synaptic change is still an open question. Here, I survey the diversity of answers that have been offered in computational models of this system over the past decades, including the recurring assumption-in line with top-down theories of associative learning-that the core function is to reduce prediction error. However, I will argue, a more bottom-up approach may ultimately reveal a richer algorithmic capacity in this still enigmatic brain neuropil.

Cognitive limits of larval Drosophila: testing for conditioned inhibition, sensory preconditioning, and second-order conditioning

Drosophila larvae are an established model system for studying the mechanisms of innate and simple forms of learned behavior. They have about 10 times fewer neurons than adult flies, and it was the low total number of their neurons that allowed for an electron microscopic reconstruction of their brain at synaptic resolution. Regarding the mushroom body, a central brain structure for many forms of associative learning in insects, it turned out that more than half of the classes of synaptic connection had previously escaped attention. Understanding the function of these circuit motifs, subsequently confirmed in adult flies, is an important current research topic. In this context, we test larval Drosophila for their cognitive abilities in three tasks that are characteristically more complex than those previously studied. Our data provide evidence for (i) conditioned inhibition, as has previously been reported for adult flies and honeybees. Unlike what is described for adult flies and honeybees, however, our data do not provide evidence for (ii) sensory preconditioning or (iii) second-order conditioning in Drosophila larvae. We discuss the methodological features of our experiments as well as four specific aspects of the organization of the larval brain that may explain why these two forms of learning are observed in adult flies and honeybees, but not in larval Drosophila.

IR76b-expressing neurons in Drosophila melanogaster are necessary for associative reward learning of an amino acid mixture

Learning where to find nutrients while at the same time avoiding toxic food is essential for survival of any animal. Using Drosophila melanogaster larvae as a study case, we investigate the role of gustatory sensory neurons expressing IR76b for associative learning of amino acids, the building blocks of proteins. We found surprising complexity in the neuronal underpinnings of sensing amino acids, and a functional division of sensory neurons. We found that the IR76b receptor is dispensable for amino acid learning, whereas the neurons expressing IR76b are specifically required for the rewarding but not the punishing effect of amino acids. This unexpected dissociation in neuronal processing of amino acids for different behavioural functions provides a study case for functional divisions of labour in gustatory systems.

Prediction error drives associative learning and conditioned behavior in a spiking model of Drosophila larva

Predicting reinforcement from sensory cues is beneficial for goal-directed behavior. In insect brains, underlying associations between cues and reinforcement, encoded by dopaminergic neurons, are formed in the mushroom body. We propose a spiking model of the Drosophila larva mushroom body. It includes a feedback motif conveying learned reinforcement expectation to dopaminergic neurons, which can compute prediction error as the difference between expected and present reinforcement. We demonstrate that this can serve as a driving force in learning. When combined with synaptic homeostasis, our model accounts for theoretically derived features of acquisition and loss of associations that depend on the intensity of the reinforcement and its temporal proximity to the cue. From modeling olfactory learning over the time course of behavioral experiments and simulating the locomotion of individual larvae toward or away from odor sources in a virtual environment, we conclude that learning driven by prediction errors can explain larval behavior.

Rewarding Capacity of Optogenetically Activating a Giant GABAergic Central-Brain Interneuron in Larval Drosophila

Larvae of the fruit fly Drosophila melanogaster are a powerful study case for understanding the neural circuits underlying behavior. Indeed, the numerical simplicity of the larval brain has permitted the reconstruction of its synaptic connectome, and genetic tools for manipulating single, identified neurons allow neural circuit function to be investigated with relative ease and precision. We focus on one of the most complex neurons in the brain of the larva (of either sex), the GABAergic anterior paired lateral neuron (APL). Using behavioral and connectomic analyses, optogenetics, Ca2+ imaging, and pharmacology, we study how APL affects associative olfactory memory. We first provide a detailed account of the structure, regional polarity, connectivity, and metamorphic development of APL, and further confirm that optogenetic activation of APL has an inhibiting effect on its main targets, the mushroom body Kenyon cells. All these findings are consistent with the previously identified function of APL in the sparsening of sensory representations. To our surprise, however, we found that optogenetically activating APL can also have a strong rewarding effect. Specifically, APL activation together with odor presentation establishes an odor-specific, appetitive, associative short-term memory, whereas naive olfactory behavior remains unaffected. An acute, systemic inhibition of dopamine synthesis as well as an ablation of the dopaminergic pPAM neurons impair reward learning through APL activation. Our findings provide a study case of complex circuit function in a numerically simple brain, and suggest a previously unrecognized capacity of central-brain GABAergic neurons to engage in dopaminergic reinforcement.SIGNIFICANCE STATEMENT The single, identified giant anterior paired lateral (APL) neuron is one of the most complex neurons in the insect brain. It is GABAergic and contributes to the sparsening of neuronal activity in the mushroom body, the memory center of insects. We provide the most detailed account yet of the structure of APL in larval Drosophila as a neurogenetically accessible study case. We further reveal that, contrary to expectations, the experimental activation of APL can exert a rewarding effect, likely via dopaminergic reward pathways. The present study both provides an example of unexpected circuit complexity in a numerically simple brain, and reports an unexpected effect of activity in central-brain GABAergic circuits.

Roles of octopamine neurons in the vertical lobe of the mushroom body for the execution of a conditioned response in cockroaches

Aminergic neurons mediate reward signals in mammals and insects. In crickets, we showed that blockade of synaptic transmission from octopamine neurons (OANs) impairs conditioning of an odor (conditioned stimulus, CS) with water or sucrose (unconditioned stimulus, US) and execution of a conditioned response (CR) to the CS. It has not yet been established, however, whether findings in crickets can be applied to other species of insects. In this study, we investigated the roles of OANs in conditioning of salivation, monitored by activities of salivary neurons, and in execution of the CR in cockroaches (Periplaneta americana). We showed that injection of epinastine (an OA receptor antagonist) into the head hemolymph impaired both conditioning and execution of the CR, in accordance with findings in crickets. Moreover, local injection of epinastine into the vertical lobes of the mushroom body (MB), the center for associative learning and control of the CR, impaired execution of the CR, whereas injection of epinastine into the calyces of the MB or the antennal lobes (primary olfactory centers) did not. We propose that OANs in the MB vertical lobes play critical roles in the execution of the CR in cockroaches. This is analogous to the fact that midbrain dopamine neurons govern execution of learned actions in mammals.

High-resolution analysis of individual Drosophila melanogaster larvae uncovers individual variability in locomotion and its neurogenetic modulation

Neuronally orchestrated muscular movement and locomotion are defining faculties of multicellular animals. Due to its simple brain and genetic accessibility, the larva of the fruit fly Drosophila melanogaster allows one to study these processes at tractable levels of complexity. However, although the faculty of locomotion clearly pertains to the individual, most studies of locomotion in larvae use measurements aggregated across animals, or animals tested one by one, an extravagance for larger-scale analyses. This prevents grasping the inter- and intra-individual variability in locomotion and its neurogenetic determinants. Here, we present the IMBA (individual maggot behaviour analyser) for analysing the behaviour of individual larvae within groups, reliably resolving individual identity across collisions. We use the IMBA to systematically describe the inter- and intra-individual variability in locomotion of wild-type animals, and how the variability is reduced by associative learning. We then report a novel locomotion phenotype of an adhesion GPCR mutant. We further investigated the modulation of locomotion across repeated activations of dopamine neurons in individual animals, and the transient backward locomotion induced by brief optogenetic activation of the brain-descending 'mooncrawler' neurons. In summary, the IMBA is an easy-to-use toolbox allowing an unprecedentedly rich view of the behaviour and its variability of individual larvae, with utility in multiple biomedical research contexts.

Optogenetically induced reward and 'frustration' memory in larval Drosophila

Humans and animals alike form oppositely valenced memories for stimuli that predict the occurrence versus the termination of a reward: appetitive 'reward' memory for stimuli associated with the occurrence of a reward and aversive 'frustration' memory for stimuli that are associated with its termination. We characterize these memories in larval Drosophila using a combination of Pavlovian conditioning, optogenetic activation of the dopaminergic central-brain DAN-i1864 neuron, and high-resolution video-tracking. This reveals their dependency on the number of training trials and the duration of DAN-i1864 activation, their temporal stability, and the parameters of locomotion that are modulated during memory expression. Together with previous results on 'punishment' versus 'relief' learning by DAN-f1 neuron activation, this reveals a 2x2 matrix of timing-dependent memory valence for the occurrence/ termination of reward/ punishment. These findings should aid the understanding and modelling of how brains decipher the predictive, causal structure of events around a target reinforcing occurrence.

Curcumin protects against manganese-induced neurotoxicity in rat by regulating oxidative stress-related gene expression via H3K27 acetylation

Long-term manganese exposure causes a neurodegenerative disorder referred to as manganese poisoning, but the mechanism remains unclear and no specific treatment is available. Oxidative stress is widely recognised as one of the main causes of manganese-induced neurotoxicity. In recent years, the role of histone acetylation in neurodegenerative diseases has been widely concerned. curcumin is a natural polyphenol compound extracted from the rhizome of turmeric and exhibits both antioxidant and neuroprotective properties. Therefore, we aimed to investigate whether and how curcumin protects against manganese-induced neurotoxicity from the perspective of histone acetylation, based on the reversibility of histone acetylation modification. In this study, rats were treated with or without curcumin and subjected to long-term manganese exposure. Results that treatment of manganese decreased the protein expression of H3K18 acetylation and H3K27 acetylation at the promoters of oxidative stress-related genes and inhibited the expression of these genes. Nevertheless, curcumin increased the H3K27 acetylation level at the manganese superoxide dismutase (SOD2) gene promoter and promoted the expression of SOD2 gene. Oxidative damage in the rat striatum as well as learning and memory dysfunction were ameliorated after curcumin treatment. Taken together, our results suggest that the regulation of oxidative stress by histone acetylation may be a key mechanism of manganese-induced neurotoxicity. In addition, curcumin ameliorates Mn-induced neurotoxicity may be due to alleviation of oxidative damage mediated by increased activation of H3K27 acetylation at the SOD2 gene promoter.

An incentive circuit for memory dynamics in the mushroom body of Drosophila melanogaster

Insects adapt their response to stimuli, such as odours, according to their pairing with positive or negative reinforcements, such as sugar or shock. Recent electrophysiological and imaging findings in Drosophila melanogaster allow detailed examination of the neural mechanisms supporting the acquisition, forgetting, and assimilation of memories. We propose that this data can be explained by the combination of a dopaminergic plasticity rule that supports a variety of synaptic strength change phenomena, and a circuit structure (derived from neuroanatomy) between dopaminergic and output neurons that creates different roles for specific neurons. Computational modelling shows that this circuit allows for rapid memory acquisition, transfer from short term to long term, and exploration/exploitation trade-off. The model can reproduce the observed changes in the activity of each of the identified neurons in conditioning paradigms and can be used for flexible behavioural control.

Neural Circuits Underlying Behavioral Flexibility: Insights From Drosophila

Behavioral flexibility is critical to survival. Animals must adapt their behavioral responses based on changes in the environmental context, internal state, or experience. Studies in Drosophila melanogaster have provided insight into the neural circuit mechanisms underlying behavioral flexibility. Here we discuss how Drosophila behavior is modulated by internal and behavioral state, environmental context, and learning. We describe general principles of neural circuit organization and modulation that underlie behavioral flexibility, principles that are likely to extend to other species.

Switch-like and persistent memory formation in individual Drosophila larvae

Associative learning allows animals to use past experience to predict future events. The circuits underlying memory formation support immediate and sustained changes in function, often in response to a single example. Larval Drosophila is a genetic model for memory formation that can be accessed at molecular, synaptic, cellular, and circuit levels, often simultaneously, but existing behavioral assays for larval learning and memory do not address individual animals, and it has been difficult to form long-lasting memories, especially those requiring synaptic reorganization. We demonstrate a new assay for learning and memory capable of tracking the changing preferences of individual larvae. We use this assay to explore how activation of a pair of reward neurons changes the response to the innately aversive gas carbon dioxide (CO2). We confirm that when coupled to CO2 presentation in appropriate temporal sequence, optogenetic reward reduces avoidance of CO2. We find that learning is switch-like: all-or-none and quantized in two states. Memories can be extinguished by repeated unrewarded exposure to CO2 but are stabilized against extinction by repeated training or overnight consolidation. Finally, we demonstrate long-lasting protein synthesis dependent and independent memory formation.

Movements, embodiment and the emergence of decisions. Insights from insect navigation

We readily infer that animals make decisions, but what this implies is usually not clearly defined. The notion of 'decision-making' ultimately stems from human introspection, and is thus loaded with anthropomorphic assumptions. Notably, the decision is made internally, is based on information, and precedes the goal directed behaviour. Also, making a decision implies that 'something' did it, thus hints at the presence of a cognitive mind, whose existence is independent of the decision itself. This view may convey some truth, but here I take the opposite stance. Using examples from research in insect navigation, this essay highlights how apparent decisions can emerge without a brain, how actions can precede information or how sophisticated goal directed behaviours can be implemented without neural decisions. This perspective requires us to shake off the idea that behaviour is a consequence of the brain; and embrace the concept that movements arise from - as much as participate in - distributed interactions between various computational centres - including the body - that reverberate in closed-loop with the environment. From this perspective we may start to picture how a cognitive mind can be the consequence, rather than the cause, of such neural and body movements.

Aversive teaching signals from individual dopamine neurons in larval Drosophila show qualitative differences in their temporal "fingerprint"

Dopamine serves many functions, and dopamine neurons are correspondingly diverse. We use a combination of optogenetics, behavioral experiments, and high-resolution video-tracking to probe for the functional capacities of two single, identified dopamine neurons in larval Drosophila. The DAN-f1 and the DAN-d1 neuron were recently found to carry aversive teaching signals during Pavlovian olfactory learning. We enquire into a fundamental feature of these teaching signals, namely their temporal "fingerprint". That is, receiving punishment feels bad, whereas being relieved from it feels good, and animals and humans alike learn with opposite valence about the occurrence and the termination of punishment (the same principle applies in the appetitive domain, with opposite sign). We find that DAN-f1 but not DAN-d1 can mediate such timing-dependent valence reversal: presenting an odor before DAN-f1 activation leads to learned avoidance of the odor (punishment memory), whereas presenting the odor upon termination of DAN-f1 activation leads to learned approach (relief memory). In contrast, DAN-d1 confers punishment memory only. These effects are further characterized in terms of the impact of the duration of optogenetic activation, the temporal stability of the memories thus established, and the specific microbehavioral patterns of locomotion through which they are expressed. Together with recent findings in the appetitive domain and from adult Drosophila, our results suggest that heterogeneity in the temporal fingerprint of teaching signals might be a more general principle of reinforcement processing through dopamine neurons.

The connectome of the adult Drosophila mushroom body provides insights into function

Making inferences about the computations performed by neuronal circuits from synapse-level connectivity maps is an emerging opportunity in neuroscience. The mushroom body (MB) is well positioned for developing and testing such an approach due to its conserved neuronal architecture, recently completed dense connectome, and extensive prior experimental studies of its roles in learning, memory, and activity regulation. Here, we identify new components of the MB circuit in Drosophila, including extensive visual input and MB output neurons (MBONs) with direct connections to descending neurons. We find unexpected structure in sensory inputs, in the transfer of information about different sensory modalities to MBONs, and in the modulation of that transfer by dopaminergic neurons (DANs). We provide insights into the circuitry used to integrate MB outputs, connectivity between the MB and the central complex and inputs to DANs, including feedback from MBONs. Our results provide a foundation for further theoretical and experimental work.