Associative Learning of Stimuli Paired and Unpaired With Reinforcement: Evaluating Evidence From Maggots, Flies, Bees, and Rats

Beyond prediction error: 25 years of modeling the associations formed in the insect mushroom body

The insect mushroom body has gained increasing attention as a system in which the computational basis of neural learning circuits can be unraveled. We now understand in detail the key locations in this circuit where synaptic associations are formed between sensory patterns and values leading to actions. However, the actual learning rule (or rules) implemented by neural activity and leading to synaptic change is still an open question. Here, I survey the diversity of answers that have been offered in computational models of this system over the past decades, including the recurring assumption-in line with top-down theories of associative learning-that the core function is to reduce prediction error. However, I will argue, a more bottom-up approach may ultimately reveal a richer algorithmic capacity in this still enigmatic brain neuropil.

Cognitive limits of larval Drosophila: testing for conditioned inhibition, sensory preconditioning, and second-order conditioning

Drosophila larvae are an established model system for studying the mechanisms of innate and simple forms of learned behavior. They have about 10 times fewer neurons than adult flies, and it was the low total number of their neurons that allowed for an electron microscopic reconstruction of their brain at synaptic resolution. Regarding the mushroom body, a central brain structure for many forms of associative learning in insects, it turned out that more than half of the classes of synaptic connection had previously escaped attention. Understanding the function of these circuit motifs, subsequently confirmed in adult flies, is an important current research topic. In this context, we test larval Drosophila for their cognitive abilities in three tasks that are characteristically more complex than those previously studied. Our data provide evidence for (i) conditioned inhibition, as has previously been reported for adult flies and honeybees. Unlike what is described for adult flies and honeybees, however, our data do not provide evidence for (ii) sensory preconditioning or (iii) second-order conditioning in Drosophila larvae. We discuss the methodological features of our experiments as well as four specific aspects of the organization of the larval brain that may explain why these two forms of learning are observed in adult flies and honeybees, but not in larval Drosophila.

IR76b-expressing neurons in Drosophila melanogaster are necessary for associative reward learning of an amino acid mixture

Learning where to find nutrients while at the same time avoiding toxic food is essential for survival of any animal. Using Drosophila melanogaster larvae as a study case, we investigate the role of gustatory sensory neurons expressing IR76b for associative learning of amino acids, the building blocks of proteins. We found surprising complexity in the neuronal underpinnings of sensing amino acids, and a functional division of sensory neurons. We found that the IR76b receptor is dispensable for amino acid learning, whereas the neurons expressing IR76b are specifically required for the rewarding but not the punishing effect of amino acids. This unexpected dissociation in neuronal processing of amino acids for different behavioural functions provides a study case for functional divisions of labour in gustatory systems.

A simple semi-automated home-tank method and procedure to explore classical associative learning in adult zebrafish

The zebrafish is a laboratory species that gained increasing popularity the last decade in a variety of subfields of biology, including toxicology, ecology, medicine, and the neurosciences. An important phenotype often measured in these fields is behaviour. Consequently, numerous new behavioural apparati and paradigms have been developed for the zebrafish, including methods for the analysis of learning and memory in adult zebrafish. Perhaps the biggest obstacle in these methods is that zebrafish is particularly sensitive to human handling. To overcome this confound, automated learning paradigms have been developed with varying success. In this manuscript, we present a semi-automated home tank-based learning/memory test paradigm utilizing visual cues, and show that it is capable of quantifying classical associative learning performance in zebrafish. We demonstrate that in this task, zebrafish successfully acquire the association between coloured-light and food reward. The hardware and software components of the task are easy and cheap to obtain and simple to assemble and set up. The procedures of the paradigm allow the test fish to remain completely undisturbed by the experimenter for several days in their home (test) tank, eliminating human handling or human interference induced stress. We demonstrate that the development of cheap and simple automated home-tank-based learning paradigms for the zebrafish is feasible. We argue that such tasks will allow us to better characterize numerous cognitive and mnemonic features of the zebrafish, including elemental as well as configural learning and memory, which will, in turn, also enhance our ability to study neurobiological mechanisms underlying learning and memory using this model organism.

Prediction error drives associative learning and conditioned behavior in a spiking model of Drosophila larva

Predicting reinforcement from sensory cues is beneficial for goal-directed behavior. In insect brains, underlying associations between cues and reinforcement, encoded by dopaminergic neurons, are formed in the mushroom body. We propose a spiking model of the Drosophila larva mushroom body. It includes a feedback motif conveying learned reinforcement expectation to dopaminergic neurons, which can compute prediction error as the difference between expected and present reinforcement. We demonstrate that this can serve as a driving force in learning. When combined with synaptic homeostasis, our model accounts for theoretically derived features of acquisition and loss of associations that depend on the intensity of the reinforcement and its temporal proximity to the cue. From modeling olfactory learning over the time course of behavioral experiments and simulating the locomotion of individual larvae toward or away from odor sources in a virtual environment, we conclude that learning driven by prediction errors can explain larval behavior.

Rewarding Capacity of Optogenetically Activating a Giant GABAergic Central-Brain Interneuron in Larval Drosophila

Larvae of the fruit fly Drosophila melanogaster are a powerful study case for understanding the neural circuits underlying behavior. Indeed, the numerical simplicity of the larval brain has permitted the reconstruction of its synaptic connectome, and genetic tools for manipulating single, identified neurons allow neural circuit function to be investigated with relative ease and precision. We focus on one of the most complex neurons in the brain of the larva (of either sex), the GABAergic anterior paired lateral neuron (APL). Using behavioral and connectomic analyses, optogenetics, Ca2+ imaging, and pharmacology, we study how APL affects associative olfactory memory. We first provide a detailed account of the structure, regional polarity, connectivity, and metamorphic development of APL, and further confirm that optogenetic activation of APL has an inhibiting effect on its main targets, the mushroom body Kenyon cells. All these findings are consistent with the previously identified function of APL in the sparsening of sensory representations. To our surprise, however, we found that optogenetically activating APL can also have a strong rewarding effect. Specifically, APL activation together with odor presentation establishes an odor-specific, appetitive, associative short-term memory, whereas naive olfactory behavior remains unaffected. An acute, systemic inhibition of dopamine synthesis as well as an ablation of the dopaminergic pPAM neurons impair reward learning through APL activation. Our findings provide a study case of complex circuit function in a numerically simple brain, and suggest a previously unrecognized capacity of central-brain GABAergic neurons to engage in dopaminergic reinforcement.SIGNIFICANCE STATEMENT The single, identified giant anterior paired lateral (APL) neuron is one of the most complex neurons in the insect brain. It is GABAergic and contributes to the sparsening of neuronal activity in the mushroom body, the memory center of insects. We provide the most detailed account yet of the structure of APL in larval Drosophila as a neurogenetically accessible study case. We further reveal that, contrary to expectations, the experimental activation of APL can exert a rewarding effect, likely via dopaminergic reward pathways. The present study both provides an example of unexpected circuit complexity in a numerically simple brain, and reports an unexpected effect of activity in central-brain GABAergic circuits.

Challenges and advanced concepts for the assessment of learning and memory function in mice

The mechanisms underlying the formation and retrieval of memories are still an active area of research and discussion. Manifold models have been proposed and refined over the years, with most assuming a dichotomy between memory processes involving non-conscious and conscious mechanisms. Despite our incomplete understanding of the underlying mechanisms, tests of memory and learning count among the most performed behavioral experiments. Here, we will discuss available protocols for testing learning and memory using the example of the most prevalent animal species in research, the laboratory mouse. A wide range of protocols has been developed in mice to test, e.g., object recognition, spatial learning, procedural memory, sequential problem solving, operant- and fear conditioning, and social recognition. Those assays are carried out with individual subjects in apparatuses such as arenas and mazes, which allow for a high degree of standardization across laboratories and straightforward data interpretation but are not without caveats and limitations. In animal research, there is growing concern about the translatability of study results and animal welfare, leading to novel approaches beyond established protocols. Here, we present some of the more recent developments and more advanced concepts in learning and memory testing, such as multi-step sequential lockboxes, assays involving groups of animals, as well as home cage-based assays supported by automated tracking solutions; and weight their potential and limitations against those of established paradigms. Shifting the focus of learning tests from the classical experimental chamber to settings which are more natural for rodents comes with a new set of challenges for behavioral researchers, but also offers the opportunity to understand memory formation and retrieval in a more conclusive way than has been attainable with conventional test protocols. We predict and embrace an increase in studies relying on methods involving a higher degree of automatization, more naturalistic- and home cage-based experimental setting as well as more integrated learning tasks in the future. We are confident these trends are suited to alleviate the burden on animal subjects and improve study designs in memory research.

Optogenetically induced reward and 'frustration' memory in larval Drosophila

Humans and animals alike form oppositely valenced memories for stimuli that predict the occurrence versus the termination of a reward: appetitive 'reward' memory for stimuli associated with the occurrence of a reward and aversive 'frustration' memory for stimuli that are associated with its termination. We characterize these memories in larval Drosophila using a combination of Pavlovian conditioning, optogenetic activation of the dopaminergic central-brain DAN-i1864 neuron, and high-resolution video-tracking. This reveals their dependency on the number of training trials and the duration of DAN-i1864 activation, their temporal stability, and the parameters of locomotion that are modulated during memory expression. Together with previous results on 'punishment' versus 'relief' learning by DAN-f1 neuron activation, this reveals a 2x2 matrix of timing-dependent memory valence for the occurrence/ termination of reward/ punishment. These findings should aid the understanding and modelling of how brains decipher the predictive, causal structure of events around a target reinforcing occurrence.

Dopamine activity in projection neurons regulates short-lasting olfactory approach memory in Drosophila

Survival in many animals requires the ability to associate certain cues with danger and others with safety. In a Drosophila melanogaster aversive olfactory conditioning paradigm, flies are exposed to two odours, one presented coincidentally with electrical shocks, and a second presented 45 s after shock cessation. When flies are later given a choice between these two odours, they avoid the shock‐paired odour and prefer the unpaired odour. While many studies have examined how flies learn to avoid the shock‐paired odour through formation of odour‐fear associations, here we demonstrate that conditioning also causes flies to actively approach the second odour. In contrast to fear memories, which are longer lasting and requires activity of D1‐like dopamine receptors only in the mushroom bodies, approach memory is short‐lasting and requires activity of D1‐like dopamine receptors in projection neurons originating from the antennal lobes, primary olfactory centers. Further, while recall of fear memories requires activity of the mushroom bodies, recall of approach memories does not. Our data suggest that olfactory approach memory is formed using different mechanisms in different brain locations compared to aversive and appetitive olfactory memories.

An incentive circuit for memory dynamics in the mushroom body of Drosophila melanogaster

Insects adapt their response to stimuli, such as odours, according to their pairing with positive or negative reinforcements, such as sugar or shock. Recent electrophysiological and imaging findings in Drosophila melanogaster allow detailed examination of the neural mechanisms supporting the acquisition, forgetting, and assimilation of memories. We propose that this data can be explained by the combination of a dopaminergic plasticity rule that supports a variety of synaptic strength change phenomena, and a circuit structure (derived from neuroanatomy) between dopaminergic and output neurons that creates different roles for specific neurons. Computational modelling shows that this circuit allows for rapid memory acquisition, transfer from short term to long term, and exploration/exploitation trade-off. The model can reproduce the observed changes in the activity of each of the identified neurons in conditioning paradigms and can be used for flexible behavioural control.

High-throughput automated methods for classical and operant conditioning of Drosophila larvae

Learning which stimuli (classical conditioning) or which actions (operant conditioning) predict rewards or punishments can improve chances of survival. However, the circuit mechanisms that underlie distinct types of associative learning are still not fully understood. Automated, high-throughput paradigms for studying different types of associative learning, combined with manipulation of specific neurons in freely behaving animals, can help advance this field. The Drosophila melanogaster larva is a tractable model system for studying the circuit basis of behaviour, but many forms of associative learning have not yet been demonstrated in this animal. Here, we developed a high-throughput (i.e. multi-larva) training system that combines real-time behaviour detection of freely moving larvae with targeted opto- and thermogenetic stimulation of tracked animals. Both stimuli are controlled in either open- or closed-loop, and delivered with high temporal and spatial precision. Using this tracker, we show for the first time that Drosophila larvae can perform classical conditioning with no overlap between sensory stimuli (i.e. trace conditioning). We also demonstrate that larvae are capable of operant conditioning by inducing a bend direction preference through optogenetic activation of reward-encoding serotonergic neurons. Our results extend the known associative learning capacities of Drosophila larvae. Our automated training rig will facilitate the study of many different forms of associative learning and the identification of the neural circuits that underpin them.

Neuroplastic Changes in the Superior Colliculus and Hippocampus in Self-rewarding Paradigm: Importance of Visual Cues

Coincident excitation via different sensory modalities encoding objects of positive salience is known to facilitate learning and memory. With a view to dissect the contribution of visual cues in inducing adaptive neural changes, we monitored the lever press activity of a rat conditioned to self-administer sweet food pellets in the presence/absence of light cues. Application of light cues facilitated learning and consolidation of long-term memory. The superior colliculus (SC) of rats trained on light cue showed increased neuronal activity, dendritic branching, and brain-derived neurotrophic factor (BDNF) protein and mRNA expression. Concomitantly, the hippocampus showed augmented neurogenesis as well as BDNF protein and mRNA expression. While intra-SC administration of U0126 (inhibitor of ERK 1/2 and long-term memory) impaired memory formation, lidocaine (local anaesthetic) hindered memory recall. The light cue-dependent sweet food pellet self-administration was coupled with increased efflux of dopamine (DA) and 3,4-dihydroxyphenylacetic acid (DOPAC) in the nucleus accumbens shell (AcbSh). In conditioned rats, pharmacological inhibition of glutamatergic signalling in dentate gyrus (DG) reduced lever press activity, as well as DA and DOPAC secretion in the AcbSh. We suggest that the neuroplastic changes in the SC and hippocampus might represent memory engrams sculpted by visual cues encoding reward information.

Circuits for integrating learned and innate valences in the insect brain

Animal behavior is shaped both by evolution and by individual experience. Parallel brain pathways encode innate and learned valences of cues, but the way in which they are integrated during action-selection is not well understood. We used electron microscopy to comprehensively map with synaptic resolution all neurons downstream of all mushroom body (MB) output neurons (encoding learned valences) and characterized their patterns of interaction with lateral horn (LH) neurons (encoding innate valences) in Drosophila larva. The connectome revealed multiple convergence neuron types that receive convergent MB and LH inputs. A subset of these receives excitatory input from positive-valence MB and LH pathways and inhibitory input from negative-valence MB pathways. We confirmed functional connectivity from LH and MB pathways and behavioral roles of two of these neurons. These neurons encode integrated odor value and bidirectionally regulate turning. Based on this, we speculate that learning could potentially skew the balance of excitation and inhibition onto these neurons and thereby modulate turning. Together, our study provides insights into the circuits that integrate learned and innate valences to modify behavior.

Switch-like and persistent memory formation in individual Drosophila larvae

Associative learning allows animals to use past experience to predict future events. The circuits underlying memory formation support immediate and sustained changes in function, often in response to a single example. Larval Drosophila is a genetic model for memory formation that can be accessed at molecular, synaptic, cellular, and circuit levels, often simultaneously, but existing behavioral assays for larval learning and memory do not address individual animals, and it has been difficult to form long-lasting memories, especially those requiring synaptic reorganization. We demonstrate a new assay for learning and memory capable of tracking the changing preferences of individual larvae. We use this assay to explore how activation of a pair of reward neurons changes the response to the innately aversive gas carbon dioxide (CO2). We confirm that when coupled to CO2 presentation in appropriate temporal sequence, optogenetic reward reduces avoidance of CO2. We find that learning is switch-like: all-or-none and quantized in two states. Memories can be extinguished by repeated unrewarded exposure to CO2 but are stabilized against extinction by repeated training or overnight consolidation. Finally, we demonstrate long-lasting protein synthesis dependent and independent memory formation.

To eat or not to eat: a Garcia effect in pond snails (Lymnaea stagnalis)

Taste aversion learning is universal. In animals, a single presentation of a novel food substance followed hours later by visceral illness causes animals to avoid that taste. This is known as bait-shyness or the Garcia effect. Humans demonstrate this by avoiding a certain food following the development of nausea after ingesting that food ('Sauce Bearnaise effect'). Here, we show that the pond snail Lymnaea stagnalis is capable of the Garcia effect. A single 'pairing' of a novel taste, a carrot slurry followed hours later by a heat shock stressor (HS) is sufficient to suppress feeding response elicited by carrot for at least 24 h. Other food tastes are not suppressed. If snails had previously been exposed to carrot as their food source, the Garcia-like effect does not occur when carrot is 'paired' with the HS. The HS up-regulates two heat shock proteins (HSPs), HSP70 and HSP40. Blocking the up-regulation of the HSPs by a flavonoid, quercetin, before the heat shock, prevented the Garcia effect in the snails. Finally, we found that snails exhibit Garcia effect following a period of food deprivation but the long-term memory (LTM) phenotype can be observed only if the animals are tested in a food satiated state.

Identification of Dopaminergic Neurons That Can Both Establish Associative Memory and Acutely Terminate Its Behavioral Expression

An adaptive transition from exploring the environment in search of vital resources to exploiting these resources once the search was successful is important to all animals. Here we study the neuronal circuitry that allows larval Drosophila melanogaster of either sex to negotiate this exploration-exploitation transition. We do so by combining Pavlovian conditioning with high-resolution behavioral tracking, optogenetic manipulation of individually identified neurons, and EM data-based analyses of synaptic organization. We find that optogenetic activation of the dopaminergic neuron DAN-i1 can both establish memory during training and acutely terminate learned search behavior in a subsequent recall test. Its activation leaves innate behavior unaffected, however. Specifically, DAN-i1 activation can establish associative memories of opposite valence after paired and unpaired training with odor, and its activation during the recall test can terminate the search behavior resulting from either of these memories. Our results further suggest that in its behavioral significance DAN-i1 activation resembles, but does not equal, sugar reward. Dendrogram analyses of all the synaptic connections between DAN-i1 and its two main targets, the Kenyon cells and the mushroom body output neuron MBON-i1, further suggest that the DAN-i1 signals during training and during the recall test could be delivered to the Kenyon cells and to MBON-i1, respectively, within previously unrecognized, locally confined branching structures. This would provide an elegant circuit motif to terminate search on its successful completion.SIGNIFICANCE STATEMENT In the struggle for survival, animals have to explore their environment in search of food. Once food is found, however, it is adaptive to prioritize exploiting it over continuing a search that would now be as pointless as searching for the glasses you are wearing. This exploration-exploitation trade-off is important for animals and humans, as well as for technical search devices. We investigate which of the only 10,000 neurons of a fruit fly larva can tip the balance in this trade-off, and identify a single dopamine neuron called DAN-i1 that can do so. Given the similarities in dopamine neuron function across the animal kingdom, this may reflect a general principle of how search is terminated once it is successful.

Spaced Training Forms Complementary Long-Term Memories of Opposite Valence in Drosophila

Forming long-term memory (LTM) often requires repetitive experience spread over time. Studies in Drosophila suggest aversive olfactory LTM is optimal after spaced training, multiple trials of differential odor conditioning with rest intervals. Memory after spaced training is frequently compared to that after the same number of trials without intervals. Here we show that, after spaced training, flies acquire additional information and form an aversive memory for the shock-paired odor and a slowly emerging and more persistent "safety-memory" for the explicitly unpaired odor. Safety-memory acquisition requires repetition, order, and spacing of the training trials and relies on triggering specific rewarding dopaminergic neurons. Co-existence of aversive and safety memories is evident as depression of odor-specific responses at different combinations of junctions in the mushroom body output network; combining two outputs appears to signal relative safety. Having complementary aversive and safety memories augments LTM performance after spaced training by making the odor preference more certain.

Visualization of a Distributed Synaptic Memory Code in the Drosophila Brain

During associative conditioning, animals learn which sensory cues are predictive for positive or negative conditions. Because sensory cues are encoded by distributed neurons, one has to monitor plasticity across many synapses to capture how learned information is encoded. We analyzed synaptic boutons of Kenyon cells of the Drosophila mushroom body γ lobe, a brain structure that mediates olfactory learning. A fluorescent Ca²⁺ sensor was expressed in single Kenyon cells so that axonal boutons could be assigned to distinct cells and Ca²⁺ could be measured across many animals. Learning induced directed synaptic plasticity in specific compartments along the axons. Moreover, we show that odor-evoked Ca²⁺ dynamics across boutons decorrelate as a result of associative learning. Information theory indicates that learning renders the stimulus representation more distinct compared with naive stimuli. These data reveal that synaptic boutons rather than cells act as individually modifiable units, and coherence among them is a memory-encoding parameter.

Dopamine Receptor Dop1R2 Stabilizes Appetitive Olfactory Memory through the Raf/MAPK Pathway in Drosophila

In Drosophila, dopamine signaling to the mushroom body intrinsic neurons, Kenyon cells (KCs), is critical to stabilize olfactory memory. Little is known about the downstream intracellular molecular signaling underlying memory stabilization. Here we address this question in the context of sugar-rewarded olfactory long-term memory (LTM). We show that associative training increases the phosphorylation of MAPK in KCs, via Dop1R2 signaling. Consistently, the attenuation of Dop1R2, Raf, or MAPK expression in KCs selectively impairs LTM, but not short-term memory. Moreover, we show that the LTM deficit caused by the knockdown of Dop1R2 can be rescued by expressing active Raf in KCs. Thus, the Dop1R2/Raf/MAPK pathway is a pivotal downstream effector of dopamine signaling for stabilizing appetitive olfactory memory.SIGNIFICANCE STATEMENT Dopaminergic input to the Kenyon cells (KCs) is pivotal to stabilize memory in Drosophila This process is mediated by dopamine receptors like Dop1R2. Nevertheless, little is known for its underlying molecular mechanism. Here we show that the Raf/MAPK pathway is specifically engaged in appetitive long-term memory in KCs. With combined biochemical and behavioral experiments, we reveal that activation of the Raf/MAPK pathway is regulated through Dop1R2, shedding light on how dopamine modulates intracellular signaling for memory stabilization.

Studying neural circuits of decision-making in Drosophila larva

To study neural circuits underlying decisions, the model organism used for that purpose has to be simple enough to be able to dissect the circuitry neuron by neuron across the nervous system and in the same time complex enough to be able to perform different types of decisions. Here, I lay out the case: (1) that Drosophila larva is an advantageous model system that balances well these two requirements and (2) the insights gained from this model, assuming that circuit principles may be shared across species, can be used to advance our knowledge of neural circuit implementation of decision-making in general, including in more complex brains.

Opponent processes in visual memories: A model of attraction and repulsion in navigating insects' mushroom bodies

Solitary foraging insects display stunning navigational behaviours in visually complex natural environments. Current literature assumes that these insects are mostly driven by attractive visual memories, which are learnt when the insect's gaze is precisely oriented toward the goal direction, typically along its familiar route or towards its nest. That way, an insect could return home by simply moving in the direction that appears most familiar. Here we show using virtual reconstructions of natural environments that this principle suffers from fundamental drawbacks, notably, a given view of the world does not provide information about whether the agent should turn or not to reach its goal. We propose a simple model where the agent continuously compares its current view with both goal and anti-goal visual memories, which are treated as attractive and repulsive respectively. We show that this strategy effectively results in an opponent process, albeit not at the perceptual level-such as those proposed for colour vision or polarisation detection-but at the level of the environmental space. This opponent process results in a signal that strongly correlates with the angular error of the current body orientation so that a single view of the world now suffices to indicate whether the agent should turn or not. By incorporating this principle into a simple agent navigating in reconstructed natural environments, we show that it overcomes the usual shortcomings and produces a step-increase in navigation effectiveness and robustness. Our findings provide a functional explanation to recent behavioural observations in ants and why and how so-called aversive and appetitive memories must be combined. We propose a likely neural implementation based on insects' mushroom bodies' circuitry that produces behavioural and neural predictions contrasting with previous models.

Reversal learning in Drosophila larvae

Adjusting behavior to changed environmental contingencies is critical for survival, and reversal learning provides an experimental handle on such cognitive flexibility. Here, we investigate reversal learning in larval Drosophila Using odor-taste associations, we establish olfactory reversal learning in the appetitive and the aversive domain, using either fructose as a reward or high-concentration sodium chloride as a punishment, respectively. Reversal learning is demonstrated both in differential and in absolute conditioning, in either valence domain. In differential conditioning, the animals are first trained such that an odor A is paired, for example, with the reward whereas odor B is not (A+/B); this is followed by a second training phase with reversed contingencies (A/B+). In absolute conditioning, odor B is omitted, such that the animals are first trained with paired presentations of A and reward, followed by unpaired training in the second training phase. Our results reveal "true" reversal learning in that the opposite associative effects of both the first and the second training phase are detectable after reversed-contingency training. In what is a surprisingly quick, one-trial contingency adjustment in the Drosophila larva, the present study establishes a simple and genetically easy accessible study case of cognitive flexibility.

An optogenetic analogue of second-order reinforcement in Drosophila

In insects, odours are coded by the combinatorial activation of ascending pathways, including their third-order representation in mushroom body Kenyon cells. Kenyon cells also receive intersecting input from ascending and mostly dopaminergic reinforcement pathways. Indeed, in Drosophila, presenting an odour together with activation of the dopaminergic mushroom body input neuron PPL1-01 leads to a weakening of the synapse between Kenyon cells and the approach-promoting mushroom body output neuron MBON-11. As a result of such weakened approach tendencies, flies avoid the shock-predicting odour in a subsequent choice test. Thus, increased activity in PPL1-01 stands for punishment, whereas reduced activity in MBON-11 stands for predicted punishment. Given that punishment-predictors can themselves serve as punishments of second order, we tested whether presenting an odour together with the optogenetic silencing of MBON-11 would lead to learned odour avoidance, and found this to be the case. In turn, the optogenetic activation of MBON-11 together with odour presentation led to learned odour approach. Thus, manipulating activity in MBON-11 can be an analogue of predicted, second-order reinforcement.

Aversive Training of Honey Bees in an Automated Y-Maze

Honeybees have remarkable learning abilities given their small brains, and have thus been established as a powerful model organism for the study of learning and memory. Most of our current knowledge is based on appetitive paradigms, in which a previously neutral stimulus (e.g., a visual, olfactory, or tactile stimulus) is paired with a reward. Here, we present a novel apparatus, the yAPIS, for aversive training of walking honey bees. This system consists in three arms of equal length and at 120° from each other. Within each arm, colored lights (λ = 375, 465 or 520 nm) or odors (here limonene or linalool) can be delivered to provide conditioned stimuli (CS). A metal grid placed on the floor and roof delivers the punishment in the form of mild electric shocks (unconditioned stimulus, US). Our training protocol followed a fully classical procedure, in which the bee was exposed sequentially to a CS paired with shocks (CS+) and to another CS not punished (CS-). Learning performance was measured during a second phase, which took advantage of the Y-shape of the apparatus and of real-time tracking to present the bee with a choice situation, e.g., between the CS+ and the CS-. Bees reliably chose the CS- over the CS+ after only a few training trials with either colors or odors, and retained this memory for at least a day, except for the shorter wavelength (λ = 375 nm) that produced mixed results. This behavior was largely the result of the bees avoiding the CS+, as no evidence was found for attraction to the CS-. Interestingly, trained bees initially placed in the CS+ spontaneously escaped to a CS- arm if given the opportunity, even though they could never do so during the training. Finally, honey bees trained with compound stimuli (color + odor) later avoided either components of the CS+. Thus, the yAPIS is a fast, versatile and high-throughput way to train honey bees in aversive paradigms. It also opens the door for controlled laboratory experiments investigating bimodal integration and learning, a field that remains in its infancy.

One-trial learning in larval Drosophila

Animals of many species are capable of “small data” learning, that is, of learning without repetition. Here we introduce larval Drosophila melanogaster as a relatively simple study case for such one-trial learning. Using odor-food associative conditioning, we first show that a sugar that is both sweet and nutritious (fructose) and sugars that are only sweet (arabinose) or only nutritious (sorbitol) all support appetitive one-trial learning. The same is the case for the optogenetic activation of a subset of dopaminergic neurons innervating the mushroom body, the memory center of the insects. In contrast, no one-trial learning is observed for an amino acid reward (aspartic acid). As regards the aversive domain, one-trial learning is demonstrated for high-concentration sodium chloride, but is not observed for a bitter tastant (quinine). Second, we provide follow-up, parametric analyses of odor-fructose learning. Specifically, we ascertain its dependency on the number and duration of training trials, the requirements for the behavioral expression of one-trial odor-fructose memory, its temporal stability, and the feasibility of one-trial differential conditioning. Our results set the stage for a neurogenetic analysis of one-trial learning and define the requirements for modeling mnemonic processes in the larva.