1
|
Webb B. Beyond prediction error: 25 years of modeling the associations formed in the insect mushroom body. Learn Mem 2024; 31:a053824. [PMID: 38862164 PMCID: PMC11199945 DOI: 10.1101/lm.053824.123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Accepted: 03/01/2024] [Indexed: 06/13/2024]
Abstract
The insect mushroom body has gained increasing attention as a system in which the computational basis of neural learning circuits can be unraveled. We now understand in detail the key locations in this circuit where synaptic associations are formed between sensory patterns and values leading to actions. However, the actual learning rule (or rules) implemented by neural activity and leading to synaptic change is still an open question. Here, I survey the diversity of answers that have been offered in computational models of this system over the past decades, including the recurring assumption-in line with top-down theories of associative learning-that the core function is to reduce prediction error. However, I will argue, a more bottom-up approach may ultimately reveal a richer algorithmic capacity in this still enigmatic brain neuropil.
Collapse
Affiliation(s)
- Barbara Webb
- School of Informatics, University of Edinburgh, Edinburgh EH8 9AB, United Kingdom
| |
Collapse
|
2
|
Fiala A, Kaun KR. What do the mushroom bodies do for the insect brain? Twenty-five years of progress. Learn Mem 2024; 31:a053827. [PMID: 38862175 PMCID: PMC11199942 DOI: 10.1101/lm.053827.123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2024] [Accepted: 04/22/2024] [Indexed: 06/13/2024]
Abstract
In 1998, a special edition of Learning & Memory was published with a discrete focus of synthesizing the state of the field to provide an overview of the function of the insect mushroom body. While molecular neuroscience and optical imaging of larger brain areas were advancing, understanding the basic functioning of neuronal circuits, particularly in the context of the mushroom body, was rudimentary. In the past 25 years, technological innovations have allowed researchers to map and understand the in vivo function of the neuronal circuits of the mushroom body system, making it an ideal model for investigating the circuit basis of sensory encoding, memory formation, and behavioral decisions. Collaborative efforts within the community have played a crucial role, leading to an interactive connectome of the mushroom body and accessible genetic tools for studying mushroom body circuit function. Looking ahead, continued technological innovation and collaborative efforts are likely to further advance our understanding of the mushroom body and its role in behavior and cognition, providing insights that generalize to other brain structures and species.
Collapse
Affiliation(s)
- André Fiala
- Department of Molecular Neurobiology of Behaviour, University of Göttingen, Göttingen 37077, Germany
| | - Karla R Kaun
- Department of Neuroscience, Brown University, Providence, Rhode Island 02806, USA
| |
Collapse
|
3
|
Sen E, El-Keredy A, Jacob N, Mancini N, Asnaz G, Widmann A, Gerber B, Thoener J. Cognitive limits of larval Drosophila: testing for conditioned inhibition, sensory preconditioning, and second-order conditioning. Learn Mem 2024; 31:a053726. [PMID: 38862170 PMCID: PMC11199949 DOI: 10.1101/lm.053726.122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Accepted: 01/18/2024] [Indexed: 06/13/2024]
Abstract
Drosophila larvae are an established model system for studying the mechanisms of innate and simple forms of learned behavior. They have about 10 times fewer neurons than adult flies, and it was the low total number of their neurons that allowed for an electron microscopic reconstruction of their brain at synaptic resolution. Regarding the mushroom body, a central brain structure for many forms of associative learning in insects, it turned out that more than half of the classes of synaptic connection had previously escaped attention. Understanding the function of these circuit motifs, subsequently confirmed in adult flies, is an important current research topic. In this context, we test larval Drosophila for their cognitive abilities in three tasks that are characteristically more complex than those previously studied. Our data provide evidence for (i) conditioned inhibition, as has previously been reported for adult flies and honeybees. Unlike what is described for adult flies and honeybees, however, our data do not provide evidence for (ii) sensory preconditioning or (iii) second-order conditioning in Drosophila larvae. We discuss the methodological features of our experiments as well as four specific aspects of the organization of the larval brain that may explain why these two forms of learning are observed in adult flies and honeybees, but not in larval Drosophila.
Collapse
Affiliation(s)
- Edanur Sen
- Department of Genetics, Leibniz Institute for Neurobiology, 39118 Magdeburg, Germany
| | - Amira El-Keredy
- Department of Genetics, Leibniz Institute for Neurobiology, 39118 Magdeburg, Germany
- Department of Genetics, Faculty of Agriculture, Tanta University, 31111 Tanta, Egypt
| | - Nina Jacob
- Department of Genetics, Leibniz Institute for Neurobiology, 39118 Magdeburg, Germany
| | - Nino Mancini
- Department of Genetics, Leibniz Institute for Neurobiology, 39118 Magdeburg, Germany
| | - Gülüm Asnaz
- Department of Molecular Neurobiology of Behavior, University of Göttingen, 37077 Göttingen, Germany
| | - Annekathrin Widmann
- Department of Molecular Neurobiology of Behavior, University of Göttingen, 37077 Göttingen, Germany
| | - Bertram Gerber
- Department of Genetics, Leibniz Institute for Neurobiology, 39118 Magdeburg, Germany
- Otto von Guericke University Magdeburg, Institute of Biology, 39106 Magdeburg, Germany
- Center for Behavioral Brain Sciences, 39106 Magdeburg, Germany
| | - Juliane Thoener
- Department of Genetics, Leibniz Institute for Neurobiology, 39118 Magdeburg, Germany
| |
Collapse
|
4
|
Davidson AM, Hige T. Roles of feedback and feed-forward networks of dopamine subsystems: insights from Drosophila studies. Learn Mem 2024; 31:a053807. [PMID: 38862171 PMCID: PMC11199952 DOI: 10.1101/lm.053807.123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Accepted: 11/10/2023] [Indexed: 06/13/2024]
Abstract
Across animal species, dopamine-operated memory systems comprise anatomically segregated, functionally diverse subsystems. Although individual subsystems could operate independently to support distinct types of memory, the logical interplay between subsystems is expected to enable more complex memory processing by allowing existing memory to influence future learning. Recent comprehensive ultrastructural analysis of the Drosophila mushroom body revealed intricate networks interconnecting the dopamine subsystems-the mushroom body compartments. Here, we review the functions of some of these connections that are beginning to be understood. Memory consolidation is mediated by two different forms of network: A recurrent feedback loop within a compartment maintains sustained dopamine activity required for consolidation, whereas feed-forward connections across compartments allow short-term memory formation in one compartment to open the gate for long-term memory formation in another compartment. Extinction and reversal of aversive memory rely on a similar feed-forward circuit motif that signals omission of punishment as a reward, which triggers plasticity that counteracts the original aversive memory trace. Finally, indirect feed-forward connections from a long-term memory compartment to short-term memory compartments mediate higher-order conditioning. Collectively, these emerging studies indicate that feedback control and hierarchical connectivity allow the dopamine subsystems to work cooperatively to support diverse and complex forms of learning.
Collapse
Affiliation(s)
- Andrew M Davidson
- Department of Biology, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599, USA
- Department of Cell Biology and Physiology, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599, USA
- Integrative Program for Biological and Genome Sciences, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599, USA
| | - Toshihide Hige
- Department of Biology, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599, USA
- Department of Cell Biology and Physiology, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599, USA
- Integrative Program for Biological and Genome Sciences, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599, USA
| |
Collapse
|
5
|
Zhuravlev AV, Vetrovoy OV, Zalomaeva ES, Egozova ES, Nikitina EA, Savvateeva-Popova EV. Overexpression of the limk1 Gene in Drosophila melanogaster Can Lead to Suppression of Courtship Memory in Males. BIOCHEMISTRY. BIOKHIMIIA 2024; 89:393-406. [PMID: 38648760 DOI: 10.1134/s0006297924030015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Revised: 12/06/2023] [Accepted: 12/08/2023] [Indexed: 04/25/2024]
Abstract
Courtship suppression is a behavioral adaptation of the fruit fly. When majority of the females in a fly population are fertilized and non-receptive for mating, a male, after a series of failed attempts, decreases its courtship activity towards all females, saving its energy and reproductive resources. The time of courtship decrease depends on both duration of unsuccessful courtship and genetically determined features of the male nervous system. Thereby, courtship suppression paradigm can be used for studying molecular mechanisms of learning and memory. p-Cofilin, a component of the actin remodeling signaling cascade and product of LIM-kinase 1 (LIMK1), regulates Drosophila melanogaster forgetting in olfactory learning paradigm. Previously, we have shown that limk1 suppression in the specific types of nervous cells differently affects fly courtship memory. Here, we used Gal4 > UAS system to induce limk1 overexpression in the same types of neurons. limk1 activation in the mushroom body, glia, and fruitless neurons decreased learning index compared to the control strain or the strain with limk1 knockdown. In cholinergic and dopaminergic/serotoninergic neurons, both overexpression and knockdown of limk1 impaired Drosophila short-term memory. Thus, proper balance of the limk1 activity is crucial for normal cognitive activity of the fruit fly.
Collapse
Affiliation(s)
- Aleksandr V Zhuravlev
- Pavlov Institute of Physiology, Russian Academy of Sciences, Saint Petersburg, 199034, Russia.
| | - Oleg V Vetrovoy
- Pavlov Institute of Physiology, Russian Academy of Sciences, Saint Petersburg, 199034, Russia.
| | - Ekaterina S Zalomaeva
- Pavlov Institute of Physiology, Russian Academy of Sciences, Saint Petersburg, 199034, Russia.
- Herzen State Pedagogical University of Russia, Saint Petersburg, 191186, Russia
| | - Ekaterina S Egozova
- Herzen State Pedagogical University of Russia, Saint Petersburg, 191186, Russia.
| | - Ekaterina A Nikitina
- Pavlov Institute of Physiology, Russian Academy of Sciences, Saint Petersburg, 199034, Russia.
- Herzen State Pedagogical University of Russia, Saint Petersburg, 191186, Russia
| | | |
Collapse
|
6
|
Jürgensen AM, Sakagiannis P, Schleyer M, Gerber B, Nawrot MP. Prediction error drives associative learning and conditioned behavior in a spiking model of Drosophila larva. iScience 2024; 27:108640. [PMID: 38292165 PMCID: PMC10824792 DOI: 10.1016/j.isci.2023.108640] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 11/10/2023] [Accepted: 12/01/2023] [Indexed: 02/01/2024] Open
Abstract
Predicting reinforcement from sensory cues is beneficial for goal-directed behavior. In insect brains, underlying associations between cues and reinforcement, encoded by dopaminergic neurons, are formed in the mushroom body. We propose a spiking model of the Drosophila larva mushroom body. It includes a feedback motif conveying learned reinforcement expectation to dopaminergic neurons, which can compute prediction error as the difference between expected and present reinforcement. We demonstrate that this can serve as a driving force in learning. When combined with synaptic homeostasis, our model accounts for theoretically derived features of acquisition and loss of associations that depend on the intensity of the reinforcement and its temporal proximity to the cue. From modeling olfactory learning over the time course of behavioral experiments and simulating the locomotion of individual larvae toward or away from odor sources in a virtual environment, we conclude that learning driven by prediction errors can explain larval behavior.
Collapse
Affiliation(s)
- Anna-Maria Jürgensen
- Computational Systems Neuroscience, Institute of Zoology, University of Cologne, 50674 Cologne, Germany
| | - Panagiotis Sakagiannis
- Computational Systems Neuroscience, Institute of Zoology, University of Cologne, 50674 Cologne, Germany
| | - Michael Schleyer
- Leibniz Institute for Neurobiology (LIN), Department of Genetics, 39118 Magdeburg, Germany
- Institute for the Advancement of Higher Education, Faculty of Science, Hokkaido University, Sapporo 060-08080, Japan
| | - Bertram Gerber
- Leibniz Institute for Neurobiology (LIN), Department of Genetics, 39118 Magdeburg, Germany
- Institute for Biology, Otto-von-Guericke University, 39120 Magdeburg, Germany
- Center for Brain and Behavioral Sciences (CBBS), Otto-von-Guericke University, 39118 Magdeburg, Germany
| | - Martin Paul Nawrot
- Computational Systems Neuroscience, Institute of Zoology, University of Cologne, 50674 Cologne, Germany
| |
Collapse
|
7
|
Rajagopalan AE, Darshan R, Hibbard KL, Fitzgerald JE, Turner GC. Reward expectations direct learning and drive operant matching in Drosophila. Proc Natl Acad Sci U S A 2023; 120:e2221415120. [PMID: 37733736 PMCID: PMC10523640 DOI: 10.1073/pnas.2221415120] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Accepted: 08/11/2023] [Indexed: 09/23/2023] Open
Abstract
Foraging animals must use decision-making strategies that dynamically adapt to the changing availability of rewards in the environment. A wide diversity of animals do this by distributing their choices in proportion to the rewards received from each option, Herrnstein's operant matching law. Theoretical work suggests an elegant mechanistic explanation for this ubiquitous behavior, as operant matching follows automatically from simple synaptic plasticity rules acting within behaviorally relevant neural circuits. However, no past work has mapped operant matching onto plasticity mechanisms in the brain, leaving the biological relevance of the theory unclear. Here, we discovered operant matching in Drosophila and showed that it requires synaptic plasticity that acts in the mushroom body and incorporates the expectation of reward. We began by developing a dynamic foraging paradigm to measure choices from individual flies as they learn to associate odor cues with probabilistic rewards. We then built a model of the fly mushroom body to explain each fly's sequential choice behavior using a family of biologically realistic synaptic plasticity rules. As predicted by past theoretical work, we found that synaptic plasticity rules could explain fly matching behavior by incorporating stimulus expectations, reward expectations, or both. However, by optogenetically bypassing the representation of reward expectation, we abolished matching behavior and showed that the plasticity rule must specifically incorporate reward expectations. Altogether, these results reveal the first synapse-level mechanisms of operant matching and provide compelling evidence for the role of reward expectation signals in the fly brain.
Collapse
Affiliation(s)
- Adithya E. Rajagopalan
- Janelia Research Campus, HHMI, Ashburn, VA20147
- Solomon H. Snyder Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, MD21205
| | - Ran Darshan
- Janelia Research Campus, HHMI, Ashburn, VA20147
- Department of Physiology and Pharmacology, Sackler Faculty of Medicine, Sagol School of Neuroscience, The School of Physics and Astronomy, Tel Aviv University, Tel Aviv6997801, Israel
| | | | | | | |
Collapse
|
8
|
Steele TJ, Lanz AJ, Nagel KI. Olfactory navigation in arthropods. J Comp Physiol A Neuroethol Sens Neural Behav Physiol 2023; 209:467-488. [PMID: 36658447 PMCID: PMC10354148 DOI: 10.1007/s00359-022-01611-9] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2022] [Revised: 12/26/2022] [Accepted: 12/31/2022] [Indexed: 01/21/2023]
Abstract
Using odors to find food and mates is one of the most ancient and highly conserved behaviors. Arthropods from flies to moths to crabs use broadly similar strategies to navigate toward odor sources-such as integrating flow information with odor information, comparing odor concentration across sensors, and integrating odor information over time. Because arthropods share many homologous brain structures-antennal lobes for processing olfactory information, mechanosensors for processing flow, mushroom bodies (or hemi-ellipsoid bodies) for associative learning, and central complexes for navigation, it is likely that these closely related behaviors are mediated by conserved neural circuits. However, differences in the types of odors they seek, the physics of odor dispersal, and the physics of locomotion in water, air, and on substrates mean that these circuits must have adapted to generate a wide diversity of odor-seeking behaviors. In this review, we discuss common strategies and specializations observed in olfactory navigation behavior across arthropods, and review our current knowledge about the neural circuits subserving this behavior. We propose that a comparative study of arthropod nervous systems may provide insight into how a set of basic circuit structures has diversified to generate behavior adapted to different environments.
Collapse
Affiliation(s)
- Theresa J Steele
- Neuroscience Institute, NYU School of Medicine, 435 E 30th St., New York, NY, 10016, USA
| | - Aaron J Lanz
- Neuroscience Institute, NYU School of Medicine, 435 E 30th St., New York, NY, 10016, USA
| | - Katherine I Nagel
- Neuroscience Institute, NYU School of Medicine, 435 E 30th St., New York, NY, 10016, USA.
| |
Collapse
|
9
|
Perisse E, Miranda M, Trouche S. Modulation of aversive value coding in the vertebrate and invertebrate brain. Curr Opin Neurobiol 2023; 79:102696. [PMID: 36871400 DOI: 10.1016/j.conb.2023.102696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 01/30/2023] [Accepted: 01/31/2023] [Indexed: 03/06/2023]
Abstract
Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.
Collapse
Affiliation(s)
- Emmanuel Perisse
- Institute of Functional Genomics, University of Montpellier, CNRS, Inserm, 141 rue de la Cardonille, 34094 Montpellier Cedex 5, France.
| | - Magdalena Miranda
- Institute of Functional Genomics, University of Montpellier, CNRS, Inserm, 141 rue de la Cardonille, 34094 Montpellier Cedex 5, France
| | - Stéphanie Trouche
- Institute of Functional Genomics, University of Montpellier, CNRS, Inserm, 141 rue de la Cardonille, 34094 Montpellier Cedex 5, France.
| |
Collapse
|
10
|
Yamada D, Bushey D, Li F, Hibbard KL, Sammons M, Funke J, Litwin-Kumar A, Hige T, Aso Y. Hierarchical architecture of dopaminergic circuits enables second-order conditioning in Drosophila. eLife 2023; 12:79042. [PMID: 36692262 PMCID: PMC9937650 DOI: 10.7554/elife.79042] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Accepted: 01/23/2023] [Indexed: 01/25/2023] Open
Abstract
Dopaminergic neurons with distinct projection patterns and physiological properties compose memory subsystems in a brain. However, it is poorly understood whether or how they interact during complex learning. Here, we identify a feedforward circuit formed between dopamine subsystems and show that it is essential for second-order conditioning, an ethologically important form of higher-order associative learning. The Drosophila mushroom body comprises a series of dopaminergic compartments, each of which exhibits distinct memory dynamics. We find that a slow and stable memory compartment can serve as an effective 'teacher' by instructing other faster and transient memory compartments via a single key interneuron, which we identify by connectome analysis and neurotransmitter prediction. This excitatory interneuron acquires enhanced response to reward-predicting odor after first-order conditioning and, upon activation, evokes dopamine release in the 'student' compartments. These hierarchical connections between dopamine subsystems explain distinct properties of first- and second-order memory long known by behavioral psychologists.
Collapse
Affiliation(s)
- Daichi Yamada
- Department of Biology, University of North Carolina at Chapel Hill, Chapel Hill, United States
| | - Daniel Bushey
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| | - Feng Li
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| | - Karen L Hibbard
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| | - Megan Sammons
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| | - Jan Funke
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| | - Ashok Litwin-Kumar
- Department of Neuroscience, Columbia University, New York, United States
| | - Toshihide Hige
- Department of Biology, University of North Carolina at Chapel Hill, Chapel Hill, United States
- Department of Cell Biology and Physiology, University of North Carolina at Chapel Hill, Chapel Hill, United States
- Integrative Program for Biological and Genome Sciences, University of North Carolina at Chapel Hill, Chapel Hill, United States
| | - Yoshinori Aso
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| |
Collapse
|