Trace conditioning in insects-keep the trace!

Analysis of fast calcium dynamics of honey bee olfactory coding

Odour processing exhibits multiple parallels between vertebrate and invertebrate olfactory systems. Insects, in particular, have emerged as relevant models for olfactory studies because of the tractability of their olfactory circuits. Here, we used fast calcium imaging to track the activity of projection neurons in the honey bee antennal lobe (AL) during olfactory stimulation at high temporal resolution. We observed a heterogeneity of response profiles and an abundance of inhibitory activities, resulting in various response latencies and stimulus-specific post-odour neural signatures. Recorded calcium signals were fed to a mushroom body (MB) model constructed implementing the fundamental features of connectivity between olfactory projection neurons, Kenyon cells (KC), and MB output neurons (MBON). The model accounts for the increase of odorant discrimination in the MB compared to the AL and reveals the recruitment of two distinct KC populations that represent odorants and their aftersmell as two separate but temporally coherent neural objects. Finally, we showed that the learning-induced modulation of KC-to-MBON synapses can explain both the variations in associative learning scores across different conditioning protocols used in bees and the bees' response latency. Thus, it provides a simple explanation of how the time contingency between the stimulus and the reward can be encoded without the need for time tracking. This study broadens our understanding of olfactory coding and learning in honey bees. It demonstrates that a model based on simple MB connectivity rules and fed with real physiological data can explain fundamental aspects of odour processing and associative learning.

Extrapolating animal consciousness

I argue that the question of animal consciousness is an extrapolation problem and, as such, is best tackled by deploying currently accepted methodology for validating experimental models of a phenomenon of interest. This methodology relies on an assessment of similarities and dissimilarities between experimental models, the partial replication of findings across complementary models, and evidence from the successes and failures of explanations, technologies and medical applications developed by extrapolating and aggregating findings from multiple models. Crucially important, this methodology does not require a commitment to any particular theory or construct of consciousness, thus avoiding theory-biased reinterpretations of empirical findings rampant in the literature.

A cognitive account of trace conditioning in insects

Trace conditioning is a form of Pavlovian learning in which the conditioned stimulus (CS) and the unconditioned stimulus (US) are separated by a temporal gap. Insects learn trace associations of variable nature (appetitive, aversive) and involving CSs of different sensory modalities (olfactory, visual). The accessibility of the insect neural system in behaving animals allowed identifying neural processes driving trace conditioning: the existence of prolonged neural responses to the CS after stimulus offset and the anticipation of US responses during the free-stimulus gap. Specific brain structures, such as the mushroom bodies seem to be allocated to this learning form. Here, we posit that a further component facilitating trace conditioning in insects relates to neuromodulatory mechanisms underlying enhanced attention. We thus propose a model based on different types of mushroom-body neurons, which provides a cognitive account of trace conditioning in insects.

How do real animals account for the passage of time during associative learning?

Animals routinely learn to associate environmental stimuli and self-generated actions with their outcomes such as rewards. One of the most popular theoretical models of such learning is the reinforcement learning (RL) framework. The simplest form of RL, model-free RL, is widely applied to explain animal behavior in numerous neuroscientific studies. More complex RL versions assume that animals build and store an explicit model of the world in memory. To apply these approaches to explain animal behavior, typical neuroscientific RL models make implicit assumptions about how real animals represent the passage of time. In this perspective, I explicitly list these assumptions and show that they have several problematic implications. I hope that the explicit discussion of these problems encourages the field to seriously examine the assumptions underlying timing and reinforcement learning. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

High-throughput automated methods for classical and operant conditioning of Drosophila larvae

Learning which stimuli (classical conditioning) or which actions (operant conditioning) predict rewards or punishments can improve chances of survival. However, the circuit mechanisms that underlie distinct types of associative learning are still not fully understood. Automated, high-throughput paradigms for studying different types of associative learning, combined with manipulation of specific neurons in freely behaving animals, can help advance this field. The Drosophila melanogaster larva is a tractable model system for studying the circuit basis of behaviour, but many forms of associative learning have not yet been demonstrated in this animal. Here, we developed a high-throughput (i.e. multi-larva) training system that combines real-time behaviour detection of freely moving larvae with targeted opto- and thermogenetic stimulation of tracked animals. Both stimuli are controlled in either open- or closed-loop, and delivered with high temporal and spatial precision. Using this tracker, we show for the first time that Drosophila larvae can perform classical conditioning with no overlap between sensory stimuli (i.e. trace conditioning). We also demonstrate that larvae are capable of operant conditioning by inducing a bend direction preference through optogenetic activation of reward-encoding serotonergic neurons. Our results extend the known associative learning capacities of Drosophila larvae. Our automated training rig will facilitate the study of many different forms of associative learning and the identification of the neural circuits that underpin them.

Fishnition: Developing Models From Cognition Toward Consciousness

A challenge to developing a model for testing animal consciousness is the pull of opposite intuitions. On one extreme, the anthropocentric view holds that consciousness is a highly sophisticated capacity involving self-reflection and conceptual categorization that is almost certainly exclusive to humans. At the opposite extreme, an anthropomorphic view attributes consciousness broadly to any behavior that involves sensory responsiveness. Yet human experience and observation of diverse species suggest that the most plausible case is that consciousness functions between these poles. In exploring the middle ground, we discuss the pros and cons of "high level" approaches such as the dual systems approach. According to this model, System 1 can be thought of as unconscious; processing is fast, automatic, associative, heuristic, parallel, contextual, and likely to be conserved across species. Consciousness is associated with System 2 processing that is slow, effortful, rule-based, serial, abstract, and exclusively human. An advantage of this model is the clear contrast between heuristic and decision-based responses, but it fails to include contextual decision-making in novel conditions which falls in between these two categories. We also review a "low level" model involving trace conditioning, which is a trained response to the first of two paired stimuli separated by an interval. This model highlights the role of consciousness in maintaining a stimulus representation over a temporal span, though it overlooks the importance of attention in subserving and also disrupting trace conditioning in humans. Through a critical analysis of these two extremes, we will develop the case for flexible behavioral response to the stimulus environment as the best model for demonstrating animal consciousness. We discuss a methodology for gauging flexibility across a wide variety of species and offer a case study in spatial navigation to illustrate our proposal. Flexibility serves the evolutionary function of enabling the complex evaluation of changing conditions, where motivation is the basis for goal valuation, and attention selects task-relevant stimuli to aid decision-making processes. We situate this evolutionary function within the Temporal Representation Theory of consciousness, which proposes that consciousness represents the present moment in order to facilitate flexible action.

What Behavioral Abilities Emerged at Key Milestones in Human Brain Evolution? 13 Hypotheses on the 600-Million-Year Phylogenetic History of Human Intelligence

This paper presents 13 hypotheses regarding the specific behavioral abilities that emerged at key milestones during the 600-million-year phylogenetic history from early bilaterians to extant humans. The behavioral, intellectual, and cognitive faculties of humans are complex and varied: we have abilities as diverse as map-based navigation, theory of mind, counterfactual learning, episodic memory, and language. But these faculties, which emerge from the complex human brain, are likely to have evolved from simpler prototypes in the simpler brains of our ancestors. Understanding the order in which behavioral abilities evolved can shed light on how and why our brains evolved. To propose these hypotheses, I review the available data from comparative psychology and evolutionary neuroscience.

Trace conditioning as a test for animal consciousness: a new approach

Trace conditioning involves the pairing of a neutral conditioned stimulus (CS), followed by a short interval with a motivationally significant unconditioned stimulus (UCS). Recently, trace conditioning has been proposed as a test for animal consciousness due to its correlation in humans with subjective report of the CS-UCS connection. We argue that the distractor task in the Clark and Squire (1998) study on trace conditioning has been overlooked. Attentional inhibition played a crucial role in disrupting trace conditioning and awareness of the CS-UCS contingency in the human participants of that study. These results may be understood within the framework of the Temporal Representation Theory that asserts consciousness serves the function of selecting information into a representation of the present moment. While neither sufficient nor necessary, attentional processes are the primary means to select stimuli for consciousness. Consciousness and attention are both needed by an animal capable of flexible behavioral response. Consciousness keeps track of the current situation; attention amplifies task-relevant stimuli and inhibits irrelevant stimuli. In light of these joint functions, we hypothesize that the failure to trace condition under distraction in an organism known to successfully trace condition otherwise can be one of several tests that indicates animal consciousness. Successful trace conditioning is widespread and by itself does not indicate consciousness.

A Mechanistic Model for Reward Prediction and Extinction Learning in the Fruit Fly

Extinction learning, the ability to update previously learned information by integrating novel contradictory information, is of high clinical relevance for therapeutic approaches to the modulation of maladaptive memories. Insect models have been instrumental in uncovering fundamental processes of memory formation and memory update. Recent experimental results in Drosophila melanogaster suggest that, after the behavioral extinction of a memory, two parallel but opposing memory traces coexist, residing at different sites within the mushroom body (MB). Here, we propose a minimalistic circuit model of the Drosophila MB that supports classical appetitive and aversive conditioning and memory extinction. The model is tailored to the existing anatomic data and involves two circuit motives of central functional importance. It employs plastic synaptic connections between Kenyon cells (KCs) and MB output neurons (MBONs) in separate and mutually inhibiting appetitive and aversive learning pathways. Recurrent modulation of plasticity through projections from MBONs to reinforcement-mediating dopaminergic neurons (DAN) implements a simple reward prediction mechanism. A distinct set of four MBONs encodes odor valence and predicts behavioral model output. Subjecting our model to learning and extinction protocols reproduced experimental results from recent behavioral and imaging studies. Simulating the experimental blocking of synaptic output of individual neurons or neuron groups in the model circuit confirmed experimental results and allowed formulation of testable predictions. In the temporal domain, our model achieves rapid learning with a step-like increase in the encoded odor value after a single pairing of the conditioned stimulus (CS) with a reward or punishment, facilitating single-trial learning.

Unlimited Associative Learning and the origins of consciousness: a primer and some predictions

Over the past two decades, Ginsburg and Jablonka have developed a novel approach to studying the evolutionary origins of consciousness: the Unlimited Associative Learning (UAL) framework. The central idea is that there is a distinctive type of learning that can serve as a transition marker for the evolutionary transition from non-conscious to conscious life. The goal of this paper is to stimulate discussion of the framework by providing a primer on its key claims (Part I) and a clear statement of its main empirical predictions (Part II).

Circuit and Cellular Mechanisms Facilitate the Transformation from Dense to Sparse Coding in the Insect Olfactory System

Transformations between sensory representations are shaped by neural mechanisms at the cellular and the circuit level. In the insect olfactory system, the encoding of odor information undergoes a transition from a dense spatiotemporal population code in the antennal lobe to a sparse code in the mushroom body. However, the exact mechanisms shaping odor representations and their role in sensory processing are incompletely identified. Here, we investigate the transformation from dense to sparse odor representations in a spiking model of the insect olfactory system, focusing on two ubiquitous neural mechanisms: spike frequency adaptation at the cellular level and lateral inhibition at the circuit level. We find that cellular adaptation is essential for sparse representations in time (temporal sparseness), while lateral inhibition regulates sparseness in the neuronal space (population sparseness). The interplay of both mechanisms shapes spatiotemporal odor representations, which are optimized for the discrimination of odors during stimulus onset and offset. Response pattern correlation across different stimuli showed a nonmonotonic dependence on the strength of lateral inhibition with an optimum at intermediate levels, which is explained by two counteracting mechanisms. In addition, we find that odor identity is stored on a prolonged timescale in the adaptation levels but not in the spiking activity of the principal cells of the mushroom body, providing a testable hypothesis for the location of the so-called odor trace.

Odors from phylogenetically-distant plants to Brassicaceae repel an herbivorous Brassica specialist

Specialist insect herbivores are constrained by highly specific odor recognition systems to accept suitable host plants. Given that odor recognition leads specialist insects to accept a limited range of plants, we hypothesized that phylogenetically distant plants produce odors that are physicochemically different from host odors and would be less attractive or even repellent to a specialist herbivore. We tested this hypothesis by examining behavioral and ovipositional responses of swede midge (Contarinia nasturtii, Diptera: Cecidomyiidae), a specialist of brassicas, to broccoli sprayed with non-host essential oils. Specifically, we asked: (1) How do essential oils from different plant species influence host-seeking and oviposition behaviors of swede midge? (2) Do odors from non-host plants that are not phylogenetically related or physicochemically similar to host plants affect host-seeking or ovipositional behavior of swede midge? In oviposition assays, we found that non-host odors varied in their ability to modify female midge behavior and that phylogenetic relatedness was negatively correlated with larval density. In y-tube assays, we found that female midges most frequently avoided non-host odors that were more similar to brassica odors. Females were less likely to oviposit on or choose any treated host plants, but particularly avoided garlic, spearmint, thyme, eucalyptus lemon, and cinnamon bark treatments. Overall, we found that plant phylogenetic relatedness and odor similarity are related to repellency. Therefore, altering the diversity of plant odors by explicitly accounting for plant phylogenetic distance and odor similarity, relative to host plants, may be an important, underexploited tactic for sustainably managing challenging pests.

Calcium in Kenyon Cell Somata as a Substrate for an Olfactory Sensory Memory in Drosophila

Animals can form associations between temporally separated stimuli. To do so, the nervous system has to retain a neural representation of the first stimulus until the second stimulus appears. The neural substrate of such sensory stimulus memories is unknown. Here, we search for a sensory odor memory in the insect olfactory system and characterize odorant-evoked Ca2+ activity at three consecutive layers of the olfactory system in Drosophila: in olfactory receptor neurons (ORNs) and projection neurons (PNs) in the antennal lobe, and in Kenyon cells (KCs) in the mushroom body. We show that the post-stimulus responses in ORN axons, PN dendrites, PN somata, and KC dendrites are odor-specific, but they are not predictive of the chemical identity of past olfactory stimuli. However, the post-stimulus responses in KC somata carry information about the identity of previous olfactory stimuli. These findings show that the Ca2+ dynamics in KC somata could encode a sensory memory of odorant identity and thus might serve as a basis for associations between temporally separated stimuli.

Olfactory learning without the mushroom bodies: Spiking neural network models of the honeybee lateral antennal lobe tract reveal its capacities in odour memory tasks of varied complexities

The honeybee olfactory system is a well-established model for understanding functional mechanisms of learning and memory. Olfactory stimuli are first processed in the antennal lobe, and then transferred to the mushroom body and lateral horn through dual pathways termed medial and lateral antennal lobe tracts (m-ALT and l-ALT). Recent studies reported that honeybees can perform elemental learning by associating an odour with a reward signal even after lesions in m-ALT or blocking the mushroom bodies. To test the hypothesis that the lateral pathway (l-ALT) is sufficient for elemental learning, we modelled local computation within glomeruli in antennal lobes with axons of projection neurons connecting to a decision neuron (LHN) in the lateral horn. We show that inhibitory spike-timing dependent plasticity (modelling non-associative plasticity by exposure to different stimuli) in the synapses from local neurons to projection neurons decorrelates the projection neurons' outputs. The strength of the decorrelations is regulated by global inhibitory feedback within antennal lobes to the projection neurons. By additionally modelling octopaminergic modification of synaptic plasticity among local neurons in the antennal lobes and projection neurons to LHN connections, the model can discriminate and generalize olfactory stimuli. Although positive patterning can be accounted for by the l-ALT model, negative patterning requires further processing and mushroom body circuits. Thus, our model explains several-but not all-types of associative olfactory learning and generalization by a few neural layers of odour processing in the l-ALT. As an outcome of the combination between non-associative and associative learning, the modelling approach allows us to link changes in structural organization of honeybees' antennal lobes with their behavioural performances over the course of their life.

Trace Conditioning in Drosophila Induces Associative Plasticity in Mushroom Body Kenyon Cells and Dopaminergic Neurons

Dopaminergic neurons (DANs) signal punishment and reward during associative learning. In mammals, DANs show associative plasticity that correlates with the discrepancy between predicted and actual reinforcement (prediction error) during classical conditioning. Also in insects, such as Drosophila, DANs show associative plasticity that is, however, less understood. Here, we study associative plasticity in DANs and their synaptic partners, the Kenyon cells (KCs) in the mushroom bodies (MBs), while training Drosophila to associate an odorant with a temporally separated electric shock (trace conditioning). In most MB compartments DANs strengthened their responses to the conditioned odorant relative to untrained animals. This response plasticity preserved the initial degree of similarity between the odorant- and the shock-induced spatial response patterns, which decreased in untrained animals. Contrary to DANs, KCs (α'/β'-type) decreased their responses to the conditioned odorant relative to untrained animals. We found no evidence for prediction error coding by DANs during conditioning. Rather, our data supports the hypothesis that DAN plasticity encodes conditioning-induced changes in the odorant's predictive power.

The Transition to Minimal Consciousness through the Evolution of Associative Learning

The minimal state of consciousness is sentience. This includes any phenomenal sensory experience - exteroceptive, such as vision and olfaction; interoceptive, such as pain and hunger; or proprioceptive, such as the sense of bodily position and movement. We propose unlimited associative learning (UAL) as the marker of the evolutionary transition to minimal consciousness (or sentience), its phylogenetically earliest sustainable manifestation and the driver of its evolution. We define and describe UAL at the behavioral and functional level and argue that the structural-anatomical implementations of this mode of learning in different taxa entail subjective feelings (sentience). We end with a discussion of the implications of our proposal for the distribution of consciousness in the animal kingdom, suggesting testable predictions, and revisiting the ongoing debate about the function of minimal consciousness in light of our approach.

Reversing Stimulus Timing in Visual Conditioning Leads to Memories with Opposite Valence in Drosophila

Animals need to associate different environmental stimuli with each other regardless of whether they temporally overlap or not. Drosophila melanogaster displays olfactory trace conditioning, where an odor is followed by electric shock reinforcement after a temporal gap, leading to conditioned odor avoidance. Reversing the stimulus timing in olfactory conditioning results in the reversal of memory valence such that an odor that follows shock is later on approached (i.e. relief conditioning). Here, we explored the effects of stimulus timing on memory in another sensory modality, using a visual conditioning paradigm. We found that flies form visual memories of opposite valence depending on stimulus timing and can associate a visual stimulus with reinforcement despite being presented with a temporal gap. These results suggest that associative memories with non-overlapping stimuli and the effect of stimulus timing on memory valence are shared across sensory modalities.