Operant conditioning in invertebrates

Termites can learn

It is generally believed that termites can't learn and are not "intelligent". This study aimed to test whether termites could have any form of memory. A Y-shaped test device with one release chamber and two identical test chambers was designed and constructed by 3D printing. A colony of damp wood termites was harvested from the wild. Worker termites were randomly selected for experiment. Repellent odors that could mimic the alarm pheromone for termites were first identified. Among all substances tested, a tea tree oil and lemon juice were found to contain repellent odors for the tested termites, as they significantly reduced the time that termites spent in the chamber treated with these substances. As control, a trail pheromone was found to be attractive. Subsequently, a second cohort of termites were operant conditioned by punishment using both tea tree oil and lemon juice, and then tested for their ability to remember the path that could lead to the repellant odors. The test device was thoroughly cleaned between trials. It was found that conditioned termites displayed a reduced tendency to choose the path that led to expectant punishment as compared with naïve termites. Thus, it is concluded that damp wood termites are capable of learning and forming "fear memory", indicative of "intelligence" in termites. This result challenges established presumption about termites' intelligence.

Associative learning in the box jellyfish Tripedalia cystophora

Associative learning, such as classical or operant conditioning, has never been unequivocally associated with animals outside bilatarians, e.g., vertebrates, arthropods, or mollusks. Learning modulates behavior and is imperative for survival in the vast majority of animals. Obstacle avoidance is one of several visually guided behaviors in the box jellyfish, Tripedalia cystophora Conant, 1897 (Cnidaria: Cubozoa), and it is intimately associated with foraging between prop roots in their mangrove habitat. The obstacle avoidance behavior (OAB) is a species-specific defense reaction (SSDR) for T. cystophora, so identifying such SSDR is essential for testing the learning capacity of a given animal. Using the OAB, we show that box jellyfish performed associative learning (operant conditioning). We found that the rhopalial nervous system is the learning center and that T. cystophora combines visual and mechanical stimuli during operant conditioning. Since T. cystophora has a dispersed central nervous system lacking a conventional centralized brain, our work challenges the notion that associative learning requires complex neuronal circuitry. Moreover, since Cnidaria is the sister group to Bilateria, it suggests the intriguing possibility that advanced neuronal processes, like operant conditioning, are a fundamental property of all nervous systems.

Repetitive nociceptive stimulation elicits complex behavioral changes in Hirudo: evidence of arousal and motivational adaptations

Appropriate responses to real or potential damaging stimuli to the body (nociception) are critical to an animal's short- and long-term survival. The initial goal of this study was to examine habituation of withdrawal reflexes (whole-body and local shortening) to repeated mechanical nociceptive stimuli (needle pokes) in the medicinal leech, Hirudo verbana, and assess whether injury altered habituation to these nociceptive stimuli. While repeated needle pokes did reduce shortening in H. verbana, a second set of behavior changes was observed. Specifically, animals began to evade subsequent stimuli by either hiding their posterior sucker underneath adjacent body segments or engaging in locomotion (crawling). Animals differed in terms of how quickly they adopted evasion behaviors during repeated stimulation, exhibiting a multi-modal distribution for early, intermediate and late evaders. Prior injury had a profound effect on this transition, decreasing the time frame in which animals began to carry out evasion and increasing the magnitude of these evasion behaviors (more locomotory evasion). The data indicate the presence in Hirudo of a complex and adaptive defensive arousal process to avoid noxious stimuli that is influenced by differences in internal states, prior experience with injury of the stimulated areas, and possibly learning-based processes.

Study of Melipona quadrifasciata brain under operant learning using proteomic and phosphoproteomic analysis

Learning to anticipate events based on the predictive relationship between an action and an outcome (operant conditioning) is a form of associative learning shared by humans and most of other living beings, including invertebrates. Several behavioral studies on the mechanisms of operant conditioning have included Melipona quadrifasciata, a honey bee that is easily manipulated due to lack of sting. In this work, brain proteomes of Melipona bees trained using operant conditioning and untrained (control) bees were compared by two-dimensional gel electrophoresis analysis within pI range of 3-10 and 4-7; in order to find proteins specifically related to this type of associative learning.One protein was detected with differential protein abundance in the brains of trained bees, when compared to not trained ones, through computational gel imaging and statistical analysis. This protein was identified by peptide mass fingerprinting and MS/MS peptide fragmentation using a MALDI-TOF/TOF mass spectrometer as one isoform of arginine kinase monomer, apparently dephosphorylated. Brain protein maps were obtained by 2-DE (Two-dimensional gel electrophoresis) from a total proteins and phosphoproteins extract of the bee Melipona quadrifasciata. One isoform of arginine kinase, probably a dephosphorylated isoform, was significantly more abundant in the brain of trained bees using operant conditioning. Arginine kinase has been reported as an important enzyme of the energy releasing process in the visual system of the bee, but it may carry out additional and unexpected functions in the bee brain for learning process.

Memristor Neural Network Circuit Based on Operant Conditioning With Immediacy and Satiety

Most of the operant conditioning only consider the basic theory, but the influencing factors such as immediacy and satiety are ignored. In this paper, a memristor neural network circuit based on operant conditioning with immediacy and satiety is proposed. The designed circuit is mainly composed of voltage control modules, synapse modules and promotion-suppression modules. The various processes of operant conditioning theory are realized. The relationship between positive reward and negative punishment is implemented through promotion-suppression modules and synapse modules. The influence of immediacy on operant conditioning is achieved through voltage control module. The effect of satiety on operant conditioning is discussed using memristive non-volatility. The study of operant conditioning may provide a reference for smarter brain-like nerves.

From Biological Synapses to “Intelligent” Robots

This selective review explores biologically inspired learning as a model for intelligent robot control and sensing technology on the basis of specific examples. Hebbian synaptic learning is discussed as a functionally relevant model for machine learning and intelligence, as explained on the basis of examples from the highly plastic biological neural networks of invertebrates and vertebrates. Its potential for adaptive learning and control without supervision, the generation of functional complexity, and control architectures based on self-organization is brought forward. Learning without prior knowledge based on excitatory and inhibitory neural mechanisms accounts for the process through which survival-relevant or task-relevant representations are either reinforced or suppressed. The basic mechanisms of unsupervised biological learning drive synaptic plasticity and adaptation for behavioral success in living brains with different levels of complexity. The insights collected here point toward the Hebbian model as a choice solution for “intelligent” robotics and sensor systems.

High-throughput automated methods for classical and operant conditioning of Drosophila larvae

Learning which stimuli (classical conditioning) or which actions (operant conditioning) predict rewards or punishments can improve chances of survival. However, the circuit mechanisms that underlie distinct types of associative learning are still not fully understood. Automated, high-throughput paradigms for studying different types of associative learning, combined with manipulation of specific neurons in freely behaving animals, can help advance this field. The Drosophila melanogaster larva is a tractable model system for studying the circuit basis of behaviour, but many forms of associative learning have not yet been demonstrated in this animal. Here, we developed a high-throughput (i.e. multi-larva) training system that combines real-time behaviour detection of freely moving larvae with targeted opto- and thermogenetic stimulation of tracked animals. Both stimuli are controlled in either open- or closed-loop, and delivered with high temporal and spatial precision. Using this tracker, we show for the first time that Drosophila larvae can perform classical conditioning with no overlap between sensory stimuli (i.e. trace conditioning). We also demonstrate that larvae are capable of operant conditioning by inducing a bend direction preference through optogenetic activation of reward-encoding serotonergic neurons. Our results extend the known associative learning capacities of Drosophila larvae. Our automated training rig will facilitate the study of many different forms of associative learning and the identification of the neural circuits that underpin them.

The Great Pond Snail (Lymnaea stagnalis) as a Model of Aging and Age-Related Memory Impairment: An Overview

With the increase of life span, normal aging and age-related memory decline are affecting an increasing number of people; however, many aspects of these processes are still not fully understood. Although vertebrate models have provided considerable insights into the molecular and electrophysiological changes associated with brain aging, invertebrates, including the widely recognized molluscan model organism, the great pond snail (Lymnaea stagnalis), have proven to be extremely useful for studying mechanisms of aging at the level of identified individual neurons and well-defined circuits. Its numerically simpler nervous system, well-characterized life cycle, and relatively long life span make it an ideal organism to study age-related changes in the nervous system. Here, we provide an overview of age-related studies on L. stagnalis and showcase this species as a contemporary choice for modeling the molecular, cellular, circuit, and behavioral mechanisms of aging and age-related memory impairment.

What can we teach Lymnaea and what can Lymnaea teach us?

This review describes the advantages of adopting a molluscan complementary model, the freshwater snail Lymnaea stagnalis, to study the neural basis of learning and memory in appetitive and avoidance classical conditioning; as well as operant conditioning of its aerial respiratory and escape behaviour. We firstly explored ‘what we can teach Lymnaea’ by discussing a variety of sensitive, solid, easily reproducible and simple behavioural tests that have been used to uncover the memory abilities of this model system. Answering this question will allow us to open new frontiers in neuroscience and behavioural research to enhance our understanding of how the nervous system mediates learning and memory. In fact, from a translational perspective, Lymnaea and its nervous system can help to understand the neural transformation pathways from behavioural output to sensory coding in more complex systems like the mammalian brain. Moving on to the second question: ‘what can Lymnaea teach us?’, it is now known that Lymnaea shares important associative learning characteristics with vertebrates, including stimulus generalization, generalization of extinction and discriminative learning, opening the possibility to use snails as animal models for neuroscience translational research.

Aversive operant conditioning alters the phototactic orientation of the marbled crayfish

Aversive learning was applied to affect the phototactic behaviour of the marbled crayfish. Animals initially showed negative phototaxis to white light and positive taxis to blue light. Using an aversive learning paradigm, we investigated the plasticity of innate behaviour following operant conditioning. The initial rate of choosing a blue-lit exit was analysed by a dual choice experiment between blue-lit and white-lit exits in pre-test conditions. During training, electrical shocks were applied to the animals when they oriented to the blue-lit exit. Memory tests were given to analyse the orientation rate to the blue-lit exit in trials 1 and 24 h after training and these rates were compared with the pre-test. In general, animals avoided the blue-lit exit in the memory tests. When training was carried out three times, the long-term memory was retained for at least 48 h, although a single bout of training was also enough to form a long-term memory. Cooling animals at 4°C or injection of cycloheximide immediately after training altered the formation of long-term memory, but had no effect on short-term memory formation. Administration of the adenylate cyclase inhibitor SQ22536, the PKA inhibitor H89 or the CREB inhibitor KG-501 immediately after training also blocked the formation of long-term memory, but had no effect on short-term memory formation. Thus, our pharmacological behavioural analyses showed that new protein synthesis was necessary to form long-term memories and that the cAMP/PKA/CREB pathway is the main signal cascade for long-term memory formation in the marbled crayfish.

How higher goals are constructed and collapse under stress: A hierarchical Bayesian control systems perspective

In this paper, we show that organisms can be modeled as hierarchical Bayesian control systems with small world and information bottleneck (bow-tie) network structure. Such systems combine hierarchical perception with hierarchical goal setting and hierarchical action control. We argue that hierarchical Bayesian control systems produce deep hierarchies of goal states, from which it follows that organisms must have some form of 'highest goals'. For all organisms, these involve internal (self) models, external (social) models and overarching (normative) models. We show that goal hierarchies tend to decompose in a top-down manner under severe and prolonged levels of stress. This produces behavior that favors short-term and self-referential goals over long term, social and/or normative goals. The collapse of goal hierarchies is universally accompanied by an increase in entropy (disorder) in control systems that can serve as an early warning sign for tipping points (disease or death of the organism). In humans, learning goal hierarchies corresponds to personality development (maturation). The failure of goal hierarchies to mature properly corresponds to personality deficits. A top-down collapse of such hierarchies under stress is identified as a common factor in all forms of episodic mental disorders (psychopathology). The paper concludes by discussing ways of testing these hypotheses empirically.

Olfactory coding in honeybees

With less than a million neurons, the western honeybee Apis mellifera is capable of complex olfactory behaviors and provides an ideal model for investigating the neurophysiology of the olfactory circuit and the basis of olfactory perception and learning. Here, we review the most fundamental aspects of honeybee's olfaction: first, we discuss which odorants dominate its environment, and how bees use them to communicate and regulate colony homeostasis; then, we describe the neuroanatomy and the neurophysiology of the olfactory circuit; finally, we explore the cellular and molecular mechanisms leading to olfactory memory formation. The vastity of histological, neurophysiological, and behavioral data collected during the last century, together with new technological advancements, including genetic tools, confirm the honeybee as an attractive research model for understanding olfactory coding and learning.

SPiDbox: design and validation of an open-source "Skinner-box" system for the study of jumping spiders

BACKGROUND

Skinner-box systems are fundamental in behavioural research. They are objective, reliable and can be used to carry out procedures otherwise impossible with manual methodologies. Recently, jumping spiders have caught the interest of scientists for their remarkable cognitive abilities. However, inquiries on their learning abilities are still few, since we lacked a proper methodology capable of overcoming the inherent difficulties that this family poses when carrying out a conditioning protocol.

NEW METHOD

In this paper, a new, automated, open-source Skinner-box, intended for the study of jumping spiders is presented. The system is 3d printable, cheap, fully open-source; is controlled with a Raspberry Pi Zero by a Python script. Since spiders are too lightweight to activate large physical object, the SPiDbox employs photo-sensors.

RESULTS

To validate the methodology, 30 Phidippus regius underwent a training procedure for a simple discrimination task to validate the effectiveness of the system. The spiders managed to learn the task, establishing the effectiveness of the SPiDbox.

COMPARISON WITH EXISTING METHODS

This automated training appears to be more reliable and effective than traditional methodologies. Moreover, its highly scalable, as many SPiDboxes could be used in parallel.

CONCLUSIONS

The SPiDbox appears to be an effective system to train jumping spiders, opening up the possibility to study learning in increasingly more complex tasks, possibly extending our understanding of jumping spiders' cognitive abilities.

Anti-instinctive Learning Behavior Revealed by Locomotion-Triggered Mild Heat Stress in Drosophila

Anti-instinctive learning, an ability to modify an animal's innate behaviors in ways that go against one's innate tendency, can confer great evolutionary advantages to animals and enable them to better adapt to the changing environment. Yet, our understanding of anti-instinctive learning and its underlying mechanisms is still limited. In this work, we describe a new anti-instinctive learning behavior of fruit flies. This learning paradigm requires the fruit fly to respond to a recurring, aversive, mild heat stress by modifying its innate locomotion behavior. We found that experiencing movement-triggered mild heat stress repeatedly significantly reduced walking activity in wild type fruit flies, indicating that fruit flies are capable of anti-instinctive learning. We also report that such learning ability is reduced in dopamine 1-like receptor 1 (Dop1R1) null mutant and dopamine 2-like receptor (Dop2R) null mutant flies, suggesting that these two dopamine receptors are involved in mediating anti-instinctive learning in flies.

An Inconvenient Truth: Some Neglected Issues in Invertebrate Learning

The burgeoning field of invertebrate behavior is moving into what was the realm of human psychology concepts. This invites comparative studies not only between invertebrate and vertebrate species but also among the diverse taxa within the invertebrates, diverse even when considering only the insects. In order to make lasting progress two issues must be addressed. The first is inconsistent use of fundamental terms defining learning. The second is a focus on similarities, giving little attention to dissimilarities. In addition, much work is needed on whether behavioral similarities are grounded in the same neuronal architecture when considering disparate phyla. These concerns identify are "inconvenient truths" that weaken comparative behavioral analysis.

A model of operant learning based on chaotically varying synaptic strength

Operant learning is learning based on reinforcement of behaviours. We propose a new hypothesis for operant learning at the single neuron level based on spontaneous fluctuations of synaptic strength caused by receptor dynamics. These fluctuations allow the neural system to explore a space of outputs. If the receptor dynamics are altered by a reinforcement signal the neural system settles to better states, i.e., to match the environmental dynamics that determine reward. Simulations show that this mechanism can support operant learning in a feed-forward neural circuit, a recurrent neural circuit, and a spiking neural circuit controlling an agent learning in a dynamic reward and punishment situation. We discuss how the new principle relates to existing learning rules and observed phenomena of short and long-term potentiation.

Operant avoidance learning in crayfish, Orconectes rusticus: Computational ethology and the development of an automated learning paradigm

Research in crustaceans offers a valuable perspective for studying the neural implementation of conserved behavioral phenomena, including motivation, escape, aggression, and drug-sensitive reward. The present work adds to this literature by demonstrating that crayfish successfully learn to respond to spatially contingent cues. An integrated video-tracking system automatically delivered a mild electric shock when a test animal entered or remained on a substrate paired with punishment. Following a few instances of shock delivery, crayfish quickly learned to avoid these areas. Comparable changes in substrate preference were not exhibited by yoked controls, but locomotion differed significantly from both pre-conditioning levels and from those of their masters receiving shock in a contingent fashion. The results of this work provide valuable insights into the principles governing avoidance learning in an invertebrate system and provide a behavioral template for exploring the neural changes during associative learning. Serving as a case study, this project introduces a new computer framework for the automated control of learning paradigms. Based on routines contained within the JavaGrinders library (free download at iEthology.com), it integrates real-time video tracking with robotic interfaces, and provides a suitable framework for implementing automated learning paradigms.

cAMP signaling mediates behavioral flexibility and consolidation of social status in Drosophila aggression

Social rituals, such as male-male aggression in Drosophila, are often stereotyped and the component behavioral patterns modular. The likelihood of transition from one behavioral pattern to another is malleable by experience and confers flexibility to the behavioral repertoire. Experience-dependent modification of innate aggressive behavior in flies alters fighting strategies during fights and establishes dominant-subordinate relationships. Dominance hierarchies resulting from agonistic encounters are consolidated to longer-lasting, social-status-dependent behavioral modifications, resulting in a robust loser effect. We showed that cAMP dynamics regulated by the calcium-calmodulin-dependent adenylyl cyclase, Rut, and the cAMP phosphodiesterase, Dnc, but not the Amn gene product, in specific neuronal groups of the mushroom body and central complex, mediate behavioral plasticity necessary to establish dominant-subordinate relationships. rut and dnc mutant flies were unable to alter fighting strategies and establish dominance relationships during agonistic interactions. This real-time flexibility during a fight was independent of changes in aggression levels. Longer-term consolidation of social status in the form of a loser effect, however, required additional Amn-dependent inputs to cAMP signaling and involved a circuit-level association between the α/β and γ neurons of the mushroom body. Our findings implicate cAMP signaling in mediating the plasticity of behavioral patterns in aggressive behavior and in the generation of a temporally stable memory trace that manifests as a loser effect.

Punish the thief-coevolution of defense and cautiousness stabilizes ownership

Abstract

Ownership of non-controllable resources usually has to be maintained by costly defense against competitors. Whether defense and thus ownership pays in terms of fitness depends on its effectiveness in preventing theft. We show that if the owners’ willingness to defend varies in the population and information about it is available to potential thieves then the ability to react to this information and thus avoid being attacked by the owner is selected for. This can lead to a positive evolutionary feedback between cautiousness in intruders and aggressiveness in owners. This feedback can maintain ownership when the actual direct effectiveness of defense in reducing theft is very low or even absent, effectively turning defense into punishment. We conclude that the deterrence effect of defense in many situations could be stronger than that of prevention and that for many real-world scenarios the purpose of defense of resources might be to punish rather than to drive away intruders.

Significance statement

Many animals defend resources against conspecifics. Resource defense can usually only evolve if its costs are paid for by foiling attempts at theft. We show that if potential thieves can detect differences in aggressiveness between owners then cautious intruders and aggressive owners coevolve so that in the end even ineffective defense deters thieves and maintains ownership. This result greatly extends the number of situations in which we expect resource defense to evolve and has the potential to unify the concepts of defense and punishment.

Operant Conditioning in Honey Bees (Apis mellifera L.): The Cap Pushing Response

The honey bee has been an important model organism for studying learning and memory. More recently, the honey bee has become a valuable model to understand perception and cognition. However, the techniques used to explore psychological phenomena in honey bees have been limited to only a few primary methodologies such as the proboscis extension reflex, sting extension reflex, and free flying target discrimination-tasks. Methods to explore operant conditioning in bees and other invertebrates are not as varied as with vertebrates. This may be due to the availability of a suitable response requirement. In this manuscript we offer a new method to explore operant conditioning in honey bees: the cap pushing response (CPR). We used the CPR to test for difference in learning curves between novel auto-shaping and more traditional explicit-shaping. The CPR protocol requires bees to exhibit a novel behavior by pushing a cap to uncover a food source. Using the CPR protocol we tested the effects of both explicit-shaping and auto-shaping techniques on operant conditioning. The goodness of fit and lack of fit of these data to the Rescorla-Wagner learning-curve model, widely used in classical conditioning studies, was tested. The model fit well to both control and explicit-shaping results, but only for a limited number of trials. Learning ceased rather than continuing to asymptotically approach the physiological most accurate possible. Rate of learning differed between shaped and control bee treatments. Learning rate was about 3 times faster for shaped bees, but for all measures of proficiency control and shaped bees reached the same level. Auto-shaped bees showed one-trial learning rather than the asymptotic approach to a maximal efficiency. However, in terms of return-time, the auto-shaped bees’ learning did not carry over to the covered-well test treatments.

Short-term plasticity as cause-effect hypothesis testing in distal reward learning

Asynchrony, overlaps, and delays in sensory-motor signals introduce ambiguity as to which stimuli, actions, and rewards are causally related. Only the repetition of reward episodes helps distinguish true cause-effect relationships from coincidental occurrences. In the model proposed here, a novel plasticity rule employs short- and long-term changes to evaluate hypotheses on cause-effect relationships. Transient weights represent hypotheses that are consolidated in long-term memory only when they consistently predict or cause future rewards. The main objective of the model is to preserve existing network topologies when learning with ambiguous information flows. Learning is also improved by biasing the exploration of the stimulus-response space toward actions that in the past occurred before rewards. The model indicates under which conditions beliefs can be consolidated in long-term memory, it suggests a solution to the plasticity-stability dilemma, and proposes an interpretation of the role of short-term plasticity.

Phospholipase A2 - nexus of aging, oxidative stress, neuronal excitability, and functional decline of the aging nervous system? Insights from a snail model system of neuronal aging and age-associated memory impairment

The aging brain undergoes a range of changes varying from subtle structural and physiological changes causing only minor functional decline under healthy normal aging conditions, to severe cognitive or neurological impairment associated with extensive loss of neurons and circuits due to age-associated neurodegenerative disease conditions. Understanding how biological aging processes affect the brain and how they contribute to the onset and progress of age-associated neurodegenerative diseases is a core research goal in contemporary neuroscience. This review focuses on the idea that changes in intrinsic neuronal electrical excitability associated with (per)oxidation of membrane lipids and activation of phospholipase A₂ (PLA₂) enzymes are an important mechanism of learning and memory failure under normal aging conditions. Specifically, in the context of this special issue on the biology of cognitive aging we portray the opportunities offered by the identifiable neurons and behaviorally characterized neural circuits of the freshwater snail Lymnaea stagnalis in neuronal aging research and recapitulate recent insights indicating a key role of lipid peroxidation-induced PLA₂ as instruments of aging, oxidative stress and inflammation in age-associated neuronal and memory impairment in this model system. The findings are discussed in view of accumulating evidence suggesting involvement of analogous mechanisms in the etiology of age-associated dysfunction and disease of the human and mammalian brain.

Operant conditioning: a minimal components requirement in artificial spiking neurons designed for bio-inspired robot's controller

In this paper, we investigate the operant conditioning (OC) learning process within a bio-inspired paradigm, using artificial spiking neural networks (ASNN) to act as robot brain controllers. In biological agents, OC results in behavioral changes learned from the consequences of previous actions, based on progressive prediction adjustment from rewarding or punishing signals. In a neurorobotics context, virtual and physical autonomous robots may benefit from a similar learning skill when facing unknown and unsupervised environments. In this work, we demonstrate that a simple invariant micro-circuit can sustain OC in multiple learning scenarios. The motivation for this new OC implementation model stems from the relatively complex alternatives that have been described in the computational literature and recent advances in neurobiology. Our elementary kernel includes only a few crucial neurons, synaptic links and originally from the integration of habituation and spike-timing dependent plasticity as learning rules. Using several tasks of incremental complexity, our results show that a minimal neural component set is sufficient to realize many OC procedures. Hence, with the proposed OC module, designing learning tasks with an ASNN and a bio-inspired robot context leads to simpler neural architectures for achieving complex behaviors.

Invertebrate models in addiction research

While drug addiction is a uniquely human problem, most research examining the biological mechanisms of the transition from substance use to addiction is conducted with vertebrate animal models. Many other fields of neuroscience have greatly benefitted from contributions from simple and manipulable invertebrate model systems. However, the potential of invertebrate research has yet to be fully capitalised on in the field of addiction neuroscience. This may be because of the complexity of addiction and the clinical imperative of addiction research. We argue that the homocentric diagnostic criteria of addiction are no more a hindrance to the use of invertebrate models than they are to vertebrate models. We highlight the strengths of the diversity of different invertebrate model systems in terms of neuroanatomy and molecular machinery, and stress that working with a range of different models will aid in understanding addiction and not be a disadvantage. Finally, we discuss the specific advantages of utilising invertebrate animals for addiction research and highlight key areas in which invertebrates are suited for making unique and meaningful contributions to this field.

Invertebrate learning and cognition: relating phenomena to neural substrate

Diverse invertebrate species have been used for studies of learning and comparative cognition. Although we have gained invaluable information from this, in this study we argue that our approach to comparative learning research is rather deficient. Generally invertebrate learning research has focused mainly on arthropods, and most of that within the Hymenoptera and Diptera. Any true comparative analysis of the distribution of comparative cognitive abilities across phyla is hampered by this bias, and more fundamentally by a reporting bias toward positive results. To understand the limits of learning and cognition for a species, knowing what animals cannot do is at least as important as reporting what they can. Finally, much more effort needs to be focused on the neurobiological analysis of different types of learning to truly understand the differences and similarities of learning types. In this review, we first give a brief overview of the various forms of learning in invertebrates. We also suggest areas where further study is needed for a more comparative understanding of learning. Finally, using what is known of learning in honeybees and the well-studied honeybee brain, we present a model of how various complex forms of learning may be accounted for with the same neural circuitry required for so-called simple learning types. At the neurobiological level, different learning phenomena are unlikely to be independent, and without considering this it is very difficult to correctly interpret the phylogenetic distribution of learning and cognitive abilities. WIREs Cogn Sci 2013, 4:561-582. doi: 10.1002/wcs.1248 For further resources related to this article, please visit the WIREs website.

APIS-a novel approach for conditioning honey bees

Honey bees perform robustly in different conditioning paradigms. This makes them excellent candidates for studying mechanisms of learning and memory at both an individual and a population level. Here we introduce a novel method of honey bee conditioning: APIS, the Automatic Performance Index System. In an enclosed walking arena where the interior is covered with an electric grid, presentation of odors from either end can be combined with weak electric shocks to form aversive associations. To quantify behavioral responses, we continuously monitor the movement of the bee by an automatic tracking system. We found that escapes from one side to the other, changes in velocity as well as distance and time spent away from the punished odor are suitable parameters to describe the bee's learning capabilities. Our data show that in a short-term memory test the response rate for the conditioned stimulus (CS) in APIS correlates well with response rate obtained from conventional Proboscis Extension Response (PER)-conditioning. Additionally, we discovered that bees modulate their behavior to aversively learned odors by reducing their rate, speed and magnitude of escapes and that both generalization and extinction seem to be different between appetitive and aversive stimuli. The advantages of this automatic system make it ideal for assessing learning rates in a standardized and convenient way, and its flexibility adds to the toolbox for studying honey bee behavior.

Reinforcement learning and Tourette syndrome

In this chapter, we report the first experimental explorations of reinforcement learning in Tourette syndrome, realized by our team in the last few years. This report will be preceded by an introduction aimed to provide the reader with the state of the art of the knowledge concerning the neural bases of reinforcement learning at the moment of these studies and the scientific rationale beyond them. In short, reinforcement learning is learning by trial and error to maximize rewards and minimize punishments. This decision-making and learning process implicates the dopaminergic system projecting to the frontal cortex-basal ganglia circuits. A large body of evidence suggests that the dysfunction of the same neural systems is implicated in the pathophysiology of Tourette syndrome. Our results show that Tourette condition, as well as the most common pharmacological treatments (dopamine antagonists), affects reinforcement learning performance in these patients. Specifically, the results suggest a deficit in negative reinforcement learning, possibly underpinned by a functional hyperdopaminergia, which could explain the persistence of tics, despite their evident inadaptive (negative) value. This idea, together with the implications of these results in Tourette therapy and the future perspectives, is discussed in Section 4 of this chapter.

Drosophila as a tool for studying the conserved genetics of pain

Survival of all animals depends on an accurate representation of the world, and an organism must be capable of prioritizing and responding to potentially hazardous conditions. This ability is dependent on nociception, the sensory process allowing animals to detect and avoid potentially harmful stimuli. Nociception is the sensory process that results in the subjective experience of 'pain' in humans. Because of its vital and broad role in animal biology, pain/nociception is a complex, whole-body physiological process that is under stringent evolutionary pressure. Here, we discuss the utility of Drosophila melanogaster as an emerging model organism for studying the conserved genetics of nociception, particularly with respect to recently developed high-throughput Drosophila 'pain' paradigms.

Functional organization and adaptability of a decision-making network in aplysia

Whereas major insights into the neuronal basis of adaptive behavior have been gained from the study of automatic behaviors, including reflexive and rhythmic motor acts, the neural substrates for goal-directed behaviors in which decision-making about action selection and initiation are crucial, remain poorly understood. However, the mollusk Aplysia is proving to be increasingly relevant to redressing this issue. The functional properties of the central circuits that govern this animal’s goal-directed feeding behavior and particularly the neural processes underlying the selection and initiation of specific feeding actions are becoming understood. In addition to relying on the intrinsic operation of central networks, goal-directed behaviors depend on external sensory inputs that through associative learning are able to shape decision-making strategies. Here, we will review recent findings on the functional design of the central network that generates Aplysia’s feeding-related movements and the sensory-derived plasticity that through learning can modify the selection and initiation of appropriate action. The animal’s feeding behavior and the implications of decision-making will be briefly described. The functional design of the underlying buccal network will then be used to illustrate how cellular diversity and the coordination of neuronal burst activity provide substrates for decision-making. The contribution of specific synaptic and neuronal membrane properties within the buccal circuit will also be discussed in terms of their role in motor pattern selection and initiation. The ability of learning to “rigidify” these synaptic and cellular properties so as to regularize network operation and lead to the expression of stereotyped rhythmic behavior will then be described. Finally, these aspects will be drawn into a conceptual framework of how Aplysia’s goal-directed circuitry compares to the central pattern generating networks for invertebrate rhythmic behaviors.

How stress alters memory in 'smart' snails

Cognitive ability varies within species, but whether this variation alters the manner in which memory formation is affected by environmental stress is unclear. The great pond snail, Lymnaea stagnalis, is commonly used as model species in studies of learning and memory. The majority of those studies used a single laboratory strain (i.e. the Dutch strain) originating from a wild population in the Netherlands. However, our recent work has identified natural populations that demonstrate significantly enhanced long-term memory (LTM) formation relative to the Dutch strain following operant conditioning of aerial respiratory behaviour. Here we assess how two populations with enhanced memory formation (i.e. ‘smart’ snails), one from Canada (Trans Canada 1: TC1) and one from the U.K. (Chilton Moor: CM) respond to ecologically relevant stressors. In control conditions the Dutch strain forms memory lasting 1–3 h following a single 0.5 h training session in our standard calcium pond water (80 mg/l [Ca²⁺]), whereas the TC1 and CM populations formed LTM lasting 5+ days following this training regime. Exposure to low environmental calcium pond water (20 mg/l [Ca²⁺]), which blocks LTM in the Dutch strain, reduced LTM retention to 24 h in the TC1 and CM populations. Crowding (20 snails in 100 ml) immediately prior to training blocks LTM in the Dutch strain, and also did so in TC1 and CM populations. Therefore, snails with enhanced cognitive ability respond to these ecologically relevant stressors in a similar manner to the Dutch strain, but are more robust at forming LTM in a low calcium environment. Despite the two populations (CM and TC1) originating from different continents, LTM formation was indistinguishable in both control and stressed conditions. This indicates that the underlying mechanisms controlling cognitive differences among populations may be highly conserved in L. stagnalis.

Neural mechanisms of operant conditioning and learning-induced behavioral plasticity in Aplysia

Associative learning in goal-directed behaviors, in contrast to reflexive behaviors, can alter processes of decision-making in the selection of appropriate action and its initiation, thereby enabling animals, including humans, to gain a predictive understanding of their external environment. In the mollusc Aplysia, recent studies on appetitive operant conditioning in which the animal learns about the positive consequences of its behavior have provided insights into this form of associative learning which, although ubiquitous, remains mechanistically poorly understood. The findings support increasing evidence that central circuit- and cell-wide sites other than chemical synaptic connections, including electrical coupling and membrane conductances controlling intrinsic neuronal excitability and underlying voltage-dependent plateauing or oscillatory mechanisms, may serve as the neural substrates for behavioral plasticity resulting from operant conditioning. Aplysia therefore continues to provide a model system for understanding learning and memory formation that enables establishing the neurobiological links between behavioral, network, and cellular levels of analysis.

Low environmental calcium blocks long-term memory formation in a freshwater pulmonate snail

The freshwater snail Lymnaea stagnalis (L.) is considered a calciphile and exhibits reduced growth and survival in environments containing less than 20 mg/l environmental calcium. Although it has no apparent effect on survival at 20 mg/l, reducing environmental calcium increases metabolic demand, and as such we consider that this level of calcium acts as a stressor on the snail. We exposed snails to acute periods of low environmental calcium and tested their ability to form intermediate-term memory (ITM) and long-term memory (LTM) following one trial operant conditioning (1TT) to reduce aerial respiratory activity in hypoxic conditions. We also assessed whether there were changes in the electrophysiological properties of a single neuron, right pedal dorsal 1 (RPeD1), which has been demonstrated to be necessary for LTM formation. Following training in high (80 mg/l) environmental calcium, L. stagnalis formed ITM and LTM lasting 24 h and demonstrated a significant reduction in all activity measured from RPeD1; however when snails were exposed to low (20 mg/l) environmental calcium they were able to form ITM but not LTM. Although no behavioral LTM was formed, a partial reduction in RPeD1 activtiy measured 24 h after training was observed, indicating a residual effect of training. The strong effect that environmental calcium concentration had on physiology and behavior in response to training to reduce aerial respiration in L. stagnalis suggests that it is an element of gastropod husbandry that needs to be carefully considered when studying other traits. This study also indicates that L. stagnalis found naturally in low calcium environments may be less able to adapt to novel stressors than populations found in harder waters.

A paradigm for operant conditioning in blow flies (Phormia terrae novae Robineau-Desvoidy, 1830)

An operant conditioning situation for the blow fly (Protophormia terrae novae) is described. Individual flies are trained to enter and reenter a hole as the operant response. Only a few sessions of contingent reinforcement are required to increase response rates. When the response is no longer followed by food, the rate of entering the hole decreases. Control procedures revealed that rate of responding is not a simple overall result of feeding or of aging. The flies entered into the hole only if the response was required to obtain the food.

Novel neural correlates of operant conditioning in normal and differentially reared Lymnaea

The aerial respiratory behaviour of the mollusc Lymnaea stagnalisis an important homeostatic behaviour that can be operantly conditioned. The central pattern generator underlying this behaviour, as well as motorneurons innervating the respiratory orifice, the pneumostome, have been identified and their activity can be monitored in the semi-intact preparation using electrophysiological recordings. In this study, we used both intact animals and semi-intact preparations to identify novel changes in the respiratory central pattern generator following operant conditioning. In addition, we reared animals in the absence of this respiratory behaviour throughout development, to investigate whether previous experience and activity-dependent plasticity during development are essential to allow neural plasticity in the adult. We found that animals raised normally (allowed to perform aerial respiratory behaviour) exhibited the expected reduction in aerial respiratory behaviour following operant conditioning. Then, using the semi-intact preparation, we identified novel neural changes within the network as a result of the conditioning. These included specific changes at the level of the central pattern generator interneurons, as well as the motor output. In the differentially reared intact animals, there was no behavioural reduction as a result of operant conditioning, although their baseline respiratory behaviour was already significantly reduced as a result of their differential rearing. There were, however, significant differences found in the network parameters in the semi-intact preparation, similar to those observed in normally reared animals. We thus provide evidence for neural plasticity within the network in the absence of significant behavioural changes in differentially reared animals, and show that plasticity was not dependent on previous activity of the network during development.

There are many ways to train a fly

A biological understanding of memory remains one of the great quests of neuroscience. For over 30 years the fruit fly Drosophila melanogaster has primarily been viewed as an excellent vehicle to find 'memory genes'. However, the recent advent of sophisticated genetic tools to manipulate neural activity has meant that these genes can now be viewed within the context of functioning neural circuits. A holistic understanding of memory in flies is therefore now a realistic goal. Larvae and adult flies exhibit remarkable behavioral complexity and they can both be trained in a number of ways. In this review, our intention is to summarize the many assays that have been developed to study plastic behaviors in flies. More specific and detailed reviews have been published by us and others, reviewed in references 1-6. While our bias for olfactory conditioning paradigms is obvious, our purpose here is not to pass judgment on each method. We would rather leave that to those readers who might be inspired to try each assay for themselves.

The perception of stress alters adaptive behaviours in Lymnaea stagnalis

Stress can alter adaptive behaviours, and as well either enhance or diminish learning, memory formation and/or memory recall. We show here that two different stressors have the ability to alter such behaviours in our model system, Lymnaea stagnalis. One, a naturally occurring stressor - the scent of a predator (crayfish) - and the other an artificially controlled one - 25 mmol l(-1) KCl - significantly alter adaptive behaviours. Both the KCl stressor and predator detection enhance long-term memory (LTM) formation; additionally predator detection alters vigilance behaviours. The predator-induced changes in behaviour are also accompanied by specific and significant alterations in the electrophysiological properties of RPeD1 - a key neuron in mediating both vigilance behaviours and memory formation. Naive lab-bred snails exposed to crayfish effluent (CE; i.e. the scent of the predator) prior to recording from RPeD1 demonstrated both a significantly reduced spontaneous firing rate and fewer bouts of bursting activity compared with non-exposed snails. Importantly, in the CE experiments we used laboratory-reared snails that have not been exposed to a naturally occurring predator for over 250 generations. These data open a new avenue of research, which may allow a direct investigation from the behavioral to the neuronal level as to how relevant stressful stimuli alter adaptive behaviours, including memory formation and recall.

Cross-modal effects on learning: a seismic stimulus improves color discrimination learning in a jumping spider

The production of multimodal signals during animal displays is extremely common, and the function of such complex signaling has received much attention. Currently, the most frequently explored hypotheses regarding the evolution and function of complex signaling focus on the signal and/or signaler, or the signaling environment, while much less attention has been placed on the receivers. However, recent studies using vertebrates suggest that receiver psychology (e.g. learning and memory) may play a large role in the evolution of complex signaling. To date, the influence of multimodal cues on receiver learning and/or memory has not been studied in invertebrates. Here, we test the hypothesis that the presence of a seismic (vibratory) stimulus improves color discrimination learning in the jumping spider Habronattus dossenus. Using a heat-aversion learning experiment, we found evidence for a cross-modal effect on color learning. Over a series of training trials, individuals exposed to a seismic stimulus jumped onto the heated color less frequently and remained there for less time than did individuals not exposed to a seismic stimulus. In addition, in a final no-heat test trial, individuals from the seismic-present treatment were more likely to avoid the previously heated color than were individuals from the seismic-absent treatment. This is the first study to demonstrate a cross-modal influence on learning in an invertebrate.

Classical reward conditioning in Drosophila melanogaster

Negatively reinforced olfactory conditioning has been widely employed to identify learning and memory genes, signal transduction pathways and neural circuitry in Drosophila. To delineate the molecular and cellular processes underlying reward-mediated learning and memory, we developed a novel assay system for positively reinforced olfactory conditioning. In this assay, flies were involuntarily exposed to the appetitive unconditioned stimulus sucrose along with a conditioned stimulus odour during training and their preference for the odour previously associated with sucrose was measured to assess learning and memory capacities. After one training session, wild-type Canton S flies displayed reliable performance, which was enhanced after two training cycles with 1-min or 15-min inter-training intervals. Higher performance scores were also obtained with increasing sucrose concentration. Memory in Canton S flies decayed slowly when measured at 30 min, 1 h and 3 h after training; whereas, it had declined significantly at 6 h and 12 h post-training. When learning mutant t beta h flies, which are deficient in octopamine, were challenged, they exhibited poor performance, validating the utility of this assay. As the Drosophila model offers vast genetic and transgenic resources, the new appetitive conditioning described here provides a useful tool with which to elucidate the molecular and cellular underpinnings of reward learning and memory. Similar to negatively reinforced conditioning, this reward conditioning represents classical olfactory conditioning. Thus, comparative analyses of learning and memory mutants in two assays may help identify the molecular and cellular components that are specific to the unconditioned stimulus information used in conditioning.

Feeding behavior of Aplysia: a model system for comparing cellular mechanisms of classical and operant conditioning

Feeding behavior of Aplysia provides an excellent model system for analyzing and comparing mechanisms underlying appetitive classical conditioning and reward operant conditioning. Behavioral protocols have been developed for both forms of associative learning, both of which increase the occurrence of biting following training. Because the neural circuitry that mediates the behavior is well characterized and amenable to detailed cellular analyses, substantial progress has been made toward a comparative analysis of the cellular mechanisms underlying these two forms of associative learning. Both forms of associative learning use the same reinforcement pathway (the esophageal nerve, En) and the same reinforcement transmitter (dopamine, DA). In addition, at least one cellular locus of plasticity (cell B51) is modified by both forms of associative learning. However, the two forms of associative learning have opposite effects on B51. Classical conditioning decreases the excitability of B51, whereas operant conditioning increases the excitability of B51. Thus, the approach of using two forms of associative learning to modify a single behavior, which is mediated by an analytically tractable neural circuit, is revealing similarities and differences in the mechanisms that underlie classical and operant conditioning.

Operant conditioning of gill withdrawal in Aplysia

A basic question in neuroscience is how different forms of learning are related. To further address that question, we examined whether gill withdrawal in Aplysia, which has already been studied extensively for neuronal mechanisms contributing to habituation, sensitization, and classical conditioning, also undergoes operant conditioning. Animals were run in pairs. During the initial training period, the contingent (experimental) animal received a siphon shock each time its gill relaxed below a criterion level, and the yoked control animal received a shock whenever the experimental animal did, regardless of its own gill position. This was followed by an extinction period when there was no shock, a retraining period when both animals were contingent, and another extinction period. The experimental animals spent more time with their gills contracted above the criterion level than did the control animals during each period, demonstrating operant conditioning. The type of gill behavior modified by learning shifted over time: the experimental animals had a larger increase in the frequency and duration of spontaneous contractions than did the control animals during the first but not the last extinction period and a larger increase in the level of tonic contraction during the last but not the first extinction period. Because many of the neurons controlling spontaneous and tonic gill withdrawal have already been identified, it should now be possible to examine the cellular locus and mechanism of operant conditioning and compare them with those for other forms of learning of the same behavior.