Classical and operant conditioning differentially modify the intrinsic properties of an identified neuron

The Role of Cerebellar Intrinsic Neuronal Excitability, Synaptic Plasticity, and Perineuronal Nets in Eyeblink Conditioning

Evidence is strong that, in addition to fine motor control, there is an important role for the cerebellum in cognition and emotion. The deep nuclei of the mammalian cerebellum also contain the highest density of perineural nets-mesh-like structures that surround neurons-in the brain, and it appears there may be a connection between these nets and cognitive processes, particularly learning and memory. Here, we review how the cerebellum is involved in eyeblink conditioning-a particularly well-understood form of learning and memory-and focus on the role of perineuronal nets in intrinsic membrane excitability and synaptic plasticity that underlie eyeblink conditioning. We explore the development and role of perineuronal nets and the in vivo and in vitro evidence that manipulations of the perineuronal net in the deep cerebellar nuclei affect eyeblink conditioning. Together, these findings provide evidence of an important role for perineuronal net in learning and memory.

Specific Plasticity Loci and Their Synergism Mediate Operant Conditioning

Despite numerous studies examining the mechanisms of operant conditioning (OC), the diversity of OC plasticity loci and their synergism have not been examined sufficiently. In the well-characterized feeding neural circuit of Aplysia, in vivo and in vitro appetitive OC increases neuronal excitability and electrical coupling among several neurons leading to an increase in expression of ingestive behavior. Here, we used the in vitro analog of OC to investigate whether OC reduces the excitability of a neuron, B4, whose inhibitory connections decrease expression of ingestive behavior. We found OC decreased the excitability of B4. This change appeared intrinsic to B4 because it could be replicated with an analog of OC in isolated cultures of B4 neurons. In addition to changes in B4 excitability, OC decreased the strength of B4's inhibitory connection to a key decision-making neuron, B51. The OC-induced changes were specific without affecting the excitability of another neuron critical for feeding behavior, B8, or the B4-to-B8 inhibitory connection. A conductance-based circuit model indicated that reducing the B4-to-B51 synapse, or increasing B51 excitability, mediated the OC phenotype more effectively than did decreasing B4 excitability. We combined these modifications to examine whether they could act synergistically. Combinations including B51 synergistically enhanced feeding. Taken together, these results suggest modifications of diverse loci work synergistically to mediate OC and that some neurons are well suited to work synergistically with plasticity in other loci.SIGNIFICANCE STATEMENT The ways in which synergism of diverse plasticity loci mediate the change in motor patterns in operant conditioning (OC) are poorly understood. Here, we found that OC was in part mediated by decreasing the intrinsic excitability of a critical neuron of Aplysia feeding behavior, and specifically reducing the strength of one of its inhibitory connections that targets a key decision-making neuron. A conductance-based computational model indicated that the known plasticity loci showed a surprising level of synergism to mediate the behavioral changes associated with OC. These results highlight the importance of understanding the diversity, specificity and synergy among different types of plasticity that encode memory. Also, because OC in Aplysia is mediated by dopamine (DA), the present study provides insights into specific and synergistic mechanisms of DA-mediated reinforcement of behaviors.

An automated homecage system for multiwhisker detection and discrimination learning in mice

Automated, homecage behavioral training for rodents has many advantages: it is low stress, requires little interaction with the experimenter, and can be easily manipulated to adapt to different experimental conditions. We have developed an inexpensive, Arduino-based, homecage training apparatus for sensory association training in freely-moving mice using multiwhisker air current stimulation coupled to a water reward. Animals learn this task readily, within 1–2 days of training, and performance progressively improves with training. We examined the parameters that regulate task acquisition using different stimulus intensities, directions, and reward valence. Learning was assessed by comparing anticipatory licking for the stimulus compared to the no-stimulus (blank) trials. At high stimulus intensities (>9 psi), animals showed markedly less participation in the task. Conversely, very weak air current intensities (1–2 psi) were not sufficient to generate rapid learning behavior. At intermediate stimulus intensities (5–6 psi), a majority of mice learned that the multiwhisker stimulus predicted the water reward after 24–48 hrs of training. Both exposure to isoflurane and lack of whiskers decreased animals’ ability to learn the task. Following training at an intermediate stimulus intensity, mice were able to transfer learning behavior when exposed to a lower stimulus intensity, an indicator of perceptual learning. Mice learned to discriminate between two directions of stimulation rapidly and accurately, even when the angular distance between the stimuli was <15 degrees. Switching the reward to a more desirable reward, aspartame, had little effect on learning trajectory. Our results show that a tactile association task in an automated homecage environment can be monitored by anticipatory licking to reveal rapid and progressive behavioral change. These Arduino-based, automated mouse cages enable high-throughput training that facilitate analysis of large numbers of genetically modified mice with targeted manipulations of neural activity.

Dopamine as a Multifunctional Neurotransmitter in Gastropod Molluscs: An Evolutionary Hypothesis

AbstractThe catecholamine 3,4-dihydroxyphenethylamine, or dopamine, acts as a neurotransmitter across a broad phylogenetic spectrum. Functions attributed to dopamine in the mammalian brain include regulation of motor circuits, valuation of sensory stimuli, and mediation of reward or reinforcement signals. Considerable evidence also supports a neurotransmitter role for dopamine in gastropod molluscs, and there is growing appreciation for its potential common functions across phylogeny. This article reviews evidence for dopamine's transmitter role in the nervous systems of gastropods. The functional properties of identified dopaminergic neurons in well-characterized neural circuits suggest a hypothetical incremental sequence by which dopamine accumulated its diverse roles. The successive acquisition of dopamine functions is proposed in the context of gastropod feeding behavior: (1) sensation of potential nutrients, (2) activation of motor circuits, (3) selection of motor patterns from multifunctional circuits, (4) valuation of sensory stimuli with reference to internal state, (5) association of motor programs with their outcomes, and (6) coincidence detection between sensory stimuli and their consequences. At each stage of this sequence, it is proposed that existing functions of dopaminergic neurons favored their recruitment to fulfill additional information processing demands. Common functions of dopamine in other intensively studied groups, ranging from mammals and insects to nematodes, suggest an ancient origin for this progression.

Rats can learn a temporal task in a single session

Fixed interval, peak interval, and temporal bisection procedures have been used to assess cognitive functions and address questions such as how animals perceive, represent, and reproduce time intervals. They have also been extensively used to test the effects of drugs on behavior, and to describe the neural correlates of interval timing. However, those procedures usually require several weeks of training for behavior to stabilize. Here, we investigated a variation of the Differential Reinforcement of Response Duration (DRRD) task with a target time of 1.2 s. We compared three types of training protocols and reported a procedure in which performance by the end of the very first session nearly matches the performance of long-term training. We also showed that the initial distribution of the responses is uni-modal and, as training evolves (and rats improve their performance), a second peak emerges and progressively shifts toward longer times. This one-day training protocol can be used to investigate temporal learning and may be especially useful to electrophysiological and neuropharmacological studies.

Changes in cerebellar intrinsic neuronal excitability and synaptic plasticity result from eyeblink conditioning

There is a long history of research documenting plasticity in the cerebellum as well as the role of the cerebellum in learning and memory. Recordings in slices of cerebellum have provided evidence of long-term depression and long-term potentiation at several excitatory and inhibitory synapses. Lesions and recordings show the cerebellum is crucial for eyeblink conditioning and it appears changes in both synaptic and membrane plasticity are involved. In addition to its role in fine motor control, there is growing consensus that the cerebellum is crucial for perceptual, cognitive, and emotional functions. In the current review, we explore the evidence that eyeblink conditioning results in significant changes in intrinsic membrane excitability as well as synaptic plasticity in Purkinje cells of the cerebellar cortex in rabbits and changes in intrinsic membrane excitability in principal neurons of the deep cerebellar nuclei in rats.

The contributions and mechanisms of changes in excitability during simple forms of learning in Aplysia

Learning and memory have long been thought to involve changes in synaptic connections between neurons. However, in many cases learning-related plasticity also involves changes in the excitability of neurons. These findings have raised questions about the relative importance of these two types of mechanisms to behavioral learning, and also about the extent to which they involve shared or unique molecular mechanisms. We have taken a reductionist approach to these questions by addressing them in a simple model organism, Aplysia californica. Studies of a semi-intact Aplysia siphon withdrawal preparation suggest that classical conditioning involves an increase in the evoked firing of sensory neurons (SNs) as well as facilitation of the monosynaptic PSP to motor neurons (MNs). Furthermore, these two mechanisms may act cooperatively at the cellular level: increased SN firing produces more PSPs, each of which is facilitated, leading to a multiplicative increase in depolarization of the MN and siphon withdrawal. The changes in SN firing and the monosynaptic PSP also share several mechanisms at the molecular level, suggesting that they may both be due in part to a decrease in K⁺ current that causes an increase in SN excitability as well as an increase in SN spike width and thus increased transmitter release. However, changes in the monosynaptic PSP also involve additional mechanisms that are not shared and may affect different aspects of synaptic transmission as well. Studies of operant conditioning of feeding suggest that it involves similar mechanisms as classical conditioning of siphon withdrawal. In particular, for both types of associative learning adenylyl cyclase appears to serve as a molecular coincidence detector that leads to increased activation of PKA and changes in excitability of key neurons in the neural circuit. Furthermore, in both cases those changes in excitability make an important contribution to the behavioral learning.

Successful and unsuccessful attempts to swallow in a reduced Aplysia preparation regulate feeding responses and produce memory at different neural sites

Sensory feedback shapes ongoing behavior and may produce learning and memory. Motor responses to edible or inedible food in a reduced Aplysia preparation were examined to test how sensory feedback affects behavior and memory. Feeding patterns were initiated by applying a cholinomimetic onto the cerebral ganglion. Feedback from buccal muscles increased the response variability and response rate. Repeated application of the cholinomimetic caused decreased responses, expressed in part by lengthening protractions. Swallowing strips of "edible" food, which in intact animals induces learning that enhances ingestion, increased the response rate, and shortened the protraction length, reflecting more swallowing. Testing memory by repeating the procedure prevented the decrease in response rate observed with the cholinomimetic alone, and shortened protractions. Training with "inedible" food that in intact animals produces learning expressed by decreased responses caused lengthened protractions. Testing memory by repeating the procedure did not cause decreased responses or lengthened protractions. After training and testing with edible or inedible food, all preparations were exposed to the cholinomimetic alone. Preparations previously trained with edible food displayed memory expressed as decreased protraction length. Preparations previously trained with inedible food showed decreases in many response parameters. Memory for inedible food may arise in part via a postsynaptic decrease in response to acetylcholine released by afferents sensing food. The lack of change in response number, and in the time that responses are maintained during the two training sessions preceding application of the cholinomimetic alone suggests that memory expression may differ from behavioral changes during training.

Role of nitric oxide in the induction of the behavioral and cellular changes produced by a common aversive stimulus in Aplysia

Although it is well documented that exposure to aversive stimuli induces modulation of neural circuits and subsequent behavioral changes, the means by which an aversive stimulus concomitantly alters behaviors of different natures (e.g., defensive and appetitive) remains unclear. Here, we addressed this issue by using the learning-induced concurrent modulation of defensive and appetitive behaviors that occurs when the mollusk Aplysia is exposed to aversive stimuli. In Aplysia, aversive stimuli concomitantly enhance withdrawal reflexes (i.e., sensitization) and suppress feeding. Sensitization and feeding suppression, which are expressed in the short term and long term, depending on the training protocol, are accompanied by increased excitability of the tail sensory neurons (TSNs) controlling the withdrawal reflexes, and by decreased excitability of feeding decision-making neuron B51, respectively. Serotonin (5-HT) has been shown to mediate sensitization, but not feeding suppression. In this study, we examined which other neurotransmitter might be responsible for feeding suppression and its underlying cellular changes. Our results indicate that nitric oxide (NO) contributes to both short-term and long-term feeding suppression, as well as to the underlying decreased B51 excitability. NO was also necessary for the induction of long-term sensitization and for the expression of short-term increased TSN excitability in vitro, revealing a previously undocumented interaction between 5-HT and NO signaling cascades in sensitization. Overall, these results revealed a scenario in which multiple modulators contribute to the widespread changes induced by sensitizing stimuli in Aplysia.

cGMP mediates short- and long-term modulation of excitability in a decision-making neuron in Aplysia

In elementary neural circuits, changes in excitability can have a strong impact in the expression of a given behavior. One example is provided by B51, a neuron with decision-making properties in the feeding neural circuit of the mollusk Aplysia. The excitability of B51 is bidirectionally modulated by external and internal stimuli in a manner that is consistent with the corresponding induced changes in feeding behavior. For example, in operant reward learning, which up-regulates feeding, B51 excitability is increased via a cAMP-dependent mechanism. Conversely, following training protocols with aversive stimuli, which down-regulate feeding, B51 excitability is decreased. In this study, we tested the hypothesis that B51 decreased excitability may be mediated by another cyclic nucleotide, cGMP. Our results revealed that iontophoretic injection of cGMP was capable of inducing both short-term (45 min) and long-term (24 h) reduction of B51 excitability. We next investigated which biochemical trigger could increase cGMP cytosolic levels. The neurotransmitter nitric oxide was found to decrease B51 excitability through the activation of the soluble guanylyl cyclase. These findings indicate that a cGMP-dependent pathway modulates B51 excitability in a manner opposite of cAMP, indicating that distinct cyclic-nucleotide pathways bidirectionally regulate the excitability of a decision-making neuron.

Effects of internal and external factors on the budgeting between defensive and non-defensive responses in Aplysia

Following exposure to aversive stimuli, organisms budget their behaviors by augmenting defensive responses and reducing/suppressing non-defensive behaviors. This budgeting process must be flexible to accommodate modifications in the animal's internal and/or external state that require the normal balance between defensive and non-defensive behaviors to be adjusted. When exposed to aversive stimuli, the mollusk Aplysia budgets its behaviors by concurrently enhancing defensive withdrawal reflexes (an elementary form of learning known as sensitization) and suppressing feeding. Sensitization and feeding suppression are consistently co-expressed following different training protocols and share common temporal domains, suggesting that they are interlocked. In this study, we attempted to uncouple the co-expression of sensitization and feeding suppression using: 1) manipulation of the animal's motivational state through prolonged food deprivation and 2) extended training with aversive stimuli that induces sensitization lasting for weeks. Both manipulations uncoupled the co-expression of the above behavioral changes. Prolonged food deprivation prevented the expression of sensitization, but not of feeding suppression. Following the extended training, sensitization and feeding suppression were co-expressed only for a limited time (i.e., 24 h), after which feeding returned to baseline levels as sensitization persisted for up to seven days. These findings indicate that sensitization and feeding suppression are not interlocked and that their co-expression can be uncoupled by internal (prolonged food deprivation) and external (extended aversive training) factors. The different strategies, by which the co-expression of sensitization and feeding suppression was altered, provide an example of how budgeting strategies triggered by an identical aversive experience can vary depending on the state of the organism.

Situating Human Sexual Conditioning

Conditioning is often thought of as a basic, automatic learning process that has limited applicability to higher-level human behavior. In addition, conditioning is seen as separable from, and even secondary to, "innate" processes. These ideas involve some misconceptions. The aim of this article is to provide a clearer, more refined sense of human sexual conditioning. After providing some background information and reviewing what is known from laboratory conditioning studies, human sexual conditioning is compared to sexual conditioning in nonhumans, to "innate" sexual responding, and to other types of human learning processes. Recommendations for moving forward in human sexual conditioning research are included.

A novel in vitro analog expressing learning-induced cellular correlates in distinct neural circuits

When presented with noxious stimuli, Aplysia exhibits concurrent sensitization of defensive responses, such as the tail-induced siphon withdrawal reflex (TSWR) and suppression of feeding. At the cellular level, sensitization of the TSWR is accompanied by an increase in the excitability of the tail sensory neurons (TSNs) that elicit the reflex, whereas feeding suppression is accompanied by decreased excitability of B51, a decision-making neuron in the feeding neural circuit. The goal of this study was to develop an in vitro analog coexpressing the above cellular correlates. We used a reduced preparation consisting of buccal, cerebral, and pleural-pedal ganglia, which contain the neural circuits controlling feeding and the TSWR, respectively. Sensitizing stimuli were delivered in vitro by electrical stimulation of afferent nerves. When trained with sensitizing stimuli, the in vitro analog expressed concomitant increased excitability in TSNs and decreased excitability in B51, which are consistent with the occurrence of sensitization and feeding suppression induced by in vivo training. This in vitro analog expressed both short-term (15 min) and long-term (24 h) excitability changes in TSNs and B51, depending on the amount of training administered. Finally, in vitro application of serotonin increased TSN excitability without altering B51 excitability, mirroring the in vivo application of the monoamine that induces sensitization, but not feeding suppression.

Use of the Aplysia feeding network to study repetition priming of an episodic behavior

Many central pattern generator (CPG)-mediated behaviors are episodic, meaning that they are not continuously ongoing; instead, there are pauses between bouts of activity. This raises an interesting possibility, that the neural networks that mediate these behaviors are not operating under "steady-state" conditions; i.e., there could be dynamic changes in motor activity as it stops and starts. Research in the feeding system of the mollusk Aplysia californica has demonstrated that this can be the case. After a pause, initial food grasping responses are relatively weak. With repetition, however, responses strengthen. In this review we describe experiments that have characterized cellular/molecular mechanisms that produce these changes in motor activity. In particular, we focus on cumulative effects of modulatory neuropeptides. Furthermore, we relate Aplysia research to work in other systems and species, and develop a hypothesis that postulates that changes in response magnitude are a reflection of an efficient feeding strategy.

Aversive Learning of Colored Lights in Walking Honeybees

The honeybee has been established as an important model organism in studies on visual learning. So far the emphasis has been on appetitive conditioning, simulating floral discrimination, and homing behavior, where bees perform exceptionally well in visual discrimination tasks. However, bees in the wild also face dangers, and recent findings suggest that what is learned about visual percepts is highly context dependent. A stimulus that follows an unpleasant period, is associated with the feeling of relief- or safety in humans and animals, thus acquiring a positive meaning. Whether this is also the case in honeybees is still an open question. Here, we conditioned bees aversively in a walking arena where each half was illuminated by light of a specific wavelength and intensity, one of which was combined with electric shocks. In this paradigm, the bees' preferences to the different lights were modified through nine conditioning trials, forming robust escape, and avoidance behaviors. Strikingly, we found that while 465 nm (human blue) and 590 nm (human yellow) lights both could acquire negative valences (inducing avoidance response), 525 nm (human green) light could not. This indicates that green light holds an innate meaning of safety which is difficult to overrule even through intensive aversive conditioning. The bees had slight initial preferences to green over the blue and the yellow lights, which could be compensated by adjusting light intensity. However, this initial bias played a minor role while the chromatic properties were the most salient characteristics of the light stimuli during aversive conditioning. Moreover, bees could learn the light signaling safety, revealing the existence of a relief component in aversive operant conditioning, similar to what has been observed in other animals.

Long-term sensitization training in Aplysia decreases the excitability of a decision-making neuron through a sodium-dependent mechanism

In Aplysia, long-term sensitization (LTS) occurs concurrently with a suppression of feeding. At the cellular level, the suppression of feeding is accompanied by decreased excitability of decision-making neuron B51. We examined the contribution of voltage-gated Na⁺ and K⁺ channels to B51 decreased excitability. In a pharmacologically isolated Na⁺ channels environment, LTS training significantly increased B51 firing threshold, compared with untrained controls. Conversely, in a pharmacologically isolated K⁺ channels environment, no differences were observed between trained and untrained animals in either amplitude or area of B51 K⁺-dependent depolarizations. These findings suggest that Na⁺ channels contribute to the decrease in B51 excitability induced by LTS training.

Inferring neuronal network functional connectivity with directed information

A major challenge in neuroscience is to develop effective tools that infer the circuit connectivity from large-scale recordings of neuronal activity patterns. In this study, context tree maximizing (CTM) was used to estimate directed information (DI), which measures causal influences among neural spike trains in order to infer putative synaptic connections. In contrast to existing methods, the method presented here is data driven and can readily identify both linear and nonlinear relations between neurons. This CTM-DI method reliably identified circuit structures underlying simulations of realistic conductance-based networks. It also inferred circuit properties from voltage-sensitive dye recordings of the buccal ganglion of Aplysia. This method can be applied to other large-scale recordings as well. It offers a systematic tool to map network connectivity and to track changes in network structure such as synaptic strengths as well as the degrees of connectivity of individual neurons, which in turn could provide insights into how modifications produced by learning are distributed in a neural network.NEW & NOTEWORTHY This study brings together the techniques of voltage-sensitive dye recording and information theory to infer the functional connectome of the feeding central pattern generating network of Aplysia. In contrast to current statistical approaches, the inference method developed in this study is data driven and validated by conductance-based model circuits, can distinguish excitatory and inhibitory connections, is robust against synaptic plasticity, and is capable of detecting network structures that mediate motor patterns.

BK Channels Are Required for Multisensory Plasticity in the Oculomotor System

Neural circuits are endowed with several forms of intrinsic and synaptic plasticity that could contribute to adaptive changes in behavior, but circuit complexities have hindered linking specific cellular mechanisms with their behavioral consequences. Eye movements generated by simple brainstem circuits provide a means for relating cellular plasticity to behavioral gain control. Here we show that firing rate potentiation, a form of intrinsic plasticity mediated by reductions in BK-type calcium-activated potassium currents in spontaneously firing neurons, is engaged during optokinetic reflex compensation for inner ear dysfunction. Vestibular loss triggers transient increases in postsynaptic excitability, occlusion of firing rate potentiation, and reductions in BK currents in vestibular nucleus neurons. Concurrently, adaptive increases in visually evoked eye movements rapidly restore oculomotor function in wild-type mice but are profoundly impaired in BK channel-null mice. Activity-dependent regulation of intrinsic excitability may be a general mechanism for adaptive control of behavioral output in multisensory circuits.

Associative learning in invertebrates

This work reviews research on neural mechanisms of two types of associative learning in the marine mollusk Aplysia, classical conditioning of the gill- and siphon-withdrawal reflex and operant conditioning of feeding behavior. Basic classical conditioning is caused in part by activity-dependent facilitation at sensory neuron-motor neuron (SN-MN) synapses and involves a hybrid combination of activity-dependent presynaptic facilitation and Hebbian potentiation, which are coordinated by trans-synaptic signaling. Classical conditioning also shows several higher-order features, which might be explained by the known circuit connections in Aplysia. Operant conditioning is caused in part by a different type of mechanism, an intrinsic increase in excitability of an identified neuron in the central pattern generator (CPG) for feeding. However, for both classical and operant conditioning, adenylyl cyclase is a molecular site of convergence of the two signals that are associated. Learning in other invertebrate preparations also involves many of the same mechanisms, which may contribute to learning in vertebrates as well.

Phylogenetic and individual variation in gastropod central pattern generators

Gastropod molluscs provide a unique opportunity to explore the neural basis of rhythmic behaviors because of the accessibility of their nervous systems and the number of species that have been examined. Detailed comparisons of the central pattern generators (CPGs) underlying rhythmic feeding and swimming behaviors highlight the presence and effects of variation in neural circuits both across and within species. The feeding motor pattern of the snail, Lymnaea, is stereotyped, whereas the feeding motor pattern in the sea hare, Aplysia, is variable. However, the Aplysia motor pattern is regularized with operant conditioning or by mimicking learning using the dynamic clamp to change properties of CPG neurons. Swimming evolved repeatedly in marine gastropods. Distinct neural mechanisms underlie dissimilar forms of swimming, with homologous neurons playing different roles. However, even similar swimming behaviors in different species can be produced by distinct neural mechanisms, resulting from different synaptic connectivity of homologous neurons. Within a species, there can be variation in the strength and even valence of synapses, which does not have functional relevance under normal conditions, but can cause some individuals to be more susceptible to lesion of the circuit. This inter- and intra-species variation provides novel insights into CPG function and plasticity.

Chronic cocaine disrupts mesocortical learning mechanisms

The addictive power of drugs of abuse such as cocaine comes from their ability to hijack natural reward and plasticity mechanisms mediated by dopamine signaling in the brain. Reward learning involves burst firing of midbrain dopamine neurons in response to rewards and cues predictive of reward. The resulting release of dopamine in terminal regions is thought to act as a teaching signaling to areas such as the prefrontal cortex and striatum. In this review, we posit that a pool of extrasynaptic dopaminergic D1-like receptors activated in response to dopamine neuron burst firing serve to enable synaptic plasticity in the prefrontal cortex in response to rewards and their cues. We propose that disruptions in these mechanisms following chronic cocaine use contribute to addiction pathology, in part due to the unique architecture of the mesocortical pathway. By blocking dopamine reuptake in the cortex, cocaine elevates dopamine signaling at these extrasynaptic receptors, prolonging D1-receptor activation and the subsequent activation of intracellular signaling cascades, and thus inducing long-lasting maladaptive plasticity. These cellular adaptations may account for many of the changes in cortical function observed in drug addicts, including an enduring vulnerability to relapse. Therefore, understanding and targeting these neuroadaptations may provide cognitive benefits and help prevent relapse in human drug addicts.

Operant conditioning: a minimal components requirement in artificial spiking neurons designed for bio-inspired robot's controller

In this paper, we investigate the operant conditioning (OC) learning process within a bio-inspired paradigm, using artificial spiking neural networks (ASNN) to act as robot brain controllers. In biological agents, OC results in behavioral changes learned from the consequences of previous actions, based on progressive prediction adjustment from rewarding or punishing signals. In a neurorobotics context, virtual and physical autonomous robots may benefit from a similar learning skill when facing unknown and unsupervised environments. In this work, we demonstrate that a simple invariant micro-circuit can sustain OC in multiple learning scenarios. The motivation for this new OC implementation model stems from the relatively complex alternatives that have been described in the computational literature and recent advances in neurobiology. Our elementary kernel includes only a few crucial neurons, synaptic links and originally from the integration of habituation and spike-timing dependent plasticity as learning rules. Using several tasks of incremental complexity, our results show that a minimal neural component set is sufficient to realize many OC procedures. Hence, with the proposed OC module, designing learning tasks with an ASNN and a bio-inspired robot context leads to simpler neural architectures for achieving complex behaviors.

Differential roles of nonsynaptic and synaptic plasticity in operant reward learning-induced compulsive behavior

BACKGROUND

Rewarding stimuli in associative learning can transform the irregularly and infrequently generated motor patterns underlying motivated behaviors into output for accelerated and stereotyped repetitive action. This transition to compulsive behavioral expression is associated with modified synaptic and membrane properties of central neurons, but establishing the causal relationships between cellular plasticity and motor adaptation has remained a challenge.

RESULTS

We found previously that changes in the intrinsic excitability and electrical synapses of identified neurons in Aplysia's central pattern-generating network for feeding are correlated with a switch to compulsive-like motor output expression induced by in vivo operant conditioning. Here, we used specific computer-simulated ionic currents in vitro to selectively replicate or suppress the membrane and synaptic plasticity resulting from this learning. In naive in vitro preparations, such experimental manipulation of neuronal membrane properties alone increased the frequency but not the regularity of feeding motor output found in preparations from operantly trained animals. On the other hand, changes in synaptic strength alone switched the regularity but not the frequency of feeding output from naive to trained states. However, simultaneously imposed changes in both membrane and synaptic properties reproduced both major aspects of the motor plasticity. Conversely, in preparations from trained animals, experimental suppression of the membrane and synaptic plasticity abolished the increase in frequency and regularity of the learned motor output expression.

CONCLUSIONS

These data establish direct causality for the contributions of distinct synaptic and nonsynaptic adaptive processes to complementary facets of a compulsive behavior resulting from operant reward learning.

Activity-dependent inhibitory gating in molecular signaling cascades induces a novel form of intermediate-term synaptic facilitation in Aplysia californica

Mechanistically distinct forms of long-lasting plasticity and memory can be induced by a variety of different training patterns. Although several studies have identified distinct molecular pathways that are engaged during these different training patterns, relatively little work has explored potential interactions between pathways when they are simultaneously engaged in the same neurons and circuits during memory formation. Aplysia californica exhibits two forms of intermediate-term synaptic facilitation (ITF) in response to two different training patterns: (1) repeated trial (RT) ITF (induced by repeated tail nerve shocks [TNSs] or repeated serotonin [5HT] application) and (2) activity-dependent (AD) ITF (induced by sensory neuron activation paired with a single TNS or 5HT pulse). RT-ITF requires PKA activation and de novo protein synthesis, while AD-ITF requires PKC activation and has no requirement for protein synthesis. Here, we explored how these distinct molecular pathways underlying ITF interact when both training patterns occur in temporal register (an "Interactive" training pattern). We found that (1) RT, AD, and Interactive training all induce ITF; (2) Interactive ITF requires PKC activity but not de novo protein synthesis; and (3), surprisingly, Interactive training blocks persistent PKA activity 1 h after training, and this block is PKC-independent. These data support the hypothesis that sensory neuron activity coincident with the last RT training trial is sufficient to convert the molecular signaling already established by RT training into an AD-like molecular phenotype.

Effect of circadian phase on memory acquisition and recall: operant conditioning vs. classical conditioning

There have been several studies on the role of circadian clocks in the regulation of associative learning and memory processes in both vertebrate and invertebrate species. The results have been quite variable and at present it is unclear to what extent the variability observed reflects species differences or differences in methodology. Previous results have shown that following differential classical conditioning in the cockroach, Rhyparobia maderae, in an olfactory discrimination task, formation of the short-term and long-term memory is under strict circadian control. In contrast, there appeared to be no circadian regulation of the ability to recall established memories. In the present study, we show that following operant conditioning of the same species in a very similar olfactory discrimination task, there is no impact of the circadian system on either short-term or long-term memory formation. On the other hand, ability to recall established memories is strongly tied to the circadian phase of training. On the basis of these data and those previously reported for phylogenetically diverse species, it is suggested that there may be fundamental differences in the way the circadian system regulates learning and memory in classical and operant conditioning.

Effects of aversive stimuli beyond defensive neural circuits: reduced excitability in an identified neuron critical for feeding in Aplysia

In Aplysia, repeated trials of aversive stimuli produce long-term sensitization (LTS) of defensive reflexes and suppression of feeding. Whereas the cellular underpinnings of LTS have been characterized, the mechanisms of feeding suppression remained unknown. Here, we report that LTS training induced a long-term decrease in the excitability of B51 (a decision-making neuron in the feeding circuit) that recovered at a time point in which LTS is no longer observed (72 h post-treatment). These findings indicate B51 as a locus of plasticity underlying feeding suppression. Finally, treatment with serotonin to induce LTS failed to alter feeding and B51 excitability, suggesting that serotonin does not mediate the effects of LTS training on the feeding circuit.

Functional organization and adaptability of a decision-making network in aplysia

Whereas major insights into the neuronal basis of adaptive behavior have been gained from the study of automatic behaviors, including reflexive and rhythmic motor acts, the neural substrates for goal-directed behaviors in which decision-making about action selection and initiation are crucial, remain poorly understood. However, the mollusk Aplysia is proving to be increasingly relevant to redressing this issue. The functional properties of the central circuits that govern this animal’s goal-directed feeding behavior and particularly the neural processes underlying the selection and initiation of specific feeding actions are becoming understood. In addition to relying on the intrinsic operation of central networks, goal-directed behaviors depend on external sensory inputs that through associative learning are able to shape decision-making strategies. Here, we will review recent findings on the functional design of the central network that generates Aplysia’s feeding-related movements and the sensory-derived plasticity that through learning can modify the selection and initiation of appropriate action. The animal’s feeding behavior and the implications of decision-making will be briefly described. The functional design of the underlying buccal network will then be used to illustrate how cellular diversity and the coordination of neuronal burst activity provide substrates for decision-making. The contribution of specific synaptic and neuronal membrane properties within the buccal circuit will also be discussed in terms of their role in motor pattern selection and initiation. The ability of learning to “rigidify” these synaptic and cellular properties so as to regularize network operation and lead to the expression of stereotyped rhythmic behavior will then be described. Finally, these aspects will be drawn into a conceptual framework of how Aplysia’s goal-directed circuitry compares to the central pattern generating networks for invertebrate rhythmic behaviors.

Nitric oxide as a regulator of behavior: new ideas from Aplysia feeding

Nitric oxide (NO) regulates Aplysia feeding by novel mechanisms, suggesting new roles for NO in controlling the behavior of higher animals. In Aplysia, (1) NO helps maintain arousal when produced by neurons responding to attempts to swallow food; (2) NO biases the motor system to reject and reposition food that resists swallowing; (3) if mechanically resistant food is not successfully swallowed, NO mediates the formation and expression of memories of food inedibility; (4) NO production at rest inhibits feeding, countering the effects of food stimuli exciting feeding. At a cellular level, NO-dependent channels contribute to the resting potential of neurons controlling food finding and food consumption. Increases in L-arginine after animals eat act as a post-feeding inhibitory signal, presumably by modulating NO production at rest. NO also signals non-feeding behaviors that are associated with feeding inhibition. Thus, depending on context, NO may enhance or inhibit feeding behavior. The different functions of NO may reflect the evolution of NO signaling from a response to tissue damage that was then elaborated and used for additional functions. These results suggest that in higher animals (1) elicited and background transmitter release may have similar effects; (2) NO may be produced by neurons without firing, influencing adjacent neurons; (3) background NO production may contribute to a neuron's resting potential; (4) circulating factors affecting background NO production may regulate spatially separated neurons; (5) L-arginine can be used to regulate neural activity; (6) L-arginine may be an effective post-ingestion metabolic signal to regulate feeding.

Rapid and persistent suppression of feeding behavior induced by sensitization training in Aplysia

In Aplysia, noxious stimuli induce sensitization of defensive responses. However, it remains largely unknown whether such stimuli also alter nondefensive behaviors. In this study, we examined the effects of noxious stimuli on feeding. Strong electric shocks, capable of inducing sensitization, also led to the suppression of feeding. The use of multiple training protocols revealed that the time course of the suppression of feeding was analogous to that of sensitization. In addition, the suppression of feeding was present only at the time points in which sensitization was expressed. These results suggest that, in Aplysia, noxious stimuli may produce concurrent changes in neural circuits controlling both defensive and nondefensive behaviors.

A theory of rate coding control by intrinsic plasticity effects

Intrinsic plasticity (IP) is a ubiquitous activity-dependent process regulating neuronal excitability and a cellular correlate of behavioral learning and neuronal homeostasis. Because IP is induced rapidly and maintained long-term, it likely represents a major determinant of adaptive collective neuronal dynamics. However, assessing the exact impact of IP has remained elusive. Indeed, it is extremely difficult disentangling the complex non-linear interaction between IP effects, by which conductance changes alter neuronal activity, and IP rules, whereby activity modifies conductance via signaling pathways. Moreover, the two major IP effects on firing rate, threshold and gain modulation, remain unknown in their very mechanisms. Here, using extensive simulations and sensitivity analysis of Hodgkin-Huxley models, we show that threshold and gain modulation are accounted for by maximal conductance plasticity of conductance that situate in two separate domains of the parameter space corresponding to sub- and supra-threshold conductance (i.e. activating below or above the spike onset threshold potential). Analyzing equivalent integrate-and-fire models, we provide formal expressions of sensitivities relating to conductance parameters, unraveling unprecedented mechanisms governing IP effects. Our results generalize to the IP of other conductance parameters and allow strong inference for calcium-gated conductance, yielding a general picture that accounts for a large repertoire of experimental observations. The expressions we provide can be combined with IP rules in rate or spiking models, offering a general framework to systematically assess the computational consequences of IP of pharmacologically identified conductance with both fine grain description and mathematical tractability. We provide an example of such IP loop model addressing the important issue of the homeostatic regulation of spontaneous discharge. Because we do not formulate any assumptions on modification rules, the present theory is also relevant to other neural processes involving excitability changes, such as neuromodulation, development, aging and neural disorders.

Over the past decades, experimental and theoretical studies of the cellular basis of learning and memory have mainly focused on synaptic plasticity, the experience-dependent modification of synapses. However, behavioral learning has also been correlated with experience-dependent changes of non-synaptic voltage-dependent ion channels. This intrinsic plasticity changes the neuron's propensity to fire action potentials in response to synaptic inputs. Thus a fundamental problem is to relate changes of the neuron input-output function with voltage-gated conductance modifications. Using a sensitivity analysis in biophysically realistic models, we depict a generic dichotomy between two classes of voltage-dependent ion channels. These two classes modify the threshold and the slope of the neuron input-output relation, allowing neurons to regulate the range of inputs they respond to and the gain of that response, respectively. We further provide analytical descriptions that enlighten the dynamical mechanisms underlying these effects and propose a concise and realistic framework for assessing the computational impact of intrinsic plasticity in neuron network models. Our results account for a large repertoire of empirical observations and may enlighten functional changes that characterize development, aging and several neural diseases, which also involve changes in voltage-dependent ion channels.

Classical conditioning analog enhanced acetylcholine responses but reduced excitability of an identified neuron

Although classical and operant conditioning are operationally distinct, it is unclear whether these two forms of learning are mechanistically distinct or similar. Feeding behavior of Aplysia provides a useful model system for addressing this issue. Both classical and operant appetitive behavioral training enhance feeding, and neuronal correlates have been identified. Behavioral training was replicated by in vitro analogs that use isolated ganglia. Moreover, a single-cell analog of operant conditioning was developed using neuron B51, a cell important for the expression of the conditioned behavior. Here, a single-cell analog of classical conditioning was developed. Acetylcholine (ACh) mediated the conditioned stimulus (CS)-elicited excitation of B51 in ganglia and mimicked the CS in the single-cell analog of classical conditioning. Pairing ACh with dopamine, which mediates the unconditioned stimulus in ganglia, decreased the excitability of B51, and increased the CS-elicited excitation of B51, similar to results following both in vivo and in vitro classical training. Finally, a D1 dopamine receptor (D1R) agonist failed to support classical conditioning in the cellular analog, whereas D1R mediates reinforcement in operant conditioning.

Neuronal activity causes rapid changes of lateral amygdala neuronal membrane properties and reduction of synaptic integration and synaptic plasticity in vivo

Neuronal membrane properties dictate neuronal responsiveness. Plasticity of membrane properties alters neuronal function and can arise in response to robust neuronal activity. Despite the potential for great impact, there is little evidence for a rapid effect of activity-dependent changes of membrane properties on many neuronal functions in vivo in mammalian brain. In this study it was tested whether periods of neuronal firing lead to a rapid change of membrane properties in neurons of a rat brain region important for some forms of learning, the lateral nucleus of the amygdala, using in vivo intracellular recordings. Our results demonstrate that rapid plasticity of membrane properties occurs in vivo, in response to action potential firing. This plasticity of membrane properties leads to changes of synaptic integration and subsequent synaptic plasticity. These changes require Ca(2+) and hyperpolarization-activated ion channels, but are NMDA independent. Furthermore, the parameters and time course of these changes would not have been predicted from most in vitro studies. The plasticity of membrane properties demonstrated here may represent a basic form of in vivo short-term plasticity that modifies neuronal function.

Neural mechanisms of operant conditioning and learning-induced behavioral plasticity in Aplysia

Associative learning in goal-directed behaviors, in contrast to reflexive behaviors, can alter processes of decision-making in the selection of appropriate action and its initiation, thereby enabling animals, including humans, to gain a predictive understanding of their external environment. In the mollusc Aplysia, recent studies on appetitive operant conditioning in which the animal learns about the positive consequences of its behavior have provided insights into this form of associative learning which, although ubiquitous, remains mechanistically poorly understood. The findings support increasing evidence that central circuit- and cell-wide sites other than chemical synaptic connections, including electrical coupling and membrane conductances controlling intrinsic neuronal excitability and underlying voltage-dependent plateauing or oscillatory mechanisms, may serve as the neural substrates for behavioral plasticity resulting from operant conditioning. Aplysia therefore continues to provide a model system for understanding learning and memory formation that enables establishing the neurobiological links between behavioral, network, and cellular levels of analysis.

PKG-mediated MAPK signaling is necessary for long-term operant memory in Aplysia

Signaling pathways necessary for memory formation, such as the mitogen-activated protein kinase (MAPK) pathway, appear highly conserved across species and paradigms. Learning that food is inedible (LFI) represents a robust form of associative, operant learning that induces short- (STM) and long-term memory (LTM) in Aplysia. We investigated the role of MAPK signaling in LFI memory in vivo. Inhibition of MAPK activation in animals prior to training blocked STM and LTM. Discontinuing MAPK signaling immediately after training inhibited LTM with no impact on STM. Therefore, MAPK signaling appears necessary early in memory formation for STM and LTM, with prolonged MAPK activity required for LTM. We found that LFI training significantly increased phospho-MAPK levels in the buccal ganglia. Increased MAPK activation was apparent immediately after training with greater than basal levels persisting for 2 h. We examined the mechanisms underlying training-induced MAPK activation and found that PKG activity was necessary for the prolonged phase of MAPK activation, but not for the early MAPK phase required for STM. Furthermore, we found that neither the immediate nor the prolonged phase of MAPK activation was dependent upon nitric oxide (NO) signaling, although expression of memory was dependent on NO as previously reported. These studies emphasize the role of MAPK and PKG in negatively reinforced operant memory and demonstrate a role for PKG-dependent MAPK signaling in invertebrate associative memory.

Nitric oxide and histamine signal attempts to swallow: A component of learning that food is inedible in Aplysia

Memory that food is inedible in Aplysia arises from training requiring three contingent events. Nitric oxide (NO) and histamine are released by a neuron responding to one of these events, attempts to swallow food. Since NO release during training is necessary for subsequent memory and NO substitutes for attempts to swallow, it was suggested that NO functions during training as a signal of attempts to swallow. However, it has been shown that NO may also be released in other contexts affecting feeding, raising the possibility that its role in learning is unrelated to signaling attempts to swallow. We confirmed that NO during learning signals attempts to swallow, by showing that a variety of behavioral effects on feeding of blocking or adding NO do not affect learning and memory that a food is inedible. In addition, histamine had effects similar to NO on learning that food is inedible, as expected if the transmitters are released together when animals attempt to swallow. Blocking histamine during training blocked long-term memory, and exogenous histamine substituted for attempts to swallow. NO also substituted for histamine during training. Histamine at concentrations relevant to learning activates neuron metacerebral cell (MCC). However, MCC activity is not a good monitor of attempts to swallow during training, since the neuron responds equally well to other stimuli. These findings support and extend the hypothesis that NO and histamine signal efforts to swallow during learning, acting on targets other than the MCC that specifically respond to attempts to swallow.

More than synaptic plasticity: role of nonsynaptic plasticity in learning and memory

Decades of research on the cellular mechanisms of memory have led to the widely held view that memories are stored as modifications of synaptic strength. These changes involve presynaptic processes, such as direct modulation of the release machinery, or postsynaptic processes, such as modulation of receptor properties. Parallel studies have revealed that memories might also be stored by nonsynaptic processes, such as modulation of voltage-dependent membrane conductances, which are expressed as changes in neuronal excitability. Although in some cases nonsynaptic changes can function as part of the engram itself, they might also serve as mechanisms through which a neural circuit is set to a permissive state to facilitate synaptic modifications that are necessary for memory storage.

Behavioral and neuronal attributes of short- and long-term habituation in the crab Chasmagnathus

Investigations using invertebrate species have led to a considerable progress in our understanding of the mechanisms underlying learning and memory. In this review we describe the main behavioral and neuronal findings obtained by studying the habituation of the escape response to a visual danger stimulus in the crab Chasmagnathus granulatus. Massed training with brief intertrial intervals lead to a rapid reduction of the escape response that recovers after a short term. Conversely, few trials of spaced training renders a slower escape reduction that endures for many days. As predicted by Wagner's associative theory of habituation, long-term habituation in the crab proved to be determined by an association between the contextual environment of the training and the unconditioned stimulus. By performing intracellular recordings in the brain of the intact animal at the same time it was learning, we identified a group of neurons that remarkably reflects the short- and long-term behavioral changes. Thus, the visual memory abilities of crabs, their relatively simple and accessible nervous system, and the recording stability that can be achieved with their neurons provide an opportunity for uncovering neurophysiological and molecular events that occur in identifiable neurons during learning.

Molecular mechanisms underlying a cellular analog of operant reward learning

Operant conditioning is a ubiquitous but mechanistically poorly understood form of associative learning in which an animal learns the consequences of its behavior. Using a single-cell analog of operant conditioning in neuron B51 of Aplysia, we examined second-messenger pathways engaged by activity and reward and how they may provide a biochemical association underlying operant learning. Conditioning was blocked by Rp-cAMP, a peptide inhibitor of PKA, a PKC inhibitor, and by expressing a dominant-negative isoform of Ca2+-dependent PKC (apl-I). Thus, both PKA and PKC were necessary for operant conditioning. Injection of cAMP into B51 mimicked the effects of operant conditioning. Activation of PKC also mimicked conditioning but was dependent on both cAMP and PKA, suggesting that PKC acted at some point upstream of PKA activation. Our results demonstrate how these molecules can interact to mediate operant conditioning in an individual neuron important for the expression of the conditioned behavior.

Double dissociation of PKC and AC manipulations on operant and classical learning in Drosophila

Learning about relationships between stimuli (i.e., classical conditioning [1]) and learning about consequences of one's own behavior (i.e., operant conditioning [2]) constitute the major part of our predictive understanding of the world. Since these forms of learning were recognized as two separate types 80 years ago [3], a recurrent concern has been the issue of whether one biological process can account for both of them [4, 5, 6, 7, 8, 9]. Today, we know the anatomical structures required for successful learning in several different paradigms, e.g., operant and classical processes can be localized to different brain regions in rodents [9] and an identified neuron in Aplysia shows opposite biophysical changes after operant and classical training, respectively [5]. We also know to some detail the molecular mechanisms underlying some forms of learning and memory consolidation. However, it is not known whether operant and classical learning can be distinguished at the molecular level. Therefore, we investigated whether genetic manipulations could differentiate between operant and classical learning in Drosophila. We found a double dissociation of protein kinase C and adenylyl cyclase on operant and classical learning. Moreover, the two learning systems interacted hierarchically such that classical predictors were learned preferentially over operant predictors.

Changes in neuronal excitability serve as a mechanism of long-term memory for operant conditioning

Learning can lead to changes in the intrinsic excitability of neurons. However, it is unclear to what extent these changes persist and what role they play in the expression of memory. Here, we report that in vitro analogues of operant conditioning produce a long-term (24 h) increase in the excitability of an identified neuron (B51) critical for the expression of feeding in Aplysia. This increase in excitability, which is cAMP dependent, contributes to the associative modification of the feeding circuitry, providing a mechanism for long-term memory storage.

Transcriptional regulation of long-term memory in the marine snail Aplysia

Whereas the induction of short-term memory involves only covalent modifications of constitutively expressed preexisting proteins, the formation of long-term memory requires gene expression, new RNA, and new protein synthesis. On the cellular level, transcriptional regulation is thought to be the starting point for a series of molecular steps necessary for both the initiation and maintenance of long-term synaptic facilitation (LTF). The core molecular features of transcriptional regulation involved in the long-term process are evolutionally conserved in Aplysia, Drosophila, and mouse, and indicate that gene regulation by the cyclic AMP response element binding protein (CREB) acting in conjunction with different combinations of transcriptional factors is critical for the expression of many forms of long-term memory. In the marine snail Aplysia, the molecular mechanisms that underlie the storage of long-term memory have been extensively studied in the monosynaptic connections between identified sensory neuron and motor neurons of the gill-withdrawal reflex. One tail shock or one pulse of serotonin (5-HT), a modulatory transmitter released by tail shocks, produces a transient facilitation mediated by the cAMP-dependent protein kinase leading to covalent modifications in the sensory neurons that results in an enhancement of transmitter release and a strengthening of synaptic connections lasting minutes. By contrast, repeated pulses of 5-hydroxytryptamine (5-HT) induce a transcription- and translation-dependent long-term facilitation (LTF) lasting more than 24 h and trigger the activation of a family of transcription factors in the presynaptic sensory neurons including ApCREB1, ApCREB2 and ApC/EBP. In addition, we have recently identified novel transcription factors that modulate the expression of ApC/EBP and also are critically involved in LTF. In this review, we examine the roles of these transcription factors during consolidation of LTF induced by different stimulation paradigms.

Training with inedible food in Aplysia causes expression of C/EBP in the buccal but not cerebral ganglion

Training with inedible food in Aplysia increased expression of the transcription factor C/EBP in the buccal ganglia, which primarily have a motor function, but not in the cerebral or pleural ganglia. C/EBP mRNA increased immediately after training, as well as 1-2 h later. The increased expression of C/EBP protein lagged the increase in mRNA. Stimulating the lips and inducing feeding responses do not lead to long-term memory and did not cause increased C/EBP expression. Blocking polyADP-ribosylation, a process necessary for long-term memory after training, did not affect the increased C/EBP mRNA expression in the buccal ganglia.