Cycle-to-cycle variability as an optimal behavioral strategy

Modulation by Neuropeptides with Overlapping Targets Results in Functional Overlap in Oscillatory Circuit Activation

Neuromodulation lends flexibility to neural circuit operation but the general notion that different neuromodulators sculpt neural circuit activity into distinct and characteristic patterns is complicated by interindividual variability. In addition, some neuromodulators converge onto the same signaling pathways, with similar effects on neurons and synapses. We compared the effects of three neuropeptides on the rhythmic pyloric circuit in the stomatogastric ganglion of male crabs, Cancer borealis Proctolin (PROC), crustacean cardioactive peptide (CCAP), and red pigment concentrating hormone (RPCH) activate the same modulatory inward current, I MI, and have convergent actions on synapses. However, while PROC targets all four neuron types in the core pyloric circuit, CCAP and RPCH target the same subset of only two neurons. After removal of spontaneous neuromodulator release, none of the neuropeptides restored the control cycle frequency, but all restored the relative timing between neuron types. Consequently, differences between neuropeptide effects were mainly found in the spiking activity of different neuron types. We performed statistical comparisons using the Euclidean distance in the multidimensional space of normalized output attributes to obtain a single measure of difference between modulatory states. Across preparations, the circuit output in PROC was distinguishable from CCAP and RPCH, but CCAP and RPCH were not distinguishable from each other. However, we argue that even between PROC and the other two neuropeptides, population data overlapped enough to prevent reliable identification of individual output patterns as characteristic for a specific neuropeptide. We confirmed this notion by showing that blind classifications by machine learning algorithms were only moderately successful.Significance Statement It is commonly assumed that distinct behaviors or circuit activities can be elicited by different neuromodulators. Yet it is unknown to what extent these characteristic actions remain distinct across individuals. We use a well-studied circuit model of neuromodulation to examine the effects of three neuropeptides, each known to produce a distinct activity pattern in controlled studies. We find that, when compared across individuals, the three peptides elicit activity patterns that are either statistically indistinguishable or show too much overlap to be labeled characteristic. We ascribe this to interindividual variability and overlapping subcellular actions of the modulators. Because both factors are common in all neural circuits, these findings have broad significance for understanding chemical neuromodulatory actions while considering interindividual variability.

From Biological Synapses to “Intelligent” Robots

This selective review explores biologically inspired learning as a model for intelligent robot control and sensing technology on the basis of specific examples. Hebbian synaptic learning is discussed as a functionally relevant model for machine learning and intelligence, as explained on the basis of examples from the highly plastic biological neural networks of invertebrates and vertebrates. Its potential for adaptive learning and control without supervision, the generation of functional complexity, and control architectures based on self-organization is brought forward. Learning without prior knowledge based on excitatory and inhibitory neural mechanisms accounts for the process through which survival-relevant or task-relevant representations are either reinforced or suppressed. The basic mechanisms of unsupervised biological learning drive synaptic plasticity and adaptation for behavioral success in living brains with different levels of complexity. The insights collected here point toward the Hebbian model as a choice solution for “intelligent” robotics and sensor systems.

The brain as a dynamically active organ

Nervous systems are typically described as static networks passively responding to external stimuli (i.e., the 'sensorimotor hypothesis'). However, for more than a century now, evidence has been accumulating that this passive-static perspective is wrong. Instead, evidence suggests that nervous systems dynamically change their connectivity and actively generate behavior so their owners can achieve goals in the world, some of which involve controlling their sensory feedback. This review provides a brief overview of the different historical perspectives on general brain function and details some select modern examples falsifying the sensorimotor hypothesis.

The neuromuscular transform in a single segment of a segmented heart tube

Leech hearts are hybrids; they are myogenic but need entrainment by a heartbeat central pattern generator (CPG) to execute functional constriction patterns. Leech hearts are modular: two lateral segmented heart tubes running the length of the animal. Moving blood through the segmented heart tubes of leeches requires sequential constrictions, timed by a heartbeat CPG and relayed to each heart segment by likewise segmental motor neurons. The heartbeat CPG produces bilaterally asymmetric coordinations: rear-to-front peristaltic on one side and nearly synchronous on the other, periodically switching sides. We examined the neuromuscular transform of isolated heart segments in response to electrical nerve stimulation to identify the range of parameters (burst duration, intraburst pulse frequency, period) allowing the heart to constrict continuously and reliably. Constriction amplitudes increased with increasing intraburst frequencies and decreased with decreasing burst durations. Similar amplitudes were achieved with longer burst durations combined with lower frequencies or with shorter burst durations combined with higher frequencies. Long burst durations delayed relaxation, leading to summation and tetanus. The time, and its variability, between stimulus onset and time to constriction onset or to peak decreased with increasing frequency. Data previously obtained in vivo showed that the heart excitatory motor neurons fired longer bursts at lower frequencies at long periods moving to shorter bursts with higher intraburst frequencies as the period shortened. In this scenario, active constriction started earlier and the time to reach full systole shortened, allowing more time for relaxation. Relaxation time before the next motor neuron burst appears critical for maintaining constriction amplitude.NEW & NOTEWORTHY Moving blood through the segmented heart tubes of leeches requires sequential constrictions driven by motor neurons controlled by a central pattern generator. In a single heart segment, we varied stimuli to explore the neuromuscular transform. Decreasing the cycle period, e.g., to increase volume pumped over time, without altering motor burst duration and intraburst spike frequency shortens relaxation time and decreases amplitude. The likely strategy to preserve constriction amplitude is to shorten burst duration while increasing spike frequency.

Synaptic mechanisms for motor variability in a feedforward network

Behavioral variability often arises from variable activity in the behavior-generating neural network. The synaptic mechanisms underlying this variability are poorly understood. We show that synaptic noise, in conjunction with weak feedforward excitation, generates variable motor output in the Aplysia feeding system. A command-like neuron (CBI-10) triggers rhythmic motor programs more variable than programs triggered by CBI-2. CBI-10 weakly excites a pivotal pattern-generating interneuron (B34) strongly activated by CBI-2. The activation properties of B34 substantially account for the degree of program variability. CBI-10- and CBI-2-induced EPSPs in B34 vary in amplitude across trials, suggesting that there is synaptic noise. Computational studies show that synaptic noise is required for program variability. Further, at network state transition points when synaptic conductance is low, maximum program variability is promoted by moderate noise levels. Thus, synaptic strength and noise act together in a nonlinear manner to determine the degree of variability within a feedforward network.

Successful and unsuccessful attempts to swallow in a reduced Aplysia preparation regulate feeding responses and produce memory at different neural sites

Sensory feedback shapes ongoing behavior and may produce learning and memory. Motor responses to edible or inedible food in a reduced Aplysia preparation were examined to test how sensory feedback affects behavior and memory. Feeding patterns were initiated by applying a cholinomimetic onto the cerebral ganglion. Feedback from buccal muscles increased the response variability and response rate. Repeated application of the cholinomimetic caused decreased responses, expressed in part by lengthening protractions. Swallowing strips of "edible" food, which in intact animals induces learning that enhances ingestion, increased the response rate, and shortened the protraction length, reflecting more swallowing. Testing memory by repeating the procedure prevented the decrease in response rate observed with the cholinomimetic alone, and shortened protractions. Training with "inedible" food that in intact animals produces learning expressed by decreased responses caused lengthened protractions. Testing memory by repeating the procedure did not cause decreased responses or lengthened protractions. After training and testing with edible or inedible food, all preparations were exposed to the cholinomimetic alone. Preparations previously trained with edible food displayed memory expressed as decreased protraction length. Preparations previously trained with inedible food showed decreases in many response parameters. Memory for inedible food may arise in part via a postsynaptic decrease in response to acetylcholine released by afferents sensing food. The lack of change in response number, and in the time that responses are maintained during the two training sessions preceding application of the cholinomimetic alone suggests that memory expression may differ from behavioral changes during training.

Phylogenetic and individual variation in gastropod central pattern generators

Gastropod molluscs provide a unique opportunity to explore the neural basis of rhythmic behaviors because of the accessibility of their nervous systems and the number of species that have been examined. Detailed comparisons of the central pattern generators (CPGs) underlying rhythmic feeding and swimming behaviors highlight the presence and effects of variation in neural circuits both across and within species. The feeding motor pattern of the snail, Lymnaea, is stereotyped, whereas the feeding motor pattern in the sea hare, Aplysia, is variable. However, the Aplysia motor pattern is regularized with operant conditioning or by mimicking learning using the dynamic clamp to change properties of CPG neurons. Swimming evolved repeatedly in marine gastropods. Distinct neural mechanisms underlie dissimilar forms of swimming, with homologous neurons playing different roles. However, even similar swimming behaviors in different species can be produced by distinct neural mechanisms, resulting from different synaptic connectivity of homologous neurons. Within a species, there can be variation in the strength and even valence of synapses, which does not have functional relevance under normal conditions, but can cause some individuals to be more susceptible to lesion of the circuit. This inter- and intra-species variation provides novel insights into CPG function and plasticity.

Cycle-by-cycle assembly of respiratory network activity is dynamic and stochastic

Rhythmically active networks are typically composed of neurons that can be classified as silent, tonic spiking, or rhythmic bursting based on their intrinsic activity patterns. Within these networks, neurons are thought to discharge in distinct phase relationships with their overall network output, and it has been hypothesized that bursting pacemaker neurons may lead and potentially trigger cycle onsets. We used multielectrode recording from 72 experiments to test these ideas in rhythmically active slices containing the pre-Bötzinger complex, a region critical for breathing. Following synaptic blockade, respiratory neurons exhibited a gradient of intrinsic spiking to rhythmic bursting activities and thus defied an easy classification into bursting pacemaker and nonbursting categories. Features of their firing activity within the functional network were analyzed for correlation with subsequent rhythmic bursting in synaptic isolation. Higher firing rates through all phases of fictive respiration statistically predicted bursting pacemaker behavior. However, a cycle-by-cycle analysis indicated that respiratory neurons were stochastically activated with each burst. Intrinsically bursting pacemakers led some population bursts and followed others. This variability was not reproduced in traditional fully interconnected computational models, while sparsely connected network models reproduced these results both qualitatively and quantitatively. We hypothesize that pacemaker neurons do not act as clock-like drivers of the respiratory rhythm but rather play a flexible and dynamic role in the initiation and stabilization of each burst. Thus, at the behavioral level, each breath can be thought of as de novo assembly of a stochastic collaboration of network topology and intrinsic properties.

Functional organization and adaptability of a decision-making network in aplysia

Whereas major insights into the neuronal basis of adaptive behavior have been gained from the study of automatic behaviors, including reflexive and rhythmic motor acts, the neural substrates for goal-directed behaviors in which decision-making about action selection and initiation are crucial, remain poorly understood. However, the mollusk Aplysia is proving to be increasingly relevant to redressing this issue. The functional properties of the central circuits that govern this animal’s goal-directed feeding behavior and particularly the neural processes underlying the selection and initiation of specific feeding actions are becoming understood. In addition to relying on the intrinsic operation of central networks, goal-directed behaviors depend on external sensory inputs that through associative learning are able to shape decision-making strategies. Here, we will review recent findings on the functional design of the central network that generates Aplysia’s feeding-related movements and the sensory-derived plasticity that through learning can modify the selection and initiation of appropriate action. The animal’s feeding behavior and the implications of decision-making will be briefly described. The functional design of the underlying buccal network will then be used to illustrate how cellular diversity and the coordination of neuronal burst activity provide substrates for decision-making. The contribution of specific synaptic and neuronal membrane properties within the buccal circuit will also be discussed in terms of their role in motor pattern selection and initiation. The ability of learning to “rigidify” these synaptic and cellular properties so as to regularize network operation and lead to the expression of stereotyped rhythmic behavior will then be described. Finally, these aspects will be drawn into a conceptual framework of how Aplysia’s goal-directed circuitry compares to the central pattern generating networks for invertebrate rhythmic behaviors.

Modulation of Cellular and Synaptic Variability in the Lamprey Spinal Cord

Variability is increasingly recognized as a characteristic feature of cellular, synaptic, and network properties. While studies have traditionally focused on mean values, significant effects can result from changes in variance. This study has examined cellular and synaptic variability in the lamprey spinal cord and its modulation by the neuropeptide substance P. Cellular and synaptic variability differed in different types of cell and synapse. Substance P reduced the variability of subthreshold locomotor-related depolarizations and spiking in motor neurons during network activity. These effects were associated with a reduction in the variability of spiking in glutamatergic excitatory network interneurons and with a reduction in the variance of excitatory interneuron-evoked excitatory postsynaptic potentials (EPSPs). Substance P also reduced the variance of postsynpatic potentials (PSPs) from crossing inhibitory and excitatory interneurons, but it increased the variance of inhibitory postsynpatic potentials (IPSPs) from ipsilateral inhibitory interneurons. The effects on the variance of different PSPs could occur with or without changes in the PSP amplitude. The reduction in the variance of excitatory interneuron-evoked EPSPs was protein kinase A, calcium, and N-methyl-d-aspartate (NMDA) dependent. The NMDA dependence suggested that substance P was acting postsynaptically. This was supported by the reduced variability of postsynaptic responses to glutamate by substance P. However, ultrastructural analyses suggested that there may also be a presynaptic component to the modulation, because substance P reduced the variability of synaptic vesicle diameters in putative glutamatergic terminals. These results suggest that cellular and synaptic variability can be targeted for modulation, making it an additional source of spinal cord plasticity.

Neuromechanics of multifunctionality during rejection in Aplysia californica

How are the same muscles and neurons used to generate qualitatively different behaviors? We studied this question by analyzing the biomechanical and neural mechanisms of rejection responses in the marine mollusk Aplysia californica and compared these mechanisms with those used to generate swallowing responses (Ye et al., 2006). During rejection, the central grasper of the feeding structure closes to push inedible food out of the buccal cavity. This contrasts with swallowing, during which the grasper is open as it moves toward the jaws (protracts). We examined how the shape change of the grasper during rejection mechanically reconfigured the surrounding musculature. Grasper shape change increased the effectiveness of protractor muscle I2. The closed grasper alters the function of another muscle, the hinge, which becomes capable of inducing ventral rotations of rejected material. In contrast, during large-amplitude swallows, the hinge muscle mediates dorsal rotations of ingested material. Finally, after the grasper opens, its change in shape induces a delay in the activation of other surrounding muscles, the I1/I3/jaw complex, whose premature activation would close the halves of the grasper and induce it to pull inedible material back inward. The delay in activation of the I1/I3/jaw complex is partially attributable to identified multiaction neurons B4/B5. The results suggest that multifunctionality emerges from a periphery in which flexible coalitions of muscles may perform different functions in different mechanical contexts and in which neural circuitry is capable of reorganizing to exploit these coalitions by changes in phasing, duration, and intensity of motor neuronal activation.

Variability of swallowing performance in intact, freely feeding aplysia

Variability in nervous systems is often taken to be merely "noise." Yet in some cases it may play a positive, active role in the production of behavior. The central pattern generator (CPG) that drives the consummatory feeding behaviors of Aplysia generates large, quasi-random variability in the parameters of the feeding motor programs from one cycle to the next; the variability then propagates through the firing patterns of the motor neurons to the contractions of the feeding muscles. We have proposed that, when the animal is faced with a new, imperfectly known feeding task in each cycle, the variability implements a trial-and-error search through the space of possible feeding movements. Although this strategy will not be successful in every cycle, over many cycles it may be the optimal strategy for feeding in an uncertain and changing environment. To play this role, however, the variability must actually appear in the feeding movements and, presumably, in the functional performance of the feeding behavior. Here we have tested this critical prediction. We have developed a technique to measure, in intact, freely feeding animals, the performance of Aplysia swallowing behavior, by continuously recording with a length transducer the movement of the seaweed strip being swallowed. Simultaneously, we have recorded with implanted electrodes activity at each of the internal levels, the CPG, motor neurons, and muscles, of the feeding neuromusculature. Statistical analysis of a large data set of these recordings suggests that functional performance is not determined strongly by one or a few parameters of the internal activity, but weakly by many. Most important, the internal variability does emerge in the behavior and its functional performance. Even when the animal is swallowing a long, perfectly regular seaweed strip, remarkably, the length swallowed from cycle to cycle is extremely variable, as variable as the parameters of the activity of the CPG, motor neurons, and muscles.