Evolution and analysis of model CPGs for walking: II. General principles and individual variability

Milking a spherical cow: Toy models in neuroscience

There are many different kinds of models, and they play many different roles in the scientific endeavour. Neuroscience, and biology more generally, has understandably tended to emphasise empirical models that are grounded in data and make specific, experimentally testable predictions. Meanwhile, strongly idealised or 'toy' models have played a central role in the theoretical development of other sciences such as physics. In this paper, we examine the nature of toy models and their prospects in neuroscience.

Variational and phase response analysis for limit cycles with hard boundaries, with applications to neuromechanical control problems

Motor systems show an overall robustness, but because they are highly nonlinear, understanding how they achieve robustness is difficult. In many rhythmic systems, robustness against perturbations involves response of both the shape and the timing of the trajectory. This makes the study of robustness even more challenging. To understand how a motor system produces robust behaviors in a variable environment, we consider a neuromechanical model of motor patterns in the feeding apparatus of the marine mollusk Aplysia californica (Shaw et al. in J Comput Neurosci 38(1):25-51, 2015; Lyttle et al. in Biol Cybern 111(1):25-47, 2017). We established in (Wang et al. in SIAM J Appl Dyn Syst 20(2):701-744, 2021. https://doi.org/10.1137/20M1344974 ) the tools for studying combined shape and timing responses of limit cycle systems under sustained perturbations and here apply them to study robustness of the neuromechanical model against increased mechanical load during swallowing. Interestingly, we discover that nonlinear biomechanical properties confer resilience by immediately increasing resistance to applied loads. In contrast, the effect of changed sensory feedback signal is significantly delayed by the firing rates' hard boundary properties. Our analysis suggests that sensory feedback contributes to robustness in swallowing primarily by shifting the timing of neural activation involved in the power stroke of the motor cycle (retraction). This effect enables the system to generate stronger retractor muscle forces to compensate for the increased load, and hence achieve strong robustness. The approaches that we are applying to understanding a neuromechanical model in Aplysia, and the results that we have obtained, are likely to provide insights into the function of other motor systems that encounter changing mechanical loads and hard boundaries, both due to mechanical and neuronal firing properties.

Control for multifunctionality: bioinspired control based on feeding in Aplysia californica

Animals exhibit remarkable feats of behavioral flexibility and multifunctional control that remain challenging for robotic systems. The neural and morphological basis of multifunctionality in animals can provide a source of bioinspiration for robotic controllers. However, many existing approaches to modeling biological neural networks rely on computationally expensive models and tend to focus solely on the nervous system, often neglecting the biomechanics of the periphery. As a consequence, while these models are excellent tools for neuroscience, they fail to predict functional behavior in real time, which is a critical capability for robotic control. To meet the need for real-time multifunctional control, we have developed a hybrid Boolean model framework capable of modeling neural bursting activity and simple biomechanics at speeds faster than real time. Using this approach, we present a multifunctional model of Aplysia californica feeding that qualitatively reproduces three key feeding behaviors (biting, swallowing, and rejection), demonstrates behavioral switching in response to external sensory cues, and incorporates both known neural connectivity and a simple bioinspired mechanical model of the feeding apparatus. We demonstrate that the model can be used for formulating testable hypotheses and discuss the implications of this approach for robotic control and neuroscience.

A quantitative model of conserved macroscopic dynamics predicts future motor commands

In simple organisms such as Caenorhabditis elegans, whole brain imaging has been performed. Here, we use such recordings to model the nervous system. Our model uses neuronal activity to predict expected time of future motor commands up to 30 s prior to the event. These motor commands control locomotion. Predictions are valid for individuals not used in model construction. The model predicts dwell time statistics, sequences of motor commands and individual neuron activation. To develop this model, we extracted loops spanned by neuronal activity in phase space using novel methodology. The model uses only two variables: the identity of the loop and the phase along it. Current values of these macroscopic variables predict future neuronal activity. Remarkably, our model based on macroscopic variables succeeds despite consistent inter-individual differences in neuronal activation. Thus, our analytical framework reconciles consistent individual differences in neuronal activation with macroscopic dynamics that operate universally across individuals.

How can we go about trying to understand an object as complex as the brain? The traditional approach is to begin by studying its component parts, cells called neurons. Once we understand how individual neurons work, we can use computers to simulate the activity of networks of neurons. The result is a computer model of the brain. By comparing this model to data from real brains, we can try to make the model as similar to a real brain as possible.

But whose brain should we try to reproduce? The roundworm C. elegans, for example, has just 302 neurons in total. Advances in brain imaging mean it is now possible to identify each of these neurons and compare its activity across worms. But doing so reveals that the activity of any given neuron varies greatly between individuals. This is true even among genetically identical worms performing the same behavior.

Researchers trying to model the roundworm brain have attempted to model the average activity of each neuron across many worms. They hoped they could use these averages to predict the behavior of other worms from their neuronal activity. But this approach did not to work. Even in roundworms, the coordinated activity of many neurons is required to generate even simple behaviors. Averaging the activity of neurons across worms thus scrambles the information that encodes each behavior.

Brennan and Proekt have now overcome this problem by developing a more abstract model that treats the nervous system as a whole. The model takes into account changes in the activity of neurons, and in the worms’ behavior, over time. A model of this type built using one set of worms can predict the behavior of another set of worms. This approach may work because in evolution natural selection acts at the level of behaviors, and not at the level of individual neurons. The activity of individual neurons can thus vary between animals, even when those neurons encode the same behavior. This means it may also be possible to model the human brain without knowing the activity of each of its billions of neurons.

Uncertainpy: A Python Toolbox for Uncertainty Quantification and Sensitivity Analysis in Computational Neuroscience

Computational models in neuroscience typically contain many parameters that are poorly constrained by experimental data. Uncertainty quantification and sensitivity analysis provide rigorous procedures to quantify how the model output depends on this parameter uncertainty. Unfortunately, the application of such methods is not yet standard within the field of neuroscience. Here we present Uncertainpy, an open-source Python toolbox, tailored to perform uncertainty quantification and sensitivity analysis of neuroscience models. Uncertainpy aims to make it quick and easy to get started with uncertainty analysis, without any need for detailed prior knowledge. The toolbox allows uncertainty quantification and sensitivity analysis to be performed on already existing models without needing to modify the model equations or model implementation. Uncertainpy bases its analysis on polynomial chaos expansions, which are more efficient than the more standard Monte-Carlo based approaches. Uncertainpy is tailored for neuroscience applications by its built-in capability for calculating characteristic features in the model output. The toolbox does not merely perform a point-to-point comparison of the "raw" model output (e.g., membrane voltage traces), but can also calculate the uncertainty and sensitivity of salient model response features such as spike timing, action potential width, average interspike interval, and other features relevant for various neural and neural network models. Uncertainpy comes with several common models and features built in, and including custom models and new features is easy. The aim of the current paper is to present Uncertainpy to the neuroscience community in a user-oriented manner. To demonstrate its broad applicability, we perform an uncertainty quantification and sensitivity analysis of three case studies relevant for neuroscience: the original Hodgkin-Huxley point-neuron model for action potential generation, a multi-compartmental model of a thalamic interneuron implemented in the NEURON simulator, and a sparsely connected recurrent network model implemented in the NEST simulator.

PredPsych: A toolbox for predictive machine learning-based approach in experimental psychology research

Recent years have seen an increased interest in machine learning-based predictive methods for analyzing quantitative behavioral data in experimental psychology. While these methods can achieve relatively greater sensitivity compared to conventional univariate techniques, they still lack an established and accessible implementation. The aim of current work was to build an open-source R toolbox - "PredPsych" - that could make these methods readily available to all psychologists. PredPsych is a user-friendly, R toolbox based on machine-learning predictive algorithms. In this paper, we present the framework of PredPsych via the analysis of a recently published multiple-subject motion capture dataset. In addition, we discuss examples of possible research questions that can be addressed with the machine-learning algorithms implemented in PredPsych and cannot be easily addressed with univariate statistical analysis. We anticipate that PredPsych will be of use to researchers with limited programming experience not only in the field of psychology, but also in that of clinical neuroscience, enabling computational assessment of putative bio-behavioral markers for both prognosis and diagnosis.

Conditions for Multi-functionality in a Rhythm Generating Network Inspired by Turtle Scratching

Rhythmic behaviors such as breathing, walking, and scratching are vital to many species. Such behaviors can emerge from groups of neurons, called central pattern generators, in the absence of rhythmic inputs. In vertebrates, the identification of the cells that constitute the central pattern generator for particular rhythmic behaviors is difficult, and often, its existence has only been inferred. For example, under experimental conditions, intact turtles generate several rhythmic scratch motor patterns corresponding to non-rhythmic stimulation of different body regions. These patterns feature alternating phases of motoneuron activation that occur repeatedly, with different patterns distinguished by the relative timing and duration of activity of hip extensor, hip flexor, and knee extensor motoneurons. While the central pattern generator network responsible for these outputs has not been located, there is hope to use motoneuron recordings to deduce its properties. To this end, this work presents a model of a previously proposed central pattern generator network and analyzes its capability to produce two distinct scratch rhythms from a single neuron pool, selected by different combinations of tonic drive parameters but with fixed strengths of connections within the network. We show through simulation that the proposed network can achieve the desired multi-functionality, even though it relies on hip unit generators to recruit appropriately timed knee extensor motoneuron activity, including a delay relative to hip activation in rostral scratch. Furthermore, we develop a phase space representation, focusing on the inputs to and the intrinsic slow variable of the knee extensor motoneuron, which we use to derive sufficient conditions for the network to realize each rhythm and which illustrates the role of a saddle-node bifurcation in achieving the knee extensor delay. This framework is harnessed to consider bistability and to make predictions about the responses of the scratch rhythms to input changes for future experimental testing.

Information Flow through a Model of the C. elegans Klinotaxis Circuit

Understanding how information about external stimuli is transformed into behavior is one of the central goals of neuroscience. Here we characterize the information flow through a complete sensorimotor circuit: from stimulus, to sensory neurons, to interneurons, to motor neurons, to muscles, to motion. Specifically, we apply a recently developed framework for quantifying information flow to a previously published ensemble of models of salt klinotaxis in the nematode worm Caenorhabditis elegans. Despite large variations in the neural parameters of individual circuits, we found that the overall information flow architecture circuit is remarkably consistent across the ensemble. This suggests structural connectivity is not necessarily predictive of effective connectivity. It also suggests information flow analysis captures general principles of operation for the klinotaxis circuit. In addition, information flow analysis reveals several key principles underlying how the models operate: (1) Interneuron class AIY is responsible for integrating information about positive and negative changes in concentration, and exhibits a strong left/right information asymmetry. (2) Gap junctions play a crucial role in the transfer of information responsible for the information symmetry observed in interneuron class AIZ. (3) Neck motor neuron class SMB implements an information gating mechanism that underlies the circuit’s state-dependent response. (4) The neck carries more information about small changes in concentration than about large ones, and more information about positive changes in concentration than about negative ones. Thus, not all directions of movement are equally informative for the worm. Each of these findings corresponds to hypotheses that could potentially be tested in the worm. Knowing the results of these experiments would greatly refine our understanding of the neural circuit underlying klinotaxis.

Sensory Feedback Reduces Individuality by Increasing Variability within Subjects

Behavioral variability is ubiquitous [1-6], yet variability is more than just noise. Indeed, humans exploit their individual motor variability to improve tracing and reaching tasks [7]. What controls motor variability? Increasing the variability of sensory input, or applying force perturbations during a task, increases task variability [8, 9]. Sensory feedback may also increase task-irrelevant variability [9, 10]. In contrast, sensory feedback during locust flight or to multiple cortical areas just prior to task performance decreases variability during task-relevant motor behavior [11, 12]. Thus, how sensory feedback affects both task-relevant and task-irrelevant motor outputs must be understood. Furthermore, since motor control is studied in populations, the effects of sensory feedback on variability must also be understood within and across subjects. For example, during locomotion, each step may vary within and across individuals, even when behavior is normalized by step cycle duration [13]. Our previous work demonstrated that motor components that matter for effective behavior show less individuality [14]. Is sensory feedback the mechanism for reducing individuality? We analyzed durations and relative timings of motor pools within swallowing motor patterns in the presence and absence of sensory feedback and related these motor program components to behavior. Here, at the level of identified motor neurons, we show that sensory feedback to motor program components highly correlated with behavioral efficacy reduces variability across subjects but-surprisingly-increases variability within subjects. By controlling intrinsic, individual differences in motor neuronal activity, sensory feedback provides each subject access to a common solution space.

Neuromechanical principles underlying movement modularity and their implications for rehabilitation

Neuromechanical principles define the properties and problems that shape neural solutions for movement. Although the theoretical and experimental evidence is debated, we present arguments for consistent structures in motor patterns, i.e., motor modules, that are neuromechanical solutions for movement particular to an individual and shaped by evolutionary, developmental, and learning processes. As a consequence, motor modules may be useful in assessing sensorimotor deficits specific to an individual and define targets for the rational development of novel rehabilitation therapies that enhance neural plasticity and sculpt motor recovery. We propose that motor module organization is disrupted and may be improved by therapy in spinal cord injury, stroke, and Parkinson's disease. Recent studies provide insights into the yet-unknown underlying neural mechanisms of motor modules, motor impairment, and motor learning and may lead to better understanding of the causal nature of modularity and its underlying neural substrates.

Phylogenetic and individual variation in gastropod central pattern generators

Gastropod molluscs provide a unique opportunity to explore the neural basis of rhythmic behaviors because of the accessibility of their nervous systems and the number of species that have been examined. Detailed comparisons of the central pattern generators (CPGs) underlying rhythmic feeding and swimming behaviors highlight the presence and effects of variation in neural circuits both across and within species. The feeding motor pattern of the snail, Lymnaea, is stereotyped, whereas the feeding motor pattern in the sea hare, Aplysia, is variable. However, the Aplysia motor pattern is regularized with operant conditioning or by mimicking learning using the dynamic clamp to change properties of CPG neurons. Swimming evolved repeatedly in marine gastropods. Distinct neural mechanisms underlie dissimilar forms of swimming, with homologous neurons playing different roles. However, even similar swimming behaviors in different species can be produced by distinct neural mechanisms, resulting from different synaptic connectivity of homologous neurons. Within a species, there can be variation in the strength and even valence of synapses, which does not have functional relevance under normal conditions, but can cause some individuals to be more susceptible to lesion of the circuit. This inter- and intra-species variation provides novel insights into CPG function and plasticity.

Motor neuronal activity varies least among individuals when it matters most for behavior

How does motor neuronal variability affect behavior? To explore this question, we quantified activity of multiple individual identified motor neurons mediating biting and swallowing in intact, behaving Aplysia californica by recording from the protractor muscle and the three nerves containing the majority of motor neurons controlling the feeding musculature. We measured multiple motor components: duration of the activity of identified motor neurons as well as their relative timing. At the same time, we measured behavioral efficacy: amplitude of grasping movement during biting and amplitude of net inward food movement during swallowing. We observed that the total duration of the behaviors varied: Within animals, biting duration shortened from the first to the second and third bites; between animals, biting and swallowing durations varied. To study other sources of variation, motor components were divided by behavior duration (i.e., normalized). Even after normalization, distributions of motor component durations could distinguish animals as unique individuals. However, the degree to which a motor component varied among individuals depended on the role of that motor component in a behavior. Motor neuronal activity that was essential for the expression of biting or swallowing was similar among animals, whereas motor neuronal activity that was not essential for that behavior varied more from individual to individual. These results suggest that motor neuronal activity that matters most for the expression of a particular behavior may vary least from individual to individual. Shaping individual variability to ensure behavioral efficacy may be a general principle for the operation of motor systems.

A physical model of sensorimotor interactions during locomotion

In this paper, we describe the development of a bipedal robot that models the neuromuscular architecture of human walking. The body is based on principles derived from human muscular architecture, using muscles on straps to mimic agonist/antagonist muscle action as well as bifunctional muscles. Load sensors in the straps model Golgi tendon organs. The neural architecture is a central pattern generator (CPG) composed of a half-center oscillator combined with phase-modulated reflexes that is simulated using a spiking neural network. We show that the interaction between the reflex system, body dynamics and CPG results in a walking cycle that is entrained to the dynamics of the system. We also show that the CPG helped stabilize the gait against perturbations relative to a purely reflexive system, and compared the joint trajectories to human walking data. This robot represents a complete physical, or 'neurorobotic', model of the system, demonstrating the usefulness of this type of robotics research for investigating the neurophysiological processes underlying walking in humans and animals.

EMULATION OF HUMAN FEELINGS AND BEHAVIORS IN AN ANIMATED ARTWORK

The behavior of an animated artwork, survivor — a classroom chair which walks, with a dynamics which some viewers find haunting — reflects an attempt to emulate (and suggest to viewers) some feelings and behaviors that are typical of survivors of landmine blasts, learning to use crutches. The artwork itself is intended for sensitizing viewers to the horror experienced by those who survive, and those who do not. The behavior of such a survivor is affected by several factors: some are due to the objective difficulty of using prosthetic legs, and some are due to emotional factors, e.g., fear, "shame" of being in such situation, and pain. The mechanical structure, strongly conditioned by artistic requirements, was combined with a control system that exhibits appropriate behaviors. Behavioral control, a technique developed for the control of mobile robots, was used in survivor, and implemented over a modified version of the traditional Brooks' subsumption architecture. This technique makes it possible to emulate normal locomotion behaviors such as the need of avoiding obstacles and typical animal feelings such as curiosity, hunger, fatigue and fear. We describe the mechanics and viewers' response, and formalize aesthetic response. We briefly survey computer modelling of emotions, robotic art, and biomimetic locomotion in robotics.

Qualitative Functional Decomposition Analysis of Evolved Neuromorphic Flight Controllers

In the previous work, it was demonstrated that one can effectively employ CTRNN-EH (a neuromorphic variant of EH method) methodology to evolve neuromorphic flight controllers for a flapping wing robot. This paper describes a novel frequency grouping-based analysis technique, developed to qualitatively decompose the evolved controllers into explainable functional control blocks. A summary of the previous work related to evolving flight controllers for two categories of the controller types, called autonomous and nonautonomous controllers, is provided, and the applicability of the newly developed decomposition analysis for both controller categories is demonstrated. Further, the paper concludes with appropriate discussion of ongoing work and implications for possible future work related to employing the CTRNN-EH methodology and the decomposition analysis techniques presented in this paper.

A COMMERCIAL OFF-THE-SHELF IMPLEMENTATION OF AN ANALOG NEURAL COMPUTER

For most applications, analog electrical circuit implementations of continuous-valued neural networks have been abandoned in favor of digital simulations. This is not surprising, as both precision and accuracy can be more easily ensured in digital computers. Still, because they use far fewer transistors and support components, analog circuits can still be orders of magnitude smaller than their digital simulations. In some application, like micro-robotics and embedded control, one might be willing to tolerate less accuracy and precision for the size and power benefits. One would not under any condition, however, tolerate significant behavioral mismatches between the differential equation and electrical circuit forms of the neural networks in question. In this paper, we will present a design for an analog neural computer that embodies the commonly used continuous time recurrent neural network. We will show that the computer possesses excellent behavioral congruence to the differential equation form even in the presence of significant practical compromises. We will also discuss the implications of this work for both practical Commercial, Off-The-Shelf (COTS) and Application-Specific Integrated Circuit (ASIC) devices.

A dynamical systems analysis of afferent control in a neuromechanical model of locomotion: II. Phase asymmetry

In this paper we analyze a closed loop neuromechanical model of locomotor rhythm generation. The model is composed of a spinal central pattern generator (CPG) and a single-joint limb, with CPG outputs projecting via motoneurons to muscles that control the limb and afferent signals from the muscles feeding back to the CPG. In a preceding companion paper (Spardy et al 2011 J. Neural Eng. 8 065003), we analyzed how the model generates oscillations in the presence or absence of feedback, identified curves in a phase plane associated with the limb that signify where feedback levels induce phase transitions within the CPG, and explained how increasing feedback strength restores oscillations in a model representation of spinal cord injury; from these steps, we derived insights about features of locomotor rhythms in several scenarios and made predictions about rhythm responses to various perturbations. In this paper, we exploit our analytical observations to construct a reduced model that retains important characteristics from the original system. We prove the existence of an oscillatory solution to the reduced model using a novel version of a Melnikov function, adapted for discontinuous systems, and also comment on the uniqueness and stability of this solution. Our analysis yields a deeper understanding of how the model must be tuned to generate oscillations and how the details of the limb dynamics shape overall model behavior. In particular, we explain how, due to the feedback signals in the model, changes in the strength of a tonic supra-spinal drive to the CPG yield asymmetric alterations in the durations of different locomotor phases, despite symmetry within the CPG itself.

Strong interactions between spinal cord networks for locomotion and scratching

Distinct rhythmic behaviors involving a common set of motoneurons and muscles can be generated by separate central nervous system (CNS) networks, a single network, or partly overlapping networks in invertebrates. Less is known for vertebrates. Simultaneous activation of two networks can reveal overlap or interactions between them. The turtle spinal cord contains networks that generate locomotion and three forms of scratching (rostral, pocket, and caudal), having different knee-hip synergies. Here, we report that in immobilized spinal turtles, simultaneous delivery of types of stimulation, which individually evoked forward swimming and one form of scratching, could 1) increase the rhythm frequency; 2) evoke switches, hybrids, and intermediate motor patterns; 3) recruit a swim motor pattern even when the swim stimulation was reduced to subthreshold intensity; and 4) disrupt rhythm generation entirely. The strength of swim stimulation could influence the result. Thus even pocket scratching and caudal scratching, which do not share a knee-hip synergy with forward swimming, can interact with swim stimulation to alter both rhythm and pattern generation. Model simulations were used to explore the compatibility of our experimental results with hypothetical network architectures for rhythm generation. Models could reproduce experimental observations only if they included interactions between neurons involved in swim and scratch rhythm generation, with maximal consistency between simulations and experiments attained using a model architecture in which certain neurons participated actively in both swim and scratch rhythmogenesis. Collectively, these findings suggest that the spinal cord networks that generate locomotion and scratching have important shared components or strong interactions between them.

Multiple models to capture the variability in biological neurons and networks

How tightly tuned are the synaptic and intrinsic properties that give rise to neuron and circuit function? Experimental work shows that these properties vary considerably across identified neurons in different animals. Given this variability in experimental data, this review describes some of the complications of building computational models to aid in understanding how system dynamics arise from the interaction of system components. We argue that instead of trying to build a single model that captures the generic behavior of a neuron or circuit, it is beneficial to construct a population of models that captures the behavior of the population that provided the experimental data. Studying a population of models with different underlying structure and similar behaviors provides opportunities to discover unsuspected compensatory mechanisms that contribute to neuron and network function.

Variability, compensation, and modulation in neurons and circuits

I summarize recent computational and experimental work that addresses the inherent variability in the synaptic and intrinsic conductances in normal healthy brains and shows that multiple solutions (sets of parameters) can produce similar circuit performance. I then discuss a number of issues raised by this observation, such as which parameter variations arise from compensatory mechanisms and which reflect insensitivity to those particular parameters. I ask whether networks with different sets of underlying parameters can nonetheless respond reliably to neuromodulation and other global perturbations. At the computational level, I describe a paradigm shift in which it is becoming increasingly common to develop families of models that reflect the variance in the biological data that the models are intended to illuminate rather than single, highly tuned models. On the experimental side, I discuss the inherent limitations of overreliance on mean data and suggest that it is important to look for compensations and correlations among as many system parameters as possible, and between each system parameter and circuit performance. This second paradigm shift will require moving away from measurements of each system component in isolation but should reveal important previously undescribed principles in the organization of complex systems such as brains.

Evolving experience-dependent robust behaviour in embodied agents

In this work, based on behavioural and dynamical evidence, a study of simulated agents with the capacity to change feedback from their bodies to accomplish a one-legged walking task is proposed to understand the emergence of coupled dynamics for robust behaviour. Agents evolve with evolutionary-defined biases that modify incoming body signals (sensory offsets). Analyses on whether these agents show further dependence to their environmental coupled dynamics than others with no feedback control is described in this article. The ability to sustain behaviours is tested during lifetime experiments with mutational and sensory perturbations after evolution. Using dynamical systems analysis, this work identifies conditions for the emergence of dynamical mechanisms that remain functional despite sensory perturbations. Results indicate that evolved agents with evolvable sensory offset depends not only on where in neural space the state of the neural system operates, but also on the transients to which the inner-system was being driven by sensory signals from its interactions with the environment, controller, and agent body. Experimental evidence here leads discussions on a dynamical systems perspective on behavioural robustness that goes beyond attractors of controller phase space.

Simplified and effective motor control based on muscle synergies to exploit musculoskeletal dynamics

The basic hypothesis of producing a range of behaviors using a small set of motor commands has been proposed in various forms to explain motor behaviors ranging from basic reflexes to complex voluntary movements. Yet many fundamental questions regarding this long-standing hypothesis remain unanswered. Indeed, given the prominent nonlinearities and high dimensionality inherent in the control of biological limbs, the basic feasibility of a low-dimensional controller and an underlying principle for its creation has remained elusive. We propose a principle for the design of such a controller, that it endeavors to control the natural dynamics of the limb, taking into account the nature of the task being performed. Using this principle, we obtained a low-dimensional model of the hindlimb and a set of muscle synergies to command it. We demonstrate that this set of synergies was capable of producing effective control, establishing the viability of this muscle synergy hypothesis. Finally, by combining the low-dimensional model and the muscle synergies we were able to build a relatively simple controller whose overall performance was close to that of the system's full-dimensional nonlinear controller. Taken together, the results of this study establish that a low-dimensional controller is capable of simplifying control without degrading performance.

Multifunctional pattern-generating circuits

The ability of distinct anatomical circuits to generate multiple behavioral patterns is widespread among vertebrate and invertebrate species. These multifunctional neuronal circuits are the result of multistable neural dynamics and modular organization. The evidence suggests multifunctional circuits can be classified by distinct architectures, yet the activity patterns of individual neurons involved in more than one behavior can vary dramatically. Several mechanisms, including sensory input, the parallel activity of projection neurons, neuromodulation, and biomechanics, are responsible for the switching between patterns. Recent advances in both analytical and experimental tools have aided the study of these complex circuits.

Inevitable evolutionary temporal elements in neural processing: a study based on evolutionary simulations

Recent studies have suggested that some neural computational mechanisms are based on the fine temporal structure of spiking activity. However, less effort has been devoted to investigating the evolutionary aspects of such mechanisms. In this paper we explore the issue of temporal neural computation from an evolutionary point of view, using a genetic simulation of the evolutionary development of neural systems. We evolve neural systems in an environment with selective pressure based on mate finding, and examine the temporal aspects of the evolved systems. In repeating evolutionary sessions, there was a significant increase during evolution in the mutual information between the evolved agent's temporal neural representation and the external environment. In ten different simulated evolutionary sessions, there was an increased effect of time -related neural ablations on the agents' fitness. These results suggest that in some fitness landscapes the emergence of temporal elements in neural computation is almost inevitable. Future research using similar evolutionary simulations may shed new light on various biological mechanisms.

Predicting adaptive behavior in the environment from central nervous system dynamics

To generate adaptive behavior, the nervous system is coupled to the environment. The coupling constrains the dynamical properties that the nervous system and the environment must have relative to each other if adaptive behavior is to be produced. In previous computational studies, such constraints have been used to evolve controllers or artificial agents to perform a behavioral task in a given environment. Often, however, we already know the controller, the real nervous system, and its dynamics. Here we propose that the constraints can also be used to solve the inverse problem--to predict from the dynamics of the nervous system the environment to which they are adapted, and so reconstruct the production of the adaptive behavior by the entire coupled system. We illustrate how this can be done in the feeding system of the sea slug Aplysia. At the core of this system is a central pattern generator (CPG) that, with dynamics on both fast and slow time scales, integrates incoming sensory stimuli to produce ingestive and egestive motor programs. We run models embodying these CPG dynamics--in effect, autonomous Aplysia agents--in various feeding environments and analyze the performance of the entire system in a realistic feeding task. We find that the dynamics of the system are tuned for optimal performance in a narrow range of environments that correspond well to those that Aplysia encounter in the wild. In these environments, the slow CPG dynamics implement efficient ingestion of edible seaweed strips with minimal sensory information about them. The fast dynamics then implement a switch to a different behavioral mode in which the system ignores the sensory information completely and follows an internal "goal," emergent from the dynamics, to egest again a strip that proves to be inedible. Key predictions of this reconstruction are confirmed in real feeding animals.

Parameter space structure of continuous-time recurrent neural networks

A fundamental challenge for any general theory of neural circuits is how to characterize the structure of the space of all possible circuits over a given model neuron. As a first step in this direction, this letter begins a systematic study of the global parameter space structure of continuous-time recurrent neural networks (CTRNNs), a class of neural models that is simple but dynamically universal. First, we explicitly compute the local bifurcation manifolds of CTRNNs. We then visualize the structure of these manifolds in net input space for small circuits. These visualizations reveal a set of extremal saddle node bifurcation manifolds that divide CTRNN parameter space into regions of dynamics with different effective dimensionality. Next, we completely characterize the combinatorics and geometry of an asymptotically exact approximation to these regions for circuits of arbitrary size. Finally, we show how these regions can be used to calculate estimates of the probability of encountering different kinds of dynamics in CTRNN parameter space.

Connection and Coordination: The Interplay Between Architecture and Dynamics in Evolved Model Pattern Generators

We undertake a systematic study of the role of neural architecture in shaping the dynamics of evolved model pattern generators for a walking task. First, we consider the minimum number of connections necessary to achieve high performance on this task. Next, we identify architectural motifs associated with high fitness. We then examine how high-fitness architectures differ in their ability to evolve. Finally, we demonstrate the existence of distinct parameter subgroups in some architectures and show that these subgroups are characterized by differences in neuron excitabilities and connection signs.

Evolving physically simulated flying creatures for efficient cruising

The body-brain coevolution of aerial life forms has not been developed as far as aquatic or terrestrial locomotion in the field of artificial life. We are studying physically simulated 3D flying creatures by evolving both wing shapes and their controllers. A creature's wing is modeled as a number of articulated cylinders, connected by triangular films (patagia). The wing structure and its motor controllers for cruising flight are generated by an evolutionary algorithm within a simulated aerodynamic environment. The most energy-efficient cruising speed and the lift and drag coefficients of each flier are calculated from its morphological characteristics and used in the fitness evaluation. To observe a wide range of creature size, the evolution is run separately for creatures categorized into three species by body weight. The resulting creatures vary in size from pigeons to pterosaurs, with various wing configurations. We discuss the characteristics of shape and motion of the evolved creatures, including flight stability and Strouhal number.

Variability of swallowing performance in intact, freely feeding aplysia

Variability in nervous systems is often taken to be merely "noise." Yet in some cases it may play a positive, active role in the production of behavior. The central pattern generator (CPG) that drives the consummatory feeding behaviors of Aplysia generates large, quasi-random variability in the parameters of the feeding motor programs from one cycle to the next; the variability then propagates through the firing patterns of the motor neurons to the contractions of the feeding muscles. We have proposed that, when the animal is faced with a new, imperfectly known feeding task in each cycle, the variability implements a trial-and-error search through the space of possible feeding movements. Although this strategy will not be successful in every cycle, over many cycles it may be the optimal strategy for feeding in an uncertain and changing environment. To play this role, however, the variability must actually appear in the feeding movements and, presumably, in the functional performance of the feeding behavior. Here we have tested this critical prediction. We have developed a technique to measure, in intact, freely feeding animals, the performance of Aplysia swallowing behavior, by continuously recording with a length transducer the movement of the seaweed strip being swallowed. Simultaneously, we have recorded with implanted electrodes activity at each of the internal levels, the CPG, motor neurons, and muscles, of the feeding neuromusculature. Statistical analysis of a large data set of these recordings suggests that functional performance is not determined strongly by one or a few parameters of the internal activity, but weakly by many. Most important, the internal variability does emerge in the behavior and its functional performance. Even when the animal is swallowing a long, perfectly regular seaweed strip, remarkably, the length swallowed from cycle to cycle is extremely variable, as variable as the parameters of the activity of the CPG, motor neurons, and muscles.

Travelling wave patterns in a model of the spinal pattern generator using spiking neurons

The aim of this study is to produce travelling waves in a planar net of artificial spiking neurons. Provided that the parameters of the waves--frequency, wavelength and orientation--can be sufficiently controlled, such a network can serve as a model of the spinal pattern generator for swimming and terrestrial quadruped locomotion. A previous implementation using non-spiking, sigmoid neurons lacked the physiological plausibility that can only be attained using more realistic spiking neurons. Simulations were conducted using three types of spiking neuronal models. First, leaky integrate-and-fire neurons were used. Second, we introduced a phenomenological bursting neuron. And third, a canonical model neuron was implemented which could reproduce the full dynamics of the Hodgkin-Huxley neuron. The conditions necessary to produce appropriate travelling waves corresponded largely to the known anatomy and physiology of the spinal cord. Especially important features for the generation of travelling waves were the topology of the local connections--so-called off-centre connectivity--the availability of dynamic synapses and, to some extent, the availability of bursting cell types. The latter were necessary to produce stable waves at the low frequencies observed in quadruped locomotion. In general, the phenomenon of travelling waves was very robust and largely independent of the network parameters and emulated cell types.

Modeling neuromuscular modulation in Aplysia. III. Interaction of central motor commands and peripheral modulatory state for optimal behavior

Recent work in computational neuroethology has emphasized that "the brain has a body": successful adaptive behavior is not simply commanded by the nervous system, but emerges from interactions of nervous system, body, and environment. Here we continue our study of these issues in the accessory radula closer (ARC) neuromuscular system of Aplysia. The ARC muscle participates in the animal's feeding behaviors, a set of cyclical, rhythmic behaviors driven by a central pattern generator (CPG). Patterned firing of the ARC muscle's two motor neurons, B15 and B16, releases not only ACh to elicit the muscle's contractions but also peptide neuromodulators that then shape the contractions through a complex network of actions on the muscle. These actions are dynamically complex: some are fast, but some are slow, so that they are temporally uncoupled from the motor neuron firing pattern in the current cycle. Under these circumstances, how can the nervous system, through just the narrow channel of the firing patterns of the motor neurons, control the contractions, movements, and behavior in the periphery? In two earlier papers, we developed a realistic mathematical model of the B15/B16-ARC neuromuscular system and its modulation. Here we use this model to study the functional performance of the system in a realistic behavioral task. We run the model with two kinds of inputs: a simple set of regular motor neuron firing patterns that allows us to examine the entire space of patterns, and the real firing patterns of B15 and B16 previously recorded in a 2 1/2-h-long meal of 749 cycles in an intact feeding animal. These real patterns are extremely irregular. Our main conclusions are the following. 1) The modulation in the periphery is necessary for superior functional performance. 2) The components of the modulatory network interact in nonlinear, context- and task-dependent combinations for best performance overall, although not necessarily in any particular cycle. 3) Both the fast and the slow dynamics of the modulatory state make important contributions. 4) The nervous system controls different components of the periphery to different degrees. To some extent the periphery operates semiautonomously. However, the structure of the peripheral modulatory network ensures robust performance under all circumstances, even with the irregular motor neuron firing patterns and even when the parameters of the functional task are randomly varied from cycle to cycle to simulate a variable feeding environment. In the variable environment, regular firing patterns, which are fine-tuned to one particular task, fail to provide robust performance. We propose that the CPG generates the irregular firing patterns, which nevertheless are guaranteed to give robust performance overall through the actions of the peripheral modulatory network, as part of a trial-and-error feeding strategy in a variable, uncertain environment.

Cycle-to-Cycle Variability of Neuromuscular Activity inAplysiaFeeding Behavior

Aplysia consummatory feeding behavior, a rhythmic cycling of biting, swallowing, and rejection movements, is often said to be stereotyped. Yet closer examination shows that cycles of the behavior are very variable. Here we have quantified and analyzed the variability at several complementary levels in the neuromuscular system. In reduced preparations, we recorded the motor programs produced by the central pattern generator, firing of the motor neurons B15 and B16, and contractions of the accessory radula closer (ARC) muscle while repetitive programs were elicited by stimulation of the esophageal nerve. In other similar experiments, we recorded firing of motor neuron B48 and contractions of the radula opener muscle. In intact animals, we implanted electrodes to record nerve or ARC muscle activity while the animals swallowed controlled strips of seaweed or fed freely. In all cases, we found large variability in all parameters examined. Some of this variability reflected systematic, slow, history-dependent changes in the character of the central motor programs. Even when these trends were factored out, however, by focusing only on the differences between successive cycles, considerable variability remained. This variability was apparently random. Nevertheless, it too was the product of central history dependency because regularizing merely the high-level timing of the programs also regularized many of the downstream neuromuscular parameters. Central motor program variability thus appears directly in the behavior. With regard to the production of functional behavior in any one cycle, the large variability may indicate broad tolerances in the operation of the neuromuscular system. Alternatively, some cycles of the behavior may be dysfunctional. Overall, the variability may be part of an optimal strategy of trial, error, and stabilization that the CNS adopts in an uncertain environment.

Dynamics from a time series: can we extract the phase resetting curve from a time series?

Recordings of the membrane potential from a bursting neuron were used to reconstruct the phase curve for that neuron for a limited set of perturbations. These perturbations were inhibitory synaptic conductance pulses able to shift the membrane potential below the most hyperpolarized level attained in the free running mode. The extraction of the phase resetting curve from such a one-dimensional time series requires reconstruction of the periodic activity in the form of a limit cycle attractor. Resetting was found to have two components. In the first component, if the pulse was applied during a burst, the burst was truncated, and the time until the next burst was shortened in a manner predicted by movement normal to the limit cycle. By movement normal to the limit cycle, we mean a switch between two well-defined solution branches of a relaxation-like oscillator in a hysteretic manner enabled by the existence of a singular dominant slow process (variable). In the second component, the onset of the burst was delayed until the end of the hyperpolarizing pulse. Thus, for the pulse amplitudes we studied, resetting was independent of amplitude but increased linearly with pulse duration. The predicted and the experimental phase resetting curves for a pyloric dilator neuron show satisfactory agreement. The method was applied to only one pulse per cycle, but our results suggest it could easily be generalized to accommodate multiple inputs.

High-dimensional analysis of evolutionary autonomous agents

This article presents a new approach to the important challenge of localizing function in a neurocontroller. The approach is based on the basic functional contribution analysis (FCA) presented earlier, which assigns contribution values to the elements of the network, such that the ability to predict the network's performance in response to multi-unit lesions is maximized. These contribution values quantify the importance of each element to the tasks the agent performs. Here we present a generalization of the basic FCA to high-dimensional analysis, using high-order compound elements. Such elements are composed of conjunctions of simple elements. Their usage enables the explicit expression of sets of neurons or synapses whose contributions are interdependent, a prerequisite for localizing the function of complex neurocontrollers. High-dimensional FCA is shown to significantly improve on the accuracy of the basic analysis, to provide new insights concerning the main subsets of simple elements in the network that interact in a complex nonlinear manner, and to systematically reveal the types of interactions that characterize the evolved neurocontroller.

Center-crossing recurrent neural networks for the evolution of rhythmic behavior

A center-crossing recurrent neural network is one in which the null-(hyper)surfaces of each neuron intersect at their exact centers of symmetry, ensuring that each neuron's activation function is centered over the range of net inputs that it receives. We demonstrate that relative to a random initial population, seeding the initial population of an evolutionary search with center-crossing networks significantly improves both the frequency and the speed with which high-fitness oscillatory circuits evolve on a simple walking task. The improvement is especially striking at low mutation variances. Our results suggest that seeding with center-crossing networks may often be beneficial, since a wider range of dynamics is more likely to be easily accessible from a population of center-crossing networks than from a population of random networks.

The influence of limit cycle topology on the phase resetting curve

Understanding the phenomenology of phase resetting is an essential step toward developing a formalism for the analysis of circuits composed of bursting neurons that receive multiple, and sometimes overlapping, inputs. If we are to use phase-resetting methods to analyze these circuits, we can either generate phase-resetting curves (PRCs) for all possible inputs and combinations of inputs, or we can develop an understanding of how to construct PRCs for arbitrary perturbations of a given neuron. The latter strategy is the goal of this study. We present a geometrical derivation of phase resetting of neural limit cycle oscillators in response to short current pulses. A geometrical phase is defined as the distance traveled along the limit cycle in the appropriate phase space. The perturbations in current are treated as displacements in the direction corresponding to membrane voltage. We show that for type I oscillators, the direction of a perturbation in current is nearly tangent to the limit cycle; hence, the projection of the displacement in voltage onto the limit cycle is sufficient to give the geometrical phase resetting. In order to obtain the phase resetting in terms of elapsed time or temporal phase, a mapping between geometrical and temporal phase is obtained empirically and used to make the conversion. This mapping is shown to be an invariant of the dynamics. Perturbations in current applied to type II oscillators produce significant normal displacements from the limit cycle, so the difference in angular velocity at displaced points compared to the angular velocity on the limit cycle must be taken into account. Empirical attempts to correct for differences in angular velocity (amplitude versus phase effects in terms of a circular coordinate system) during relaxation back to the limit cycle achieved some success in the construction of phase-resetting curves for type II model oscillators. The ultimate goal of this work is the extension of these techniques to biological circuits comprising type II neural oscillators, which appear frequently in identified central pattern-generating circuits.

Failure of averaging in the construction of a conductance-based neuron model

Parameters for models of biological systems are often obtained by averaging over experimental results from a number of different preparations. To explore the validity of this procedure, we studied the behavior of a conductance-based model neuron with five voltage-dependent conductances. We randomly varied the maximal conductance of each of the active currents in the model and identified sets of maximal conductances that generate bursting neurons that fire a single action potential at the peak of a slow membrane potential depolarization. A model constructed using the means of the maximal conductances of this population is not itself a one-spike burster, but rather fires three action potentials per burst. Averaging fails because the maximal conductances of the population of one-spike bursters lie in a highly concave region of parameter space that does not contain its mean. This demonstrates that averages over multiple samples can fail to characterize a system whose behavior depends on interactions involving a number of highly variable components.

Evolutionary autonomous agents: a neuroscience perspective

In this article, I discuss the use of neurally driven evolutionary autonomous agents (EAAs) in neuroscientific investigations. Two fundamental questions are addressed. Can EAA studies shed new light on the structure and function of biological nervous systems? And can these studies lead to the development of new tools for neuroscientific analysis? The value and significant potential of EAA modelling in both respects is demonstrated and discussed. Although the study of EAAs for neuroscience research still faces difficult conceptual and technical challenges, it is a promising and timely endeavour.

Global structure, robustness, and modulation of neuronal models

The electrical characteristics of many neurons are remarkably robust in the face of changing internal and external conditions. At the same time, neurons can be highly sensitive to neuromodulators. We find correlates of this dual robustness and sensitivity in a global analysis of the structure of a conductance-based model neuron. We vary the maximal conductance parameters of the model neuron and, for each set of parameters tested, characterize the activity pattern generated by the cell as silent, tonically firing, or bursting. Within the parameter space of the five maximal conductances of the model, we find directions, representing concerted changes in multiple conductances, along which the basic pattern of neural activity does not change. In other directions, relatively small concurrent changes in a few conductances can induce transitions between these activity patterns. The global structure of the conductance-space maps implies that neuromodulators that alter a sensitive set of conductances will have powerful, and possibly state-dependent, effects. Other modulators that may have no direct impact on the activity of the neuron may nevertheless change the effects of such direct modulators via this state dependence. Some of the results and predictions arising from the model studies are replicated and verified in recordings of stomatogastric ganglion neurons using the dynamic clamp.

Emergence of memory-driven command neurons in evolved artificial agents

Using evolutionary simulations, we develop autonomous agents controlled by artificial neural networks (ANNs). In simple lifelike tasks of foraging and navigation, high performance levels are attained by agents equipped with fully recurrent ANN controllers. In a set of experiments sharing the same behavioral task but differing in the sensory input available to the agents, we find a common structure of a command neuron switching the dynamics of the network between radically different behavioral modes. When sensory position information is available, the command neuron reflects a map of the environment, acting as a location-dependent cell sensitive to the location and orientation of the agent. When such information is unavailable, the command neuron's activity is based on a spontaneously evolving short-term memory mechanism, which underlies its apparent place-sensitive activity. A two-parameter stochastic model for this memory mechanism is proposed. We show that the parameter values emerging from the evolutionary simulations are near optimal; evolution takes advantage of seemingly harmful features of the environment to maximize the agent's foraging efficiency. The accessibility of evolved ANNs for a detailed inspection, together with the resemblance of some of the results to known findings from neurobiology, places evolved ANNs as an excellent candidate model for the study of structure and function relationship in complex nervous systems.

Evolution and analysis of model CPGs for walking: I. Dynamical modules

Can one develop an abstract description of the dynamics of pattern generators that provides quantitative insight into their operation? We explored this question by examining the dynamics of a model central pattern generator that was created using an evolutionary algorithm. We propose an abstract description based on the concept of a dynamical module, a set of neurons that simultaneously make their transitions from one quasistable state to another while the synaptic inputs that they receive from other neurons remain essentially constant, thus temporarily reducing the dimensionality of the circuit dynamics. Using the mathematical tools of dynamical systems theory, we describe a method for identifying dynamical modules and demonstrate that this concept can be used to quantitatively characterize constraints on neural architecture, account for phase durations, and predict the effects of parameter changes. Moreover, this abstract description reveals coordinated parameter changes that leave the overall circuit dynamics essentially unchanged. In a companion article we employ this abstract description to examine the relationship between general principles and individual variability in large populations of evolved model pattern generators.