A modular artificial neural net for controlling a six-legged walking system

The manifold structure of limb coordination in walking Drosophila

Terrestrial locomotion requires animals to coordinate their limb movements to efficiently traverse their environment. While previous studies in hexapods have reported that limb coordination patterns can vary substantially, the structure of this variability is not yet well understood. Here, we characterized the symmetric and asymmetric components of variation in walking kinematics in the genetic model organism Drosophila. We found that Drosophila use a single continuum of coordination patterns without evidence for preferred configurations. Spontaneous symmetric variability was associated with modulation of a single control parameter-stance duration-while asymmetric variability consisted of small, limb-specific modulations along multiple dimensions of the underlying symmetric pattern. Commands that modulated walking speed, originating from artificial neural activation or from the visual system, evoked modulations consistent with spontaneous behavior. Our findings suggest that Drosophila employ a low-dimensional control architecture, which provides a framework for understanding the neural circuits that regulate hexapod legged locomotion.

Swing Velocity Profiles of Small Limbs Can Arise from Transient Passive Torques of the Antagonist Muscle Alone

In large limbs, changing motor neuron activity typically controls within-movement velocity. For example, sequential agonist-antagonist-agonist motor neuron firing typically underlies the slowing often present at the end of human reaches. In physiological movements of large limbs, antagonistic muscle passive torque is generally negligible. In small limbs, alternatively, passive torques can determine limb rest position, generate restoring movements to it, and decrease agonist-generated movement amplitude and velocity maxima. These observations suggest that, in small limbs, passive forces might also control velocity changes within movements. We investigated this issue in stick insect middle leg femur-tibia (FT) joint. During swing, the FT joint extensor muscle actively shortens and the flexor muscle passively lengthens. As in human reaching, after its initial acceleration, FT joint velocity continuously decreases. We measured flexor passive forces during imposed stretches spanning the ranges of FT joint angles, angular velocities, and movement amplitudes present in leg swings. The viscoelastic "transient" passive force that occurs during and soon after stretch depended on all three variables and could be tens of times larger than the "steady-state" passive force commonly measured long after stretch end. We combined these data, the flexor and extensor moment arms, and an existing extensor model to simulate FT joint swing. To measure only passive (flexor) muscle-dependent effects, we used constant extensor activations in these simulations. In simulations using data from ten flexor muscles, flexor passive torque could always produce swings with, after swing initiation, continuously decreasing velocities. Antagonist muscle passive torques alone can thus control within-movement velocity.

Controlling legs for locomotion-insights from robotics and neurobiology

Walking is the most common terrestrial form of locomotion in animals. Its great versatility and flexibility has led to many attempts at building walking machines with similar capabilities. The control of walking is an active research area both in neurobiology and robotics, with a large and growing body of work. This paper gives an overview of the current knowledge on the control of legged locomotion in animals and machines and attempts to give walking control researchers from biology and robotics an overview of the current knowledge in both fields. We try to summarize the knowledge on the neurobiological basis of walking control in animals, emphasizing common principles seen in different species. In a section on walking robots, we review common approaches to walking controller design with a slight emphasis on biped walking control. We show where parallels between robotic and neurobiological walking controllers exist and how robotics and biology may benefit from each other. Finally, we discuss where research in the two fields diverges and suggest ways to bridge these gaps.

Walknet, a bio-inspired controller for hexapod walking

Walknet comprises an artificial neural network that allows for the simulation of a considerable amount of behavioral data obtained from walking and standing stick insects. It has been tested by kinematic and dynamic simulations as well as on a number of six-legged robots. Over the years, various different expansions of this network have been provided leading to different versions of Walknet. This review summarizes the most important biological findings described by Walknet and how they can be simulated. Walknet shows how a number of properties observed in insects may emerge from a decentralized architecture. Examples are the continuum of so-called "gaits," coordination of up to 18 leg joints during stance when walking forward or backward over uneven surfaces and negotiation of curves, dealing with leg loss, as well as being able following motion trajectories without explicit precalculation. The different Walknet versions are compared to other approaches describing insect-inspired hexapod walking. Finally, we briefly address the ability of this decentralized reactive controller to form the basis for the simulation of higher-level cognitive faculties exceeding the capabilities of insects.

Dominance of local sensory signals over inter-segmental effects in a motor system: experiments

Legged locomotion requires that information local to one leg, and inter-segmental signals coming from the other legs are processed appropriately to establish a coordinated walking pattern.However, very little is known about the relative importance of local and inter-segmental signals when they converge upon the central pattern generators (CPGs) of different leg joints.We investigated this question on the CPG of the middle leg coxa–trochanter (CTr)-joint of the stick insect which is responsible for lifting and lowering the leg.We used a semi-intact preparation with an intact front leg stepping on a treadmill, and simultaneously stimulated load sensors of the middle leg.We found that middle leg load signals induce bursts in the middle leg depressor motoneurons(MNs). The same local load signals could also elicit rhythmic activity in the CPG of the middle leg CTr-joint when the stimulation of middle leg load sensors coincided with front leg stepping. However, the influence of front leg stepping was generally weak such that front leg stepping alone was only rarely accompanied by switching between middle leg levator and depressor MN activity. We therefore conclude that the impact of the local sensory signals on the levator–depressor motor system is stronger than the inter-segmental influence through front leg stepping.

On the role of sensory feedbacks in rowat-selverston CpG to improve robot legged locomotion

This paper presents the use of Rowat and Selverston-type of central pattern generator (CPG) to control locomotion. It focuses on the role of afferent exteroceptive and proprioceptive signals in the dynamic phase synchronization in CPG legged robots. The sensori-motor neural network architecture is evaluated to control a two-joint planar robot leg that slips on a rail. Then, the closed loop between the CPG and the mechanical system allows to study the modulation of rhythmic patterns and the effect of the sensing loop via sensory neurons during the locomotion task. Firstly simulations show that the proposed architecture easily allows to modulate rhythmic patterns of the leg, and therefore the velocity of the robot. Secondly, simulations show that sensori-feedbacks from foot/ground contact of the leg make the hip velocity smoother and larger. The results show that the Rowat–Selverston-type CPG with sensory feedbacks is an effective choice for building adaptive neural CPGs for legged robots.

Sensory feedback induced by front-leg stepping entrains the activity of central pattern generators in caudal segments of the stick insect walking system

Legged locomotion results from a combination of central pattern generating network (CPG) activity and intralimb and interlimb sensory feedback. Data on the neural basis of interlimb coordination are very limited. We investigated here the influence of stepping in one leg on the activities of neighboring-leg thorax-coxa (TC) joint CPGs in the stick insect (Carausius morosus). We used a new approach combining single-leg stepping with pharmacological activation of segmental CPGs, sensory stimulation, and additional stepping legs. Stepping of a single front leg could activate the ipsilateral mesothoracic TC CPG. Activation of the metathoracic TC CPG required that both ipsilateral front and middle legs were present and that one of these legs was stepping. Unlike the situation in real walking, ipsilateral mesothoracic and metathoracic TC CPGs activated by front-leg stepping fired in phase with the front-leg stepping. Local (intralimb) sensory feedback from load sensors could override this intersegmental influence of front-leg stepping, shifting retractor motoneuron activity relative to the front-leg step cycle and thereby uncoupling them from front-leg stepping. These data suggest that front-leg stepping in isolation would result in in-phase activity of all ipsilateral legs, and functional stepping gaits (in which the three ipsilateral legs do not step in synchrony) emerge because of local load sensory feedback overriding this in-phase influence.

A dynamic model of thoracic differentiation for the control of turning in the stick insect

Leg movements of stick insects (Carausius morosus) making turns towards visual targets are examined in detail, and a dynamic model of this behaviour is proposed. Initial results suggest that front legs shape most of the body trajectory, while the middle and hind legs just follow external forces (Rosano H, Webb B, in The control of turning in real and simulated stick insects, vol. 4095, pp 145-156, 2006). However, some limitations of this explanation and dissimilarities in the turning behaviour of the insect and the model were found. A second set of behavioural experiments was made by blocking front tarsi to further investigate the active role of the other legs for the control of turning. The results indicate that it is necessary to have different roles for each pair of legs to replicate insect behaviour. We demonstrate that the rear legs actively rotate the body while the middle legs move sideways tangentially to the hind inner leg. Furthermore, we show that on average the middle inner and hind outer leg contribute to turning while the middle outer leg and hind inner leg oppose body rotation. These behavioural results are incorporated into a 3D dynamic robot simulation. We show that the simulation can now replicate more precisely the turns made by the stick insect.

Insect walking is based on a decentralized architecture revealing a simple and robust controller

Control of walking in rugged terrain requires one to incorporate different issues, such as the mechanical properties of legs and muscles, the neuronal control structures for the single leg, the mechanics and neuronal control structures for the coordination between legs, as well as central decisions that are based on external information and on internal states. Walking in predictable environments and fast running, to a large degree, rely on muscle mechanics. Conversely, slow walking in unpredictable terrain, e.g. climbing in rugged structures, has to rely on neuronal systems that monitor and intelligently react to specific properties of the environment. An arthropod model system that shows the latter abilities is the stick insect, based on which this review will be focused. An insect, when moving its six legs, has to control 18 joints, three per leg, and therefore has to control 18 degrees of freedom (d.f.). As the body position in space is determined by 6 d.f. only, there are 12 d.f. open to be selected. Therefore, a fundamental problem is as to how these extra d.f. are controlled. Based mainly on behavioural experiments and simulation studies, but also including neurophysiological results, the following control structures have been revealed. Legs act as basically independent systems. The quasi-rhythmic movement of the individual leg can be described to result from a structure that exploits mechanical coupling of the legs via the ground and the body. Furthermore, neuronally mediated influences act locally between neighbouring legs, leading to the emergence of insect-type gaits. The underlying controller can be described as a free gait controller. Cooperation of the legs being in stance mode is assumed to be based on mechanical coupling plus local positive feedback controllers. These controllers, acting on individual leg joints, transform a passive displacement of a joint into an active movement, generating synergistic assistance reflexes in all mechanically coupled joints. This architecture is summarized in the form of the artificial neural network, Walknet, that is heavily dependent on sensory feedback at the proprioceptive level. Exteroceptive feedback is exploited for global decisions, such as the walking direction and velocity.

Experiments and models of sensorimotor interactions during locomotion

During locomotion sensory information from cutaneous and muscle receptors is continuously integrated with the locomotor central pattern generator (CPG) to generate an appropriate motor output to meet the demands of the environment. Sensory signals from peripheral receptors can strongly impact the timing and amplitude of locomotor activity. This sensory information is gated centrally depending on the state of the system (i.e., rest vs. locomotion) but is also modulated according to the phase of a given task. Consequently, if one is to devise biologically relevant walking models it is imperative that these sensorimotor interactions at the spinal level be incorporated into the control system.

Assessing sensory function in locomotor systems using neuro-mechanical simulations

Computer simulations are being used increasingly to gain an understanding of the complex interactions between the neuronal, sensory, muscular and mechanical components of locomotor systems. Recent neuro-mechanical simulations of walking in humans, cats and insects, and of swimming in lampreys, have provided new information on the functional role of specific groups of sensory receptors in regulating locomotion. As we discuss in this review, these studies also make it clear that a full understanding of the neural and mechanical mechanisms that underlie locomotion can be achieved only by using simulations in parallel with physiological investigations. The widespread implementation of this approach would be enhanced by the development of freely available and easy-to-use software tools.

Control of swing movement: influences of differently shaped substrate

Stick insects were studied while walking on different substrates. The trajectories of swing movements are recorded. The starting position of a swing movement is varied in vertical direction and in the direction parallel to body long axis. The trajectories found cannot be predicted by an ANN (Swingnet1) proposed earlier to describe swing movements. However, a modified network (Swingnet2) allows for a satisfying description of the behavioral results. Walking on a narrow treadwheel leads to different swing trajectories compared to walking on a broad treadwheel. These trajectories cannot be described by Swingnet1, too. The form of the swing trajectory may depend on the direction of the force vector by which the leg acts on the ground in the preceding stance. Based on this assumption, an alternative hypothesis (Swingnet3) is proposed that can quantitatively describe all results of our experiment. When stick insects walk from a wide to a narrow substrate, transition between different swing trajectories does not change gradually over time. Rather, the form of the trajectory is determined by the current sensory input of the leg on a step-to-step basis. Finally, four different avoidance reflexes and their implementation into swing movements are investigated and described by a quantitative simulation.

Schema-based learning of adaptable and flexible prey-catching in anurans I. The basic architecture

A motor action often involves the coordination of several motor synergies and requires flexible adjustment of the ongoing execution based on feedback signals. To elucidate the neural mechanisms underlying the construction and selection of motor synergies, we study prey-capture in anurans. Experimental data demonstrate the intricate interaction between different motor synergies, including the interplay of their afferent feedback signals (Weerasuriya 1991; Anderson and Nishikawa 1996). Such data provide insights for the general issues concerning two-way information flow between sensory centers, motor circuits and periphery in motor coordination. We show how different afferent feedback signals about the status of the different components of the motor apparatus play a critical role in motor control as well as in learning. This paper, along with its companion paper, extend the model by Liaw et al. (1994) by integrating a number of different motor pattern generators, different types of afferent feedback, as well as the corresponding control structure within an adaptive framework we call Schema-Based Learning. We develop a model of the different MPGs involved in prey-catching as a vehicle to investigate the following questions: What are the characteristic features of the activity of a single muscle? How can these features be controlled by the premotor circuit? What are the strategies employed to generate and synchronize motor synergies? What is the role of afferent feedback in shaping the activity of a MPG? How can several MPGs share the same underlying circuitry and yet give rise to different motor patterns under different input conditions? In the companion paper we also extend the model by incorporating learning components that give rise to more flexible, adaptable and robust behaviors. To show these aspects we incorporate studies on experiments on lesions and the learning processes that allow the animal to recover its proper functioning.

Behaviour-based modelling of hexapod locomotion: linking biology and technical application

Walking in insects and most six-legged robots requires simultaneous control of up to 18 joints. Moreover, the number of joints that are mechanically coupled via body and ground varies from one moment to the next, and external conditions such as friction, compliance and slope of the substrate are often unpredictable. Thus, walking behaviour requires adaptive, context-dependent control of many degrees of freedom. As a consequence, modelling legged locomotion addresses many aspects of any motor behaviour in general. Based on results from behavioural experiments on arthropods, we describe a kinematic model of hexapod walking: the distributed artificial neural network controller walknet. Conceptually, the model addresses three basic problems in legged locomotion. (I) First, coordination of several legs requires coupling between the step cycles of adjacent legs, optimising synergistic propulsion, but ensuring stability through flexible adjustment to external disturbances. A set of behaviourally derived leg coordination rules can account for decentralised generation of different gaits, and allows stable walking of the insect model as well as of a number of legged robots. (II) Second, a wide range of different leg movements must be possible, e.g. to search for foothold, grasp for objects or groom the body surface. We present a simple neural network controller that can simulate targeted swing trajectories, obstacle avoidance reflexes and cyclic searching-movements. (III) Third, control of mechanically coupled joints of the legs in stance is achieved by exploiting the physical interactions between body, legs and substrate. A local positive displacement feedback, acting on individual leg joints, transforms passive displacement of a joint into active movement, generating synergistic assistance reflexes in all mechanically coupled joints.

A brain perspective on language mechanisms: from discrete neuronal ensembles to serial order

Language is constituted by discrete building blocks, sounds and words, which can be concatenated according to serial order principles. The neurobiological organization of these building blocks, in particular words, has been illuminated by recent metabolic and neurophysiological imaging studies. When humans process words of different kinds, various sets of cortical areas have been found to become active differentially. The old concept of two language centers processing all words alike must therefore be replaced by a model according to which words are organized as discrete distributed neuron ensembles that differ in their cortical topographies. The meaning of a word, more precisely, aspects of its reference, may be crucial for determining which set of cortical areas becomes involved in its processing. Whereas the serial order of sounds constituting a word may be established by serially aligned sets of neurons called synfire chains, different mechanisms are necessary for establishing word order in sentences. The serial order of words may be organized by higher-order neuronal sets, called sequence detectors here, which are being activated by sequential excitation of neuronal sets representing words. Sets of sequence detectors are proposed to process aspects of the syntactic information contained in a sentence. Other syntactic rules can be related to general features of the dynamics of cortical activation and deactivation. These postulates about the brain mechanisms of language, which are rooted in principles known from neuroanatomy and neurophysiology, may provide a framework for theory-driven neuroscientific research on language.

Robots in invertebrate neuroscience

Can we now build artificial animals? A combination of robot technology and neuroethological knowledge is enabling the development of realistic physical models of biological systems. And such systems are not only of interest to engineers. By exploring identified neural control circuits in the appropriate functional and environmental context, new insights are also provided to biologists.

Impact of movement and movement-related feedback on the lamprey central pattern generator for locomotion

A semi-reduced, minimally restrained lamprey preparation was used to investigate the impact of movement and movement-related feedback during d-glutamate-induced locomotion. The preparation consisted of the trunk alone with the spinal cord exposed to the bathing solution. Two conditions were compared using electromyography or nerve recording: (i) muscle and spinal cord, (ii) spinal cord alone supported by the notochord. Compared with the isolated spinal cord, movement in the presence of muscle consistently and significantly increased the frequency of the motor output and reduced the phase delay among the segments. In moving preparations, coupling among the segments was reduced by two staggered hemisections to permit the strength and direction of intersegmental coupling to be estimated. The estimates revealed that movement increased the total intersegmental coupling strength and increased the proportion of the coupling that was descending over those of the isolated spinal cord.

The effects on the phase and frequency of bursting can be explained in the light of the excitation evoked by bending that we have reported previously. Thus, we demonstrate that movement and movement-related feedback that arise from spinally induced motor patterns can alter the form of the movement and the functional coupling strength among the segments of the lamprey spinal cord.

Stereotypic leg searching movements in the stick insect: kinematic analysis, behavioural context and simulation

Insects are capable of efficient locomotion in a spatially complex environment, such as walking on a forest floor or climbing in a bush. One behavioural mechanism underlying such adaptability is the searching movement that occurs after loss of ground contact. Here, the kinematic sequence of leg searching movements of the stick insect Carausius morosus is analysed. Searching movements are shown to be stereotypic rhythmic movement sequences consisting of several loops. The typical loop structure allows the mean tarsus trajectory to be calculated using a feature-based averaging procedure. Thus, it is possible to describe the common underlying structure of this movement pattern. Phase relationships between joint angles, analysed for searching front legs, indicate a central role for the thorax-coxa joint in searching movements. Accordingly, the stereotyped loop structure of searching differs between front-, middle- and hindlegs, with leg-specific patterns being caused by differing protraction/retraction movements in the thorax-coxa joint. A simple artificial neural network that had originally been devised to generate simple swing movements allows two essential features of empirical searching trajectories to be simulated: (i) cyclic movements and (ii) the smooth transition into a search trajectory as a non-terminated swing movement. It is possible to generate several loops of a middle-leg search, but the precise size and shape of the loops fall short of a real-life approximation. Incorporation of front-leg retraction or hind-leg protraction during searching will also require an extension to the current model. Finally, front-leg searching occurs simultaneously with antennal movements. Also, because leg searching movements are a local behaviour, the legs remaining on the ground continue their stance phase, causing a forward shift of the body, including the searching leg. As a result of this shift, the centre of the searched space is close to the anterior extreme position of the tarsus during walking, representing the location of most likely ground contact according to past experience. Therefore, the behavioural relevance of searching movements arises from the combined actions of several limbs.

Load-regulating mechanisms in gait and posture: comparative aspects

How is load sensed by receptors, and how is this sensory information used to guide locomotion? Many insights in this domain have evolved from comparative studies since it has been realized that basic principles concerning load sensing and regulation can be found in a wide variety of animals, both vertebrate and invertebrate. Feedback about load is not only derived from specific load receptors but also from other types of receptors that previously were thought to have other functions. In the central nervous system of many species, a convergence is found between specific and nonspecific load receptors. Furthermore, feedback from load receptors onto central circuits involved in the generation of rhythmic locomotor output is commonly found. During the stance phase, afferent activity from various load detectors can activate the extensor part in such circuits, thereby providing reinforcing force feedback. At the same time, the flexion is suppressed. The functional role of this arrangement is that activity in antigravity muscles is promoted while the onset of the next flexion is delayed as long as the limb is loaded. This type of reinforcing force feedback is present during gait but absent in the immoble resting animal.

Locomotor strategy for pedaling: muscle groups and biomechanical functions

A group of coexcited muscles alternating with another group is a common element of motor control, including locomotor pattern generation. This study used computer simulation to investigate human pedaling with each muscle assigned at times to a group. Simulations were generated by applying patterns of muscle excitations to a musculoskeletal model that includes the dynamic properties of the muscles, the limb segments, and the crank load. Raasch et al. showed that electromyograms, pedal reaction forces, and limb and crank kinematics recorded during maximum-speed start-up pedaling could be replicated with two signals controlling the excitation of four muscle groups (1 group alternating with another to form a pair). Here a four-muscle-group control also is shown to replicate steady pedaling. However, simulations show that three signals controlling six muscle groups (i.e., 3 pairs) is much more biomechanically robust, such that a wide variety of forward and backward pedaling tasks can be executed well. We found the biomechanical functions necessary for pedaling, and how these functions can be executed by the muscle groups. Specifically, the phasing of two pairs with respect to limb extension and flexion and the transitions between extension and flexion do not change with pedaling direction. One pair of groups (uniarticular hip and knee extensors alternating with their anatomic antagonists) generates the energy required for limb and crank propulsion during limb extension and flexion, respectively. In the second pair, the ankle plantarflexors transfer the energy from the limb inertia to the crank during the latter part of limb extension and the subsequent limb extension-to-flexion transition. The dorsiflexors alternate with the plantarflexors. The phasing of the third pair (the biarticular thigh muscles) reverses with pedaling direction. In forward pedaling, the hamstring is excited during the extension-to-flexion transition and in backward pedaling during the opposite transition. In both cases hamstrings propel the crank posteriorly through the transition. Rectus femoris alternates with hamstrings and propels the crank anteriorly through the transitions. With three control signals, one for each pair of groups, different cadences (or power outputs) can be achieved by adjusting the overall excitatory drive to the pattern generating elements, and different pedaling goals (e.g., smooth, or energy-efficient pedaling; 1- or 2-legged pedaling) by adjusting the relative excitation levels among the muscle groups. These six muscle groups are suggested to be elements of a general strategy for pedaling control, which may be generally applicable to other human locomotor tasks.

Oscillatory network controlling six-legged locomotion. Optimization of model parameters

The model of a legged locomotory system is optimized to ensure stable motion, reliable with respect to different initial conditions. The cost function suggested is based on the frequency of the model's loss of stability evaluated for randomly chosen initial leg positions. The optimized model can start from the majority of allowed leg configurations, demonstrating stable walking at low and moderate speeds. Furthermore, an acceleration procedure is designed to permit the model to pick up practically every speed and then walk successfully.