A hexapod walker using a heterarchical architecture for action selection

The cognitive map debate in insects: A historical perspective on what is at stake

Though well established in mammals, the cognitive map hypothesis has engendered a decades-long, ongoing debate in insect navigation studies involving many of the field's most prominent researchers. In this paper, I situate the debate within the broader context of 20th century animal behavior research and argue that the debate persists because competing research groups are guided by different constellations of epistemic aims, theoretical commitments, preferred animal subjects, and investigative practices. The expanded history of the cognitive map provided in this paper shows that more is at stake in the cognitive map debate than the truth value of propositions characterizing insect cognition. What is at stake is the future direction of an extraordinarily productive tradition of insect navigation research stretching back to Karl von Frisch. Disciplinary labels like ethology, comparative psychology, and behaviorism became less relevant at the turn of the 21st century, but as I show, the different ways of knowing animals associated with these disciplines continue to motivate debates about animal cognition. This examination of scientific disagreement surrounding the cognitive map hypothesis also has significant consequences for philosophers' use of cognitive map research as a case study.

neuroWalknet, a controller for hexapod walking allowing for context dependent behavior

Decentralized control has been established as a key control principle in insect walking and has been successfully leveraged to account for a wide range of walking behaviors in the proposed neuroWalknet architecture. This controller allows for walking patterns at different velocities in both, forward and backward direction-quite similar to the behavior shown in stick insects-, for negotiation of curves, and for robustly dealing with various disturbances. While these simulations focus on the cooperation of different, decentrally controlled legs, here we consider a set of biological experiments not yet been tested by neuroWalknet, that focus on the function of the individual leg and are context dependent. These intraleg studies deal with four groups of interjoint reflexes. The reflexes are elicited by stimulation of the femoral chordotonal organ (fCO) or groups of campaniform sensilla (CS). Motor output signals are recorded from the alpha-joint, the beta-joint or the gamma-joint of the leg. Furthermore, the influence of these sensory inputs to artificially induced oscillations by application of pilocarpine has been studied. Although these biological data represent results obtained from different local reflexes in different contexts, they fit with and are embedded into the behavior shown by the global structure of neuroWalknet. In particular, a specific and intensively studied behavior, active reaction, has since long been assumed to represent a separate behavioral element, from which it is not clear why it occurs in some situations, but not in others. This question could now be explained as an emergent property of the holistic structure of neuroWalknet which has shown to be able to produce artificially elicited pilocarpine-driven oscillation that can be controlled by sensory input without the need of explicit innate CPG structures. As the simulation data result from a holistic system, further results were obtained that could be used as predictions to be tested in further biological experiments.

Sideways crab-walking is faster and more efficient than forward walking for a hexapod robot

Articulated legs enable the selection of robot gaits, including walking in different directions such as forward or sideways. For longer distances, the best gaits might maximize velocity or minimize the cost of transport (COT). While animals often have morphology suited to walking either forward (like insects) or sideways (like crabs), hexapod robots often default to forward walking. In this paper, we compare forward walking with crab-like sideways walking. To do this, a simple gait design method is introduced for determining forward and sideways gaits with equivalent body heights and step heights. Specifically, the frequency and stride lengths are tuned within reasonable constraints to find gaits that represent a robot's performance potential in terms of speed and energy cost. Experiments are performed in both dynamic simulation in Webots and a laboratory environment with our 18 degree-of-freedom hexapod robot, Sebastian. With the common three joint leg design, the results show that sideways walking is overall better (75% greater walking speed and 40% lower COT). The performance of sideways walking was better on both hard floors and granular media (dry play sand). This supports development of future crab-like walking robots for future applications. In future work, this approach may be used to develop nominal gaits without extensive optimization, and to explore whether the advantages of sideways walking persist for other hexapod designs.

Adaptive Interlimb Coordination Mechanism for Hexapod Locomotion Based on Active Load Sensing

Insects can flexibly coordinate their limbs to adapt to various locomotor conditions, e.g., complex environments, changes in locomotion speed, and leg amputation. An interesting aspect of insect locomotion is that the gait patterns are not necessarily stereotypical but are often highly variable, e.g., searching behavior to obtain stable footholds in complex environments. Several previous studies have focused on the mechanism for the emergence of variable limb coordination patterns. However, the proposed mechanisms are complicated and the essential mechanism underlying insect locomotion remains elusive. To address this issue, we proposed a simple mathematical model for the mechanism of variable interlimb coordination in insect locomotion. The key idea of the proposed model is “decentralized active load sensing,” wherein each limb actively moves and detects the reaction force from the ground to judge whether it plays a pivotal role in maintaining the steady support polygon. Based on active load sensing, each limb stays in the stance phase when the limb is necessary for body support. To evaluate the proposed model, we conducted simulation experiments using a hexapod robot. The results showed that the proposed simple mechanism allows the hexapod robot to exhibit typical gait patterns in response to the locomotion speed. Furthermore, the proposed mechanism improves the adaptability of the hexapod robot for leg amputations and lack of footholds by changing each limb's walking and searching behavior in a decentralized manner based on the physical interaction between the body and the environment.

NeuroVis: Real-Time Neural Information Measurement and Visualization of Embodied Neural Systems

Understanding the real-time dynamical mechanisms of neural systems remains a significant issue, preventing the development of efficient neural technology and user trust. This is because the mechanisms, involving various neural spatial-temporal ingredients [i.e., neural structure (NS), neural dynamics (ND), neural plasticity (NP), and neural memory (NM)], are too complex to interpret and analyze altogether. While advanced tools have been developed using explainable artificial intelligence (XAI), node-link diagram, topography map, and other visualization techniques, they still fail to monitor and visualize all of these neural ingredients online. Accordingly, we propose here for the first time "NeuroVis," real-time neural spatial-temporal information measurement and visualization, as a method/tool to measure temporal neural activities and their propagation throughout the network. By using this neural information along with the connection strength and plasticity, NeuroVis can visualize the NS, ND, NM, and NP via i) spatial 2D position and connection, ii) temporal color gradient, iii) connection thickness, and iv) temporal luminous intensity and change of connection thickness, respectively. This study presents three use cases of NeuroVis to evaluate its performance: i) function approximation using a modular neural network with recurrent and feedforward topologies together with supervised learning, ii) robot locomotion control and learning using the same modular network with reinforcement learning, and iii) robot locomotion control and adaptation using another larger-scale adaptive modular neural network. The use cases demonstrate how NeuroVis tracks and analyzes all neural ingredients of various (embodied) neural systems in real-time under the robot operating system (ROS) framework. To this end, it will offer the opportunity to better understand embodied dynamic neural information processes, boost efficient neural technology development, and enhance user trust.

Decentralized control and local information for robust and adaptive decentralized Deep Reinforcement Learning

Decentralization is a central characteristic of biological motor control that allows for fast responses relying on local sensory information. In contrast, the current trend of Deep Reinforcement Learning (DRL) based approaches to motor control follows a centralized paradigm using a single, holistic controller that has to untangle the whole input information space. This motivates to ask whether decentralization as seen in biological control architectures might also be beneficial for embodied sensori-motor control systems when using DRL. To answer this question, we provide an analysis and comparison of eight control architectures for adaptive locomotion that were derived for a four-legged agent, but with their degree of decentralization varying systematically between the extremes of fully centralized and fully decentralized. Our comparison shows that learning speed is significantly enhanced in distributed architectures-while still reaching the same high performance level of centralized architectures-due to smaller search spaces and local costs providing more focused information for learning. Second, we find an increased robustness of the learning process in the decentralized cases-it is less demanding to hyperparameter selection and less prone to becoming trapped in poor local minima. Finally, when examining generalization to uneven terrains-not used during training-we find best performance for an intermediate architecture that is decentralized, but integrates only local information from both neighboring legs. Together, these findings demonstrate beneficial effects of distributing control into decentralized units and relying on local information. This appears as a promising approach towards more robust DRL and better generalization towards adaptive behavior.

Achieving natural behavior in a robot using neurally inspired hierarchical perceptual control

Terrestrial locomotion presents tremendous computational challenges on account of the enormous degrees of freedom in legged animals, and complex, unpredictable properties of natural environments, including the body and its effectors, yet the nervous system can achieve locomotion with ease. Here we introduce a quadrupedal robot that is capable of posture control and goal-directed locomotion across uneven terrain. The control architecture is a hierarchical network of simple negative feedback control systems inspired by the organization of the vertebrate nervous system. This robot is capable of robust posture control and locomotion in novel environments with unpredictable disturbances. Unlike current robots, our robot does not use internal inverse and forward models, nor does it require any training in order to perform successfully in novel environments.

•

Inspired by a neural hierarchy with control of input at each level

•

Higher level specifies reference state for lower level

•

Successful posture control and locomotion despite unpredictable disturbances

•

No need for training or computation of inverse or forward kinematics

Decentralized control of insect walking: A simple neural network explains a wide range of behavioral and neurophysiological results

Controlling the six legs of an insect walking in an unpredictable environment is a challenging task, as many degrees of freedom have to be coordinated. Solutions proposed to deal with this task are usually based on the highly influential concept that (sensory-modulated) central pattern generators (CPG) are required to control the rhythmic movements of walking legs. Here, we investigate a different view. To this end, we introduce a sensor based controller operating on artificial neurons, being applied to a (simulated) insectoid robot required to exploit the "loop through the world" allowing for simplification of neural computation. We show that such a decentralized solution leads to adaptive behavior when facing uncertain environments which we demonstrate for a broad range of behaviors never dealt with in a single system by earlier approaches. This includes the ability to produce footfall patterns such as velocity dependent "tripod", "tetrapod", "pentapod" as well as various stable intermediate patterns as observed in stick insects and in Drosophila. These patterns are found to be stable against disturbances and when starting from various leg configurations. Our neuronal architecture easily allows for starting or interrupting a walk, all being difficult for CPG controlled solutions. Furthermore, negotiation of curves and walking on a treadmill with various treatments of individual legs is possible as well as backward walking and performing short steps. This approach can as well account for the neurophysiological results usually interpreted to support the idea that CPGs form the basis of walking, although our approach is not relying on explicit CPG-like structures. Application of CPGs may however be required for very fast walking. Our neuronal structure allows to pinpoint specific neurons known from various insect studies. Interestingly, specific common properties observed in both insects and crustaceans suggest a significance of our controller beyond the realm of insects.

A neuromechanical model exploring the role of the common inhibitor motor neuron in insect locomotion

In this work, we analyze a simplified, dynamical, closed-loop, neuromechanical simulation of insect joint control. We are specifically interested in two elements: (1) how slow muscle fibers may serve as temporal integrators of sensory feedback and (2) the role of common inhibitory (CI) motor neurons in resetting this integration when the commanded position changes, particularly during steady-state walking. Despite the simplicity of the model, we show that slow muscle fibers increase the accuracy of limb positioning, even for motions much shorter than the relaxation time of the fiber; this increase in accuracy is due to the slow dynamics of the fibers; the CI motor neuron plays a critical role in accelerating muscle relaxation when the limb moves to a new position; as in the animal, this architecture enables the control of the stance phase speed, independent of swing phase amplitude or duration, by changing the gain of sensory feedback to the stance phase muscles. We discuss how this relates to other models, and how it could be applied to robotic control.

A Synthetic Nervous System Controls a Simulated Cockroach

The purpose of this work is to better understand how animals control locomotion. This knowledge can then be applied to neuromechanical design to produce more capable and adaptable robot locomotion. To test hypotheses about animal motor control, we model animals and their nervous systems with dynamical simulations, which we call synthetic nervous systems (SNS). However, one major challenge is picking parameter values that produce the intended dynamics. This paper presents a design process that solves this problem without the need for global optimization. We test this method by selecting parameter values for SimRoach2, a dynamical model of a cockroach. Each leg joint is actuated by an antagonistic pair of Hill muscles. A distributed SNS was designed based on pathways known to exist in insects, as well as hypothetical pathways that produced insect-like motion. Each joint’s controller was designed to function as a proportional-integral (PI) feedback loop and tuned with numerical optimization. Once tuned, SimRoach2 walks through a simulated environment, with several cockroach-like features. A model with such reliable low-level performance is necessary to investigate more sophisticated locomotion patterns in the future.

Mantisbot is a robotic model of visually guided motion in the praying mantis

Insects use highly distributed nervous systems to process exteroception from head sensors, compare that information with state-based goals, and direct posture or locomotion toward those goals. To study how descending commands from brain centers produce coordinated, goal-directed motion in distributed nervous systems, we have constructed a conductance-based neural system for our robot MantisBot, a 29 degree-of-freedom, 13.3:1 scale praying mantis robot. Using the literature on mantis prey tracking and insect locomotion, we designed a hierarchical, distributed neural controller that establishes the goal, coordinates different joints, and executes prey-tracking motion. In our controller, brain networks perceive the location of prey and predict its future location, store this location in memory, and formulate descending commands for ballistic saccades like those seen in the animal. The descending commands are simple, indicating only 1) whether the robot should walk or stand still, and 2) the intended direction of motion. Each joint's controller uses the descending commands differently to alter sensory-motor interactions, changing the sensory pathways that coordinate the joints' central pattern generators into one cohesive motion. Experiments with one leg of MantisBot show that visual input produces simple descending commands that alter walking kinematics, change the walking direction in a predictable manner, enact reflex reversals when necessary, and can control both static posture and locomotion with the same network.

Adaptive Control Strategies for Interlimb Coordination in Legged Robots: A Review

Walking animals produce adaptive interlimb coordination during locomotion in accordance with their situation. Interlimb coordination is generated through the dynamic interactions of the neural system, the musculoskeletal system, and the environment, although the underlying mechanisms remain unclear. Recently, investigations of the adaptation mechanisms of living beings have attracted attention, and bio-inspired control systems based on neurophysiological findings regarding sensorimotor interactions are being developed for legged robots. In this review, we introduce adaptive interlimb coordination for legged robots induced by various factors (locomotion speed, environmental situation, body properties, and task). In addition, we show characteristic properties of adaptive interlimb coordination, such as gait hysteresis and different time-scale adaptations. We also discuss the underlying mechanisms and control strategies to achieve adaptive interlimb coordination and the design principle for the control system of legged robots.

Template for the neural control of directed stepping generalized to all legs of MantisBot

We previously developed a neural controller for one leg of our six-legged robot, MantisBot, that could direct locomotion toward a goal by modulating leg-local reflexes with simple descending commands from a head sensor. In this work, we successfully apply an automated method to tune the control network for all three pairs of legs of our hexapod robot MantisBot in only 90 s with a desktop computer. Each foot's motion changes appropriately as the body's intended direction of travel changes. In addition, several results from studies of walking insects are captured by this model. This paper both demonstrates the broad applicability of this control method for robots, and suggests neural mechanisms underlying observations from walking insects.

Development and Training of a Neural Controller for Hind Leg Walking in a Dog Robot

Animals dynamically adapt to varying terrain and small perturbations with remarkable ease. These adaptations arise from complex interactions between the environment and biomechanical and neural components of the animal's body and nervous system. Research into mammalian locomotion has resulted in several neural and neuro-mechanical models, some of which have been tested in simulation, but few “synthetic nervous systems” have been implemented in physical hardware models of animal systems. One reason is that the implementation into a physical system is not straightforward. For example, it is difficult to make robotic actuators and sensors that model those in the animal. Therefore, even if the sensorimotor circuits were known in great detail, those parameters would not be applicable and new parameter values must be found for the network in the robotic model of the animal. This manuscript demonstrates an automatic method for setting parameter values in a synthetic nervous system composed of non-spiking leaky integrator neuron models. This method works by first using a model of the system to determine required motor neuron activations to produce stable walking. Parameters in the neural system are then tuned systematically such that it produces similar activations to the desired pattern determined using expected sensory feedback. We demonstrate that the developed method successfully produces adaptive locomotion in the rear legs of a dog-like robot actuated by artificial muscles. Furthermore, the results support the validity of current models of mammalian locomotion. This research will serve as a basis for testing more complex locomotion controllers and for testing specific sensory pathways and biomechanical designs. Additionally, the developed method can be used to automatically adapt the neural controller for different mechanical designs such that it could be used to control different robotic systems.

ReaCog, a Minimal Cognitive Controller Based on Recruitment of Reactive Systems

It has often been stated that for a neuronal system to become a cognitive one, it has to be large enough. In contrast, we argue that a basic property of a cognitive system, namely the ability to plan ahead, can already be fulfilled by small neuronal systems. As a proof of concept, we propose an artificial neural network, termed reaCog, that, first, is able to deal with a specific domain of behavior (six-legged-walking). Second, we show how a minor expansion of this system enables the system to plan ahead and deploy existing behavioral elements in novel contexts in order to solve current problems. To this end, the system invents new solutions that are not possible for the reactive network. Rather these solutions result from new combinations of given memory elements. This faculty does not rely on a dedicated system being more or less independent of the reactive basis, but results from exploitation of the reactive basis by recruiting the lower-level control structures in a way that motor planning becomes possible as an internal simulation relying on internal representation being grounded in embodied experiences.

Effects of head and tail as swinging appendages on the dynamic walking performance of a quadruped robot

We designed a quadruped robot with a one-degree-of-freedom (1-DOF)-pitch head, a 1-DOF-roll tail, and 14 active DOFs in total, which are controlled via a central pattern generator (CPG) based on a Hopf oscillator. Head and tail movements are coupled to the leg movements with fixed phase differences. Experiments show that tail swinging in roll can equilibrate feet–ground reaction forces (GRF), reducing yaw errors and enabling the robot to maintain its direction when trotting. Head swing in pitch has the potential to increase flight time and stride length of the swinging legs and increase the robot's forward velocity when running in bounds.

Neuromechanical model of praying mantis explores the role of descending commands in pre-strike pivots

Praying mantises hunt by standing on their meso- and metathoracic legs and using them to rotate and translate (together, 'pivot') their bodies toward prey. We have developed a neuromechanical software model of the praying mantis Tenodera sinensis to use as a platform for testing postural controllers that the animal may use while hunting. Previous results showed that a feedforward model was insufficient for capturing the diversity of posture observed in the animal (Szczecinski et al 2014 Biomimetic and Biohybrid Syst. 3 296-307). Therefore we have expanded upon this model to make a flexible controller with feedback that more closely mimics the animal. The controller actuates 24 joints in the legs of a dynamical model to orient the head and translate the thorax toward prey. It is controlled by a simulation of nonspiking neurons assembled as a highly simplified version of networks that may exist in the mantid central complex and thoracic ganglia. Because of the distributed nature of these networks, we hypothesize that descending commands that orient the mantis toward prey may be simple direction-of-intent signals, which are turned into motor commands by the structure of low-level networks in the thoracic ganglia. We verify this through a series of experiments with the model. It captures the speed and range of mantid pivots as reported in other work (Yamawaki et al 2011 J. Insect Physiol. 57 1010-6). It is capable of pivoting toward prey from a variety of initial postures, as seen in the animal. Finally, we compare the model's joint kinematics during pivots to preliminary 3D kinematics collected from Tenodera.

Behavior control in the sensorimotor loop with short-term synaptic dynamics induced by self-regulating neurons

The behavior and skills of living systems depend on the distributed control provided by specialized and highly recurrent neural networks. Learning and memory in these systems is mediated by a set of adaptation mechanisms, known collectively as neuronal plasticity. Translating principles of recurrent neural control and plasticity to artificial agents has seen major strides, but is usually hampered by the complex interactions between the agent's body and its environment. One of the important standing issues is for the agent to support multiple stable states of behavior, so that its behavioral repertoire matches the requirements imposed by these interactions. The agent also must have the capacity to switch between these states in time scales that are comparable to those by which sensory stimulation varies. Achieving this requires a mechanism of short-term memory that allows the neurocontroller to keep track of the recent history of its input, which finds its biological counterpart in short-term synaptic plasticity. This issue is approached here by deriving synaptic dynamics in recurrent neural networks. Neurons are introduced as self-regulating units with a rich repertoire of dynamics. They exhibit homeostatic properties for certain parameter domains, which result in a set of stable states and the required short-term memory. They can also operate as oscillators, which allow them to surpass the level of activity imposed by their homeostatic operation conditions. Neural systems endowed with the derived synaptic dynamics can be utilized for the neural behavior control of autonomous mobile agents. The resulting behavior depends also on the underlying network structure, which is either engineered or developed by evolutionary techniques. The effectiveness of these self-regulating units is demonstrated by controlling locomotion of a hexapod with 18 degrees of freedom, and obstacle-avoidance of a wheel-driven robot.

A neuromechanical simulation of insect walking and transition to turning of the cockroach Blaberus discoidalis

A neuromechanical simulation of the cockroach Blaberus discoidalis was developed to explore changes in locomotion when the animal transitions from walking straight to turning. The simulation was based upon the biological data taken from three sources. Neural circuitry was adapted from the extensive literature primarily obtained from the studies of neural connections within thoracic ganglia of stick insect and adapted to cockroach. The 3D joint kinematic data on straight, forward walking for cockroach were taken from a paper that describes these movements in all joints simultaneously as the cockroach walked on an oiled-plate tether (Bender et al. in PloS one 5(10):1-15, 2010b). Joint kinematics for turning were only available for some leg joints (Mu and Ritzmann in J Comp Physiol A Neuroethol Sens Neural Behav Physiol 191(11):1037-54, 2005) and thus had to be obtained using the methods that were applied for straight walking by Bender et al. (PloS one 5(10):1-15, 2010b). Once walking, inside turning, and outside turning were characterized, phase and amplitude changes for each joint of each leg were quantified. Apparent reflex reversals and joint activity changes were used to modify sensory coupling pathways between the CPG at each joint of the simulation. Oiled-plate experiments in simulation produced tarsus trajectories in stance similar to those seen in the animal. Simulations including forces that would be experienced if the insect was walking freely (i.e., weight support and friction) again produced similar results. These data were not considered during the design of the simulation, suggesting that the simulation captures some key underlying the principles of walking, turning, and transitioning in the cockroach. In addition, since the nervous system was modeled with realistic neuron models, biologically plausible reflex reversals are simulated, motivating future neurobiological research.

Biologically-inspired adaptive obstacle negotiation behavior of hexapod robots

Neurobiological studies have shown that insects are able to adapt leg movements and posture for obstacle negotiation in changing environments. Moreover, the distance to an obstacle where an insect begins to climb is found to be a major parameter for successful obstacle negotiation. Inspired by these findings, we present an adaptive neural control mechanism for obstacle negotiation behavior in hexapod robots. It combines locomotion control, backbone joint control, local leg reflexes, and neural learning. While the first three components generate locomotion including walking and climbing, the neural learning mechanism allows the robot to adapt its behavior for obstacle negotiation with respect to changing conditions, e.g., variable obstacle heights and different walking gaits. By successfully learning the association of an early, predictive signal (conditioned stimulus, CS) and a late, reflex signal (unconditioned stimulus, UCS), both provided by ultrasonic sensors at the front of the robot, the robot can autonomously find an appropriate distance from an obstacle to initiate climbing. The adaptive neural control was developed and tested first on a physical robot simulation, and was then successfully transferred to a real hexapod robot, called AMOS II. The results show that the robot can efficiently negotiate obstacles with a height up to 85% of the robot's leg length in simulation and 75% in a real environment.

Walknet, a bio-inspired controller for hexapod walking

Walknet comprises an artificial neural network that allows for the simulation of a considerable amount of behavioral data obtained from walking and standing stick insects. It has been tested by kinematic and dynamic simulations as well as on a number of six-legged robots. Over the years, various different expansions of this network have been provided leading to different versions of Walknet. This review summarizes the most important biological findings described by Walknet and how they can be simulated. Walknet shows how a number of properties observed in insects may emerge from a decentralized architecture. Examples are the continuum of so-called "gaits," coordination of up to 18 leg joints during stance when walking forward or backward over uneven surfaces and negotiation of curves, dealing with leg loss, as well as being able following motion trajectories without explicit precalculation. The different Walknet versions are compared to other approaches describing insect-inspired hexapod walking. Finally, we briefly address the ability of this decentralized reactive controller to form the basis for the simulation of higher-level cognitive faculties exceeding the capabilities of insects.