Flexible Coordination of Flexible Limbs: Decentralized Control Scheme for Inter- and Intra-Limb Coordination in Brittle Stars' Locomotion

A methodological exploration to study 2D arm kinematics in Ophiuroidea (Echinodermata)

Brittle stars, unlike most other echinoderms, do not use their small tube feet for locomotion but instead use their flexible arms to produce a rowing or reverse rowing movement. They are among the fastest-moving echinoderms with the ability of complex locomotory behaviors. Considering the high species diversity and variability in morphotypes, a proper understanding of intra- and interspecies variation in arm flexibility and movement is lacking. This study focuses on the exploration of the methods to investigate the variability in brittle star locomotion and individual arm use. We performed a two-dimensional (2D) image processing on horizontal movement only. The result indicated that sinuosity, disc displacement and arm angle are important parameters to interpret ophiuroid locomotion. A dedicated Python script to calculate the studied movement parameters and visualize the results applicable to all 5-armed brittle stars was developed. These results can serve as the basis for further research in robotics inspired by brittle star locomotion.

Modularity in Nervous Systems—a Key to Efficient Adaptivity for Deep Reinforcement Learning

Modularity as observed in biological systems has proven valuable for guiding classical motor theories towards good answers about action selection and execution. New challenges arise when we turn to learning: Trying to scale current computational models, such as deep reinforcement learning (DRL), to action spaces, input dimensions, and time horizons seen in biological systems still faces severe obstacles unless vast amounts of training data are available. This leads to the question: does biological modularity also hold an important key for better answers to obtain efficient adaptivity for deep reinforcement learning? We review biological experimental work on modularity in biological motor control and link this with current examples of (deep) RL approaches. Analyzing outcomes of simulation studies, we show that these approaches benefit from forms of modularization as found in biological systems. We identify three different strands of modularity exhibited in biological control systems. Two of them—modularity in state (i) and in action (ii) spaces—appear as a consequence of local interconnectivity (as in reflexes) and are often modulated by higher levels in a control hierarchy. A third strand arises from chunking of action elements along a (iii) temporal dimension. Usually interacting in an overarching spatio-temporal hierarchy of the overall system, the three strands offer major “factors” decomposing the entire modularity structure. We conclude that modularity with its above strands can provide an effective prior for DRL approaches to speed up learning considerably and making learned controllers more robust and adaptive.

Decentralized control and local information for robust and adaptive decentralized Deep Reinforcement Learning

Decentralization is a central characteristic of biological motor control that allows for fast responses relying on local sensory information. In contrast, the current trend of Deep Reinforcement Learning (DRL) based approaches to motor control follows a centralized paradigm using a single, holistic controller that has to untangle the whole input information space. This motivates to ask whether decentralization as seen in biological control architectures might also be beneficial for embodied sensori-motor control systems when using DRL. To answer this question, we provide an analysis and comparison of eight control architectures for adaptive locomotion that were derived for a four-legged agent, but with their degree of decentralization varying systematically between the extremes of fully centralized and fully decentralized. Our comparison shows that learning speed is significantly enhanced in distributed architectures-while still reaching the same high performance level of centralized architectures-due to smaller search spaces and local costs providing more focused information for learning. Second, we find an increased robustness of the learning process in the decentralized cases-it is less demanding to hyperparameter selection and less prone to becoming trapped in poor local minima. Finally, when examining generalization to uneven terrains-not used during training-we find best performance for an intermediate architecture that is decentralized, but integrates only local information from both neighboring legs. Together, these findings demonstrate beneficial effects of distributing control into decentralized units and relying on local information. This appears as a promising approach towards more robust DRL and better generalization towards adaptive behavior.

Resilience of neural networks for locomotion

Locomotion is an essential behaviour for the survival of all animals. The neural circuitry underlying locomotion is therefore highly robust to a wide variety of perturbations, including injury and abrupt changes in the environment. In the short term, fault tolerance in neural networks allows locomotion to persist immediately after mild to moderate injury. In the longer term, in many invertebrates and vertebrates, neural reorganization including anatomical regeneration can restore locomotion after severe perturbations that initially caused paralysis. Despite decades of research, very little is known about the mechanisms underlying locomotor resilience at the level of the underlying neural circuits and coordination of central pattern generators (CPGs). Undulatory locomotion is an ideal behaviour for exploring principles of circuit organization, neural control and resilience of locomotion, offering a number of unique advantages including experimental accessibility and modelling tractability. In comparing three well-characterized undulatory swimmers, lampreys, larval zebrafish and Caenorhabditis elegans, we find similarities in the manifestation of locomotor resilience. To advance our understanding, we propose a comparative approach, integrating experimental and modelling studies, that will allow the field to begin identifying shared and distinct solutions for overcoming perturbations to persist in orchestrating this essential behaviour.

Body and Tail Coordination in the Bluespot Salamander (Ambystoma laterale) During Limb Regeneration

Animals are incredibly good at adapting to changes in their environment, a trait envied by most roboticists. Many animals use different gaits to seamlessly transition between land and water and move through non-uniform terrains. In addition to adjusting to changes in their environment, animals can adjust their locomotion to deal with missing or regenerating limbs. Salamanders are an amphibious group of animals that can regenerate limbs, tails, and even parts of the spinal cord in some species. After the loss of a limb, the salamander successfully adjusts to constantly changing morphology as it regenerates the missing part. This quality is of particular interest to roboticists looking to design devices that can adapt to missing or malfunctioning components. While walking, an intact salamander uses its limbs, body, and tail to propel itself along the ground. Its body and tail are coordinated in a distinctive wave-like pattern. Understanding how their bending kinematics change as they regrow lost limbs would provide important information to roboticists designing amphibious machines meant to navigate through unpredictable and diverse terrain. We amputated both hindlimbs of blue-spotted salamanders (Ambystoma laterale) and measured their body and tail kinematics as the limbs regenerated. We quantified the change in the body wave over time and compared them to an amphibious fish species, Polypterus senegalus. We found that salamanders in the early stages of regeneration shift their kinematics, mostly around their pectoral girdle, where there is a local increase in undulation frequency. Amputated salamanders also show a reduced range of preferred walking speeds and an increase in the number of bending waves along the body. This work could assist roboticists working on terrestrial locomotion and water to land transitions.

General Distributed Neural Control and Sensory Adaptation for Self-Organized Locomotion and Fast Adaptation to Damage of Walking Robots

Walking animals such as invertebrates can effectively perform self-organized and robust locomotion. They can also quickly adapt their gait to deal with injury or damage. Such a complex achievement is mainly performed via coordination between the legs, commonly known as interlimb coordination. Several components underlying the interlimb coordination process (like distributed neural control circuits, local sensory feedback, and body-environment interactions during movement) have been recently identified and applied to the control systems of walking robots. However, while the sensory pathways of biological systems are plastic and can be continuously readjusted (referred to as sensory adaptation), those implemented on robots are typically static. They first need to be manually adjusted or optimized offline to obtain stable locomotion. In this study, we introduce a fast learning mechanism for online sensory adaptation. It can continuously adjust the strength of sensory pathways, thereby introducing flexible plasticity into the connections between sensory feedback and neural control circuits. We combine the sensory adaptation mechanism with distributed neural control circuits to acquire the adaptive and robust interlimb coordination of walking robots. This novel approach is also general and flexible. It can automatically adapt to different walking robots and allow them to perform stable self-organized locomotion as well as quickly deal with damage within a few walking steps. The adaptation of plasticity after damage or injury is considered here as lesion-induced plasticity. We validated our adaptive interlimb coordination approach with continuous online sensory adaptation on simulated 4-, 6-, 8-, and 20-legged robots. This study not only proposes an adaptive neural control system for artificial walking systems but also offers a possibility of invertebrate nervous systems with flexible plasticity for locomotion and adaptation to injury.

The structural origins of brittle star arm kinematics: An integrated tomographic, additive manufacturing, and parametric modeling-based approach

Brittle stars are known for the high flexibility of their arms, a characteristic required for locomotion, food grasping, and for holding onto a great diversity of substrates. Their high agility is facilitated by the numerous discrete skeletal elements (ossicles) running through the center of each arm and embedded in the skin. While much has been learned regarding the structural diversity of these ossicles, which are important characters for taxonomic purposes, their impact on the arms' range of motion, by contrast, is poorly understood. In the present study, we set out to investigate how ossicle morphology and skeletal organization affect the flexibility of brittle star arms. Here, we present the results of an in-depth analysis of three brittle star species (Ophioplocus esmarki, Ophiopteris papillosa, and Ophiothrix spiculata), chosen for their different ranges of motion, as well as spine size and orientation. Using an integrated approach that combines behavioral studies with parametric modeling, additive manufacturing, micro-computed tomography, scanning electron microscopy, and finite element simulations, we present a high-throughput workflow that provides a fundamental understanding of 3D structure-kinematic relationships in brittle star skeletal systems.