Neurodynamic modeling of the fruit fly Drosophila melanogaster

Sensorimotor delays constrain robust locomotion in a 3D kinematic model of fly walking

Walking animals must maintain stability in the presence of external perturbations, despite significant temporal delays in neural signaling and muscle actuation. Here, we develop a 3D kinematic model with a layered control architecture to investigate how sensorimotor delays constrain robustness of walking behavior in the fruit fly, Drosophila. Motivated by the anatomical architecture of insect locomotor control circuits, our model consists of three component layers: a neural network that generates realistic 3D joint kinematics for each leg, an optimal controller that executes the joint kinematics while accounting for delays, and an inter-leg coordinator. The model generates realistic simulated walking that matches real fly walking kinematics and sustains walking even when subjected to unexpected perturbations, generalizing beyond its training data. However, we found that the model's robustness to perturbations deteriorates when sensorimotor delay parameters exceed the physiological range. These results suggest that fly sensorimotor control circuits operate close to the temporal limit at which they can detect and respond to external perturbations. More broadly, we show how a modular, layered model architecture can be used to investigate physiological constraints on animal behavior.

Mechanical modeling of mechanosensitive insect strain sensors as a tool to investigate exoskeletal interfaces

During walking, sensory information is measured and monitored by sensory organs that can be found on and within various limb segments. Strain can be monitored by insect load sensors, campaniform sensilla (CS), which have components embedded within the exoskeleton. CS vary in eccentricity, size, and orientation, which can affect their sensitivity to specific strains. Directly investigating the mechanical interfaces that these sensors utilize to encode changes in load bears various obstacles, such as modeling of viscoelastic properties. To circumvent the difficulties of modeling and performing biological experiments in small insects, we developed 3-dimensional printed resin models based on high-resolution imaging of CS. Through the utilization of strain gauges and a motorized tensile tester, physiologically plausible strain can be mimicked while investigating the compression and tension forces that CS experience; here, this was performed for a field of femoral CS inDrosophila melanogaster. Different loading scenarios differentially affected CS compression and the likely neuronal activity of these sensors and elucidate population coding of stresses acting on the cuticle.

Phase shift between joint rotation and actuation reflects dominant forces and predicts muscle activation patterns

During behavior, the work done by actuators on the body can be resisted by the body's inertia, elastic forces, gravity, or viscosity. The dominant forces that resist actuation have major consequences on the control of that behavior. In the literature, features and actuation of locomotion, for example, have been successfully predicted by nondimensional numbers (e.g. Froude number and Reynolds number) that generally express the ratio between two of these forces (gravitational, inertial, elastic, and viscous). However, animals of different sizes or motions at different speeds may not share the same dominant forces within a behavior, making ratios of just two of these forces less useful. Thus, for a broad comparison of behavior across many orders of magnitude of limb length and cycle period, a dimensionless number that includes gravitational, inertial, elastic, and viscous forces is needed. This study proposes a nondimensional number that relates these four forces: the phase shift (ϕ) between the displacement of the limb and the actuator force that moves it. Using allometric scaling laws, ϕ for terrestrial walking is expressed as a function of the limb length and the cycle period at which the limb steps. Scale-dependent values of ϕ are used to explain and predict the electromyographic (EMG) patterns employed by different animals as they walk.

SNS-Toolbox: An Open Source Tool for Designing Synthetic Nervous Systems and Interfacing Them with Cyber-Physical Systems

One developing approach for robotic control is the use of networks of dynamic neurons connected with conductance-based synapses, also known as Synthetic Nervous Systems (SNS). These networks are often developed using cyclic topologies and heterogeneous mixtures of spiking and non-spiking neurons, which is a difficult proposition for existing neural simulation software. Most solutions apply to either one of two extremes, the detailed multi-compartment neural models in small networks, and the large-scale networks of greatly simplified neural models. In this work, we present our open-source Python package SNS-Toolbox, which is capable of simulating hundreds to thousands of spiking and non-spiking neurons in real-time or faster on consumer-grade computer hardware. We describe the neural and synaptic models supported by SNS-Toolbox, and provide performance on multiple software and hardware backends, including GPUs and embedded computing platforms. We also showcase two examples using the software, one for controlling a simulated limb with muscles in the physics simulator Mujoco, and another for a mobile robot using ROS. We hope that the availability of this software will reduce the barrier to entry when designing SNS networks, and will increase the prevalence of SNS networks in the field of robotic control.

A virtuous cycle between invertebrate and robotics research: perspective on a decade of Living Machines research

Many invertebrates are ideal model systems on which to base robot design principles due to their success in solving seemingly complex tasks across domains while possessing smaller nervous systems than vertebrates. Three areas are particularly relevant for robot designers: Research on flying and crawling invertebrates has inspired new materials and geometries from which robot bodies (their morphologies) can be constructed, enabling a new generation of softer, smaller, and lighter robots. Research on walking insects has informed the design of new systems for controlling robot bodies (their motion control) and adapting their motion to their environment without costly computational methods. And research combining wet and computational neuroscience with robotic validation methods has revealed the structure and function of core circuits in the insect brain responsible for the navigation and swarming capabilities (their mental faculties) displayed by foraging insects. The last decade has seen significant progress in the application of principles extracted from invertebrates, as well as the application of biomimetic robots to model and better understand how animals function. This Perspectives paper on the past 10 years of the Living Machines conference outlines some of the most exciting recent advances in each of these fields before outlining lessons gleaned and the outlook for the next decade of invertebrate robotic research.

Adaptive load feedback robustly signals force dynamics in robotic model of Carausius morosus stepping

Animals utilize a number of neuronal systems to produce locomotion. One type of sensory organ that contributes in insects is the campaniform sensillum (CS) that measures the load on their legs. Groups of the receptors are found on high stress regions of the leg exoskeleton and they have significant effects in adapting walking behavior. Recording from these sensors in freely moving animals is limited by technical constraints. To better understand the load feedback signaled by CS to the nervous system, we have constructed a dynamically scaled robotic model of the Carausius morosus stick insect middle leg. The leg steps on a treadmill and supports weight during stance to simulate body weight. Strain gauges were mounted in the same positions and orientations as four key CS groups (Groups 3, 4, 6B, and 6A). Continuous data from the strain gauges were processed through a previously published dynamic computational model of CS discharge. Our experiments suggest that under different stepping conditions (e.g., changing "body" weight, phasic load stimuli, slipping foot), the CS sensory discharge robustly signals increases in force, such as at the beginning of stance, and decreases in force, such as at the end of stance or when the foot slips. Such signals would be crucial for an insect or robot to maintain intra- and inter-leg coordination while walking over extreme terrain.

Insect-Inspired Robots: Bridging Biological and Artificial Systems

This review article aims to address common research questions in hexapod robotics. How can we build intelligent autonomous hexapod robots that can exploit their biomechanics, morphology, and computational systems, to achieve autonomy, adaptability, and energy efficiency comparable to small living creatures, such as insects? Are insects good models for building such intelligent hexapod robots because they are the only animals with six legs? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying relevant and future directions in the field of hexapod robotics over the next decade. After an introduction in section (1), the sections will respectively cover the following three key areas: (2) biomechanics focused on the design of smart legs; (3) locomotion control; and (4) high-level cognition control. These interconnected and interdependent areas are all crucial to improving the level of performance of hexapod robotics in terms of energy efficiency, terrain adaptability, autonomy, and operational range. We will also discuss how the next generation of bioroboticists will be able to transfer knowledge from biology to robotics and vice versa.

Investigating the role of low level reinforcement reflex loops in insect locomotion

Insects are highly capable walkers, but many questions remain regarding how the insect nervous system controls locomotion. One particular question is how information is communicated between the 'lower level' ventral nerve cord (VNC) and the 'higher level' head ganglia to facilitate control. In this work, we seek to explore this question by investigating how systems traditionally described as 'positive feedback' may initiate and maintain stepping in the VNC with limited information exchanged between lower and higher level centers. We focus on the 'reflex reversal' of the stick insect femur-tibia joint between a resistance reflex (RR) and an active reaction in response to joint flexion, as well as the activation of populations of descending dorsal median unpaired (desDUM) neurons from limb strain as our primary reflex loops. We present the development of a neuromechanical model of the stick insect (Carausius morosus) femur-tibia (FTi) and coxa-trochanter joint control networks 'in-the-loop' with a physical robotic limb. The control network generates motor commands for the robotic limb, whose motion and forces generate sensory feedback for the network. We based our network architecture on the anatomy of the non-spiking interneuron joint control network that controls the FTi joint, extrapolated network connectivity based on known muscle responses, and previously developed mechanisms to produce 'sideways stepping'. Previous studies hypothesized that RR is enacted by selective inhibition of sensory afferents from the femoral chordotonal organ, but no study has tested this hypothesis with a model of an intact limb. We found that inhibiting the network's flexion position and velocity afferents generated a reflex reversal in the robot limb's FTi joint. We also explored the intact network's ability to sustain steady locomotion on our test limb. Our results suggested that the reflex reversal and limb strain reinforcement mechanisms are both necessary but individually insufficient to produce and maintain rhythmic stepping in the limb, which can be initiated or halted by brief, transient descending signals. Removing portions of this feedback loop or creating a large enough disruption can halt stepping independent of the higher-level centers. We conclude by discussing why the nervous system might control motor output in this manner, as well as how to apply these findings to generalized nervous system understanding and improved robotic control.

Anipose: A toolkit for robust markerless 3D pose estimation

Quantifying movement is critical for understanding animal behavior. Advances in computer vision now enable markerless tracking from 2D video, but most animals move in 3D. Here, we introduce Anipose, an open-source toolkit for robust markerless 3D pose estimation. Anipose is built on the 2D tracking method DeepLabCut, so users can expand their existing experimental setups to obtain accurate 3D tracking. It consists of four components: (1) a 3D calibration module, (2) filters to resolve 2D tracking errors, (3) a triangulation module that integrates temporal and spatial regularization, and (4) a pipeline to structure processing of large numbers of videos. We evaluate Anipose on a calibration board as well as mice, flies, and humans. By analyzing 3D leg kinematics tracked with Anipose, we identify a key role for joint rotation in motor control of fly walking. To help users get started with 3D tracking, we provide tutorials and documentation at http://anipose.org/.

A computational model of insect campaniform sensilla predicts encoding of forces during walking

Control of forces is essential in both animals and walking machines. Insects measure forces as strains in their exoskeletons via campaniform sensilla (CS). Deformations of cuticular caps embedded in the exoskeleton excite afferents that project to the central nervous system. CS afferent firing frequency (i.e. 'discharge') is highly dynamic, correlating with the rate of change of the force. Discharges adapt over time to tonic forces and exhibit hysteresis during cyclic loading.In this study we characterized a phenomenological model that predicts CS discharge, in which discharge is proportional to the instantaneous stimulus force relative to an adaptive variable. In contrast to previous studies of sensory adaptation, our model (1) is nonlinear and (2) reproduces the characteristic power-law adaptation with first order dynamics only (i.e. no 'fractional derivatives' are required to explain dynamics). We solve the response of the system analytically in multiple cases and use these solutions to derive the dynamics of the adaptive variable. We show that the model can reproduce responses of insect CS to many different force stimuli after being tuned to reproduce only one response, suggesting that the model captures the underlying dynamics of the system. We show that adaptation to tonic forces, rate-sensitivity, and hysteresis are different manifestations of the same underlying mechanism: the adaptive variable. We tune the model to replicate the dynamics of three different CS groups from two insects (cockroach and stick insect), demonstrating that it is generalizable. We also invert the model to estimate the stimulus force given the discharge recording from the animal. We discuss the adaptive neural and mechanical processes that the model may mimic and the model's use for understanding the role of load feedback in insect motor control. A preliminary model and results were previously published in the proceedings of the Conference on Biohybrid and Biomimetic Systems.

A leg to stand on: computational models of proprioception

Dexterous motor control requires feedback from proprioceptors, internal mechanosensory neurons that sense the body's position and movement. An outstanding question in neuroscience is how diverse proprioceptive feedback signals contribute to flexible motor control. Genetic tools now enable targeted recording and perturbation of proprioceptive neurons in behaving animals; however, these experiments can be challenging to interpret, due to the tight coupling of proprioception and motor control. Here, we argue that understanding the role of proprioceptive feedback in controlling behavior will be aided by the development of multiscale models of sensorimotor loops. We review current phenomenological and structural models for proprioceptor encoding and discuss how they may be integrated with existing models of posture, movement, and body state estimation.