A Neurocomputational Model of Goal-Directed Navigation in Insect-Inspired Artificial Agents

A model-free method to learn multiple skills in parallel on modular robots

Legged robots are well-suited for deployment in unstructured environments but require a unique control scheme specific for their design. As controllers optimised in simulation do not transfer well to the real world (the infamous sim-to-real gap), methods enabling quick learning in the real world, without any assumptions on the specific robot model and its dynamics, are necessary. In this paper, we present a generic method based on Central Pattern Generators, that enables the acquisition of basic locomotion skills in parallel, through very few trials. The novelty of our approach, underpinned by a mathematical analysis of the controller model, is to search for good initial states, instead of optimising connection weights. Empirical validation in six different robot morphologies demonstrates that our method enables robots to learn primary locomotion skills in less than 15 minutes in the real world. In the end, we showcase our skills in a targeted locomotion experiment.

An insect-inspired model facilitating autonomous navigation by incorporating goal approaching and collision avoidance

Being one of the most fundamental and crucial capacity of robots and animals, autonomous navigation that consists of goal approaching and collision avoidance enables completion of various tasks while traversing different environments. In light of the impressive navigational abilities of insects despite their tiny brains compared to mammals, the idea of seeking solutions from insects for the two key problems of navigation, i.e., goal approaching and collision avoidance, has fascinated researchers and engineers for many years. However, previous bio-inspired studies have focused on merely one of these two problems at one time. Insect-inspired navigation algorithms that synthetically incorporate both goal approaching and collision avoidance, and studies that investigate the interactions of these two mechanisms in the context of sensory-motor closed-loop autonomous navigation are lacking. To fill this gap, we propose an insect-inspired autonomous navigation algorithm to integrate the goal approaching mechanism as the global working memory inspired by the sweat bee's path integration (PI) mechanism, and the collision avoidance model as the local immediate cue built upon the locust's lobula giant movement detector (LGMD) model. The presented algorithm is utilized to drive agents to complete navigation task in a sensory-motor closed-loop manner within a bounded static or dynamic environment. Simulation results demonstrate that the synthetic algorithm is capable of guiding the agent to complete challenging navigation tasks in a robust and efficient way. This study takes the first tentative step to integrate the insect-like navigation mechanisms with different functionalities (i.e., global goal and local interrupt) into a coordinated control system that future research avenues could build upon.

No Need for Landmarks: An Embodied Neural Controller for Robust Insect-Like Navigation Behaviors

Bayesian filters have been considered to help refine and develop theoretical views on spatial cell functions for self-localization. However, extending a Bayesian filter to reproduce insect-like navigation behaviors (e.g., home searching) remains an open and challenging problem. To address this problem, we propose an embodied neural controller for self-localization, foraging, backward homing (BH), and home searching of an advanced mobility sensor (AMOS)-driven insect-like robot. The controller, comprising a navigation module for the Bayesian self-localization and goal-directed control of AMOS and a locomotion module for coordinating the 18 joints of AMOS, leads to its robust insect-like navigation behaviors. As a result, the proposed controller enables AMOS to perform robust foraging, BH, and home searching against various levels of sensory noise, compared to conventional controllers. Its implementation relies only on self-localization and heading perception, rather than global positioning and landmark guidance. Interestingly, the proposed controller makes AMOS achieve spiral searching patterns comparable to those performed by real insects. We also demonstrated the performance of the controller for real-time indoor and outdoor navigation in a real insect-like robot without any landmark and cognitive map.

A gecko-inspired robot with CPG-based neural control for locomotion and body height adaptation

Today's gecko-inspired robots have shown the ability of omnidirectional climbing on slopes with a low centre of mass. However, such an ability cannot efficiently cope with bumpy terrains or terrains with obstacles. In this study, we developed a gecko-inspired robot (Nyxbot) with an adaptable body height to overcome this limitation. Based on an analysis of the skeletal system and kinematics of real geckos, the adhesive mechanism and leg structure design of the robot were designed to endow it with adhesion and adjustable body height capabilities. Neural control with exteroceptive sensory feedback is utilised to realise body height adaptability while climbing on a slope. The locomotion performance and body adaptability of the robot were tested by conducting slope climbing and obstacle crossing experiments. The gecko robot can climb a 30° slope with spontaneous obstacle crossing (maximum obstacle height of 38% of the body height) and can climb even steeper slopes (up to 60°) without an obstacle or bump. Using 3D force measuring platforms for ground reaction force analysis of geckos and the robot, we show that the motions of the developed robot driven by neural control and the motions of geckos are dynamically comparable. To this end, this study provides a basis for developing climbing robots with adaptive bump/obstacle crossing on slopes towards more agile and versatile gecko-like locomotion.

Locomotion Control With Frequency and Motor Pattern Adaptations

Existing adaptive locomotion control mechanisms for legged robots are usually aimed at one specific type of adaptation and rarely combined with others. Adaptive mechanisms thus stay at a conceptual level without their coupling effect with other mechanisms being investigated. However, we hypothesize that the combination of adaptation mechanisms can be exploited for enhanced and more efficient locomotion control as in biological systems. Therefore, in this work, we present a central pattern generator (CPG) based locomotion controller integrating both a frequency and motor pattern adaptation mechanisms. We use the state-of-the-art Dual Integral Learner for frequency adaptation, which can automatically and quickly adapt the CPG frequency, enabling the entire motor pattern or output signal of the CPG to be followed at a proper high frequency with low tracking error. Consequently, the legged robot can move with high energy efficiency and perform the generated locomotion with high precision. The versatile state-of-the-art CPG-RBF network is used as a motor pattern adaptation mechanism. Using this network, the motor patterns or joint trajectories can be adapted to fit the robot's morphology and perform sensorimotor integration enabling online motor pattern adaptation based on sensory feedback. The results show that the two adaptation mechanisms can be combined for adaptive locomotion control of a hexapod robot in a complex environment. Using the CPG-RBF network for motor pattern adaptation, the hexapod learned basic straight forward walking, steering, and step climbing. In general, the frequency and motor pattern mechanisms complement each other well and their combination can be seen as an essential step toward further studies on adaptive locomotion control.

Anesthesia disrupts distance, but not direction, of path integration memory

Solitary foraging insects, such as ants, maintain an estimate of the direction and distance to their starting location as they move away from it, in a process known as path integration. This estimate, commonly known as the "home vector," is updated continuously as the ant moves^1-4 and is reset as soon as it enters its nest,⁵ yet ants prevented from returning to their nest can still use their home vector when released several hours later.^6,7 This conjunction of fast update and long persistence of the home vector memory does not directly map to existing accounts of short-, mid-, and long-term memory;^2,8-12 hence, the substrate of this memory remains unknown. Chill-coma anesthesia^13-15 has previously been shown to affect associative memory retention in fruit flies^14,16 and honeybees.^9,17,18 We investigate the nature of path integration memory by anesthetizing ants after they have accumulated home vector information and testing if the memory persists on recovery. We show that after anesthesia the memory of the distance ants have traveled is degraded, but the memory of the direction is retained. We also show that this is consistent with models of path integration that maintain the memory in a redundant Cartesian coordinate system and with the hypothesis that chill-coma produces a proportional reduction of the memory, rather than a subtractive reduction or increase of noise. The observed effect is not compatible with a memory based on recurrent circuit activity and points toward an activity-dependent molecular process as the basis of path integration memory.

A connectome of the Drosophila central complex reveals network motifs suitable for flexible navigation and context-dependent action selection

Flexible behaviors over long timescales are thought to engage recurrent neural networks in deep brain regions, which are experimentally challenging to study. In insects, recurrent circuit dynamics in a brain region called the central complex (CX) enable directed locomotion, sleep, and context- and experience-dependent spatial navigation. We describe the first complete electron microscopy-based connectome of the Drosophila CX, including all its neurons and circuits at synaptic resolution. We identified new CX neuron types, novel sensory and motor pathways, and network motifs that likely enable the CX to extract the fly's head direction, maintain it with attractor dynamics, and combine it with other sensorimotor information to perform vector-based navigational computations. We also identified numerous pathways that may facilitate the selection of CX-driven behavioral patterns by context and internal state. The CX connectome provides a comprehensive blueprint necessary for a detailed understanding of network dynamics underlying sleep, flexible navigation, and state-dependent action selection.

Active Inferants: An Active Inference Framework for Ant Colony Behavior

In this paper, we introduce an active inference model of ant colony foraging behavior, and implement the model in a series of in silico experiments. Active inference is a multiscale approach to behavioral modeling that is being applied across settings in theoretical biology and ethology. The ant colony is a classic case system in the function of distributed systems in terms of stigmergic decision-making and information sharing. Here we specify and simulate a Markov decision process (MDP) model for ant colony foraging. We investigate a well-known paradigm from laboratory ant colony behavioral experiments, the alternating T-maze paradigm, to illustrate the ability of the model to recover basic colony phenomena such as trail formation after food location discovery. We conclude by outlining how the active inference ant colony foraging behavioral model can be extended and situated within a nested multiscale framework and systems approaches to biology more generally.

LAMB2 novel variant c.2885-9 C>A affects RNA splicing in a minigene assay

Background

Both Pierson syndrome (PS) and isolated nephrotic syndrome can be caused by LAMB2 biallelic pathogenic variants. Only 15 causative splicing variants in the LAMB2 gene have been reported. However, the pathogenicity of most of these variants has not been verified, which may lead to incorrect interpretation of the functional consequence of these variants.

Methods

Using high‐throughput DNA sequencing and Sanger sequencing, we detected variants in a female with clinically suspected PS. A minigene splicing assay was performed to assess the effect of LAMB2 intron 20 c.2885‐9C>A on RNA splicing. We also performed the immunohistochemical analysis of laminin beta‐2 in kidney tissues.

Results

Two novel LAMB2 heteroallelic variants were found: a paternally inherited variant c.2885‐9C>A in intron 20 and a maternally inherited variant c. 3658C>T (p. (Gln1220Ter)). In vitro minigene assay showed that the variant c.2885‐9C>A caused erroneous integration of a 7 bp sequence into intron 20. Immunohistochemical analysis revealed the absence of glomerular expression of laminin beta‐2, the protein encoded by LAMB2.

Conclusion

We demonstrated the impact of a novel LAMB2 intronic variant on RNA splicing using the minigene assay firstly. Our results extend the mutational spectrum of LAMB2.

A dynamical model exploring sensory integration in the insect central complex substructures

It is imperative that an animal has the ability to contextually integrate received sensory information to formulate appropriate behavioral responses. Determining a body heading based on a multitude of ego-motion cues and visual landmarks is an example of such a task that requires this context dependent integration. The work presented here simulates a sensory integrator in the insect brain called the central complex (CX). Based on the architecture of the CX, we assembled a dynamical neural simulation of two structures called the protocerebral bridge (PB) and the ellipsoid body (EB). Using non-spiking neuronal dynamics, our simulation was able to recreate in vivo neuronal behavior such as correlating body rotation direction and speed to activity bumps within the EB as well as updating the believed heading with quick secondary system updates. With this model, we performed sensitivity analysis of certain neuronal parameters as a possible means to control multi-system gains during sensory integration. We found that modulation of synapses in the memory network and EB inhibition are two possible mechanisms in which a sensory system could affect the memory stability and gain of another input, respectively. This model serves as an exploration in network design for integrating simultaneous idiothetic and allothetic cues in the task of body tracking and determining contextually dependent behavioral outputs.

Principles of Insect Path Integration

Continuously monitoring its position in space relative to a goal is one of the most essential tasks for an animal that moves through its environment. Species as diverse as rats, bees, and crabs achieve this by integrating all changes of direction with the distance covered during their foraging trips, a process called path integration. They generate an estimate of their current position relative to a starting point, enabling a straight-line return, following what is known as a home vector. While in theory path integration always leads the animal precisely back home, in the real world noise limits the usefulness of this strategy when operating in isolation. Noise results from stochastic processes in the nervous system and from unreliable sensory information, particularly when obtaining heading estimates. Path integration, during which angular self-motion provides the sole input for encoding heading (idiothetic path integration), results in accumulating errors that render this strategy useless over long distances. In contrast, when using an external compass this limitation is avoided (allothetic path integration). Many navigating insects indeed rely on external compass cues for estimating body orientation, whereas they obtain distance information by integration of steps or optic-flow-based speed signals. In the insect brain, a region called the central complex plays a key role for path integration. Not only does the central complex house a ring-attractor network that encodes head directions, neurons responding to optic flow also converge with this circuit. A neural substrate for integrating direction and distance into a memorized home vector has therefore been proposed in the central complex. We discuss how behavioral data and the theoretical framework of path integration can be aligned with these neural data.

Merging information in the entorhinal cortex: what can we learn from robotics experiments and modeling?

Place recognition is a complex process involving idiothetic and allothetic information. In mammals, evidence suggests that visual information stemming from the temporal and parietal cortical areas (‘what’ and ‘where’ information) is merged at the level of the entorhinal cortex (EC) to build a compact code of a place. Local views extracted from specific feature points can provide information important for view cells (in primates) and place cells (in rodents) even when the environment changes dramatically. Robotics experiments using conjunctive cells merging ‘what’ and ‘where’ information related to different local views show their important role for obtaining place cells with strong generalization capabilities. This convergence of information may also explain the formation of grid cells in the medial EC if we suppose that: (1) path integration information is computed outside the EC, (2) this information is compressed at the level of the EC owing to projection (which follows a modulo principle) of cortical activities associated with discretized vector fields representing angles and/or path integration, and (3) conjunctive cells merge the projections of different modalities to build grid cell activities. Applying modulo projection to visual information allows an interesting compression of information and could explain more recent results on grid cells related to visual exploration. In conclusion, the EC could be dedicated to the build-up of a robust yet compact code of cortical activity whereas the hippocampus proper recognizes these complex codes and learns to predict the transition from one state to another.

The internal maps of insects

Insect navigation is strikingly geometric. Many species use path integration to maintain an accurate estimate of their distance and direction (a vector) to their nest and can store the vector information for multiple salient locations in the world, such as food sources, in a common coordinate system. Insects can also use remembered views of the terrain around salient locations or along travelled routes to guide return, which is a fundamentally geometric process. Recent modelling of these abilities shows convergence on a small set of algorithms and assumptions that appear sufficient to account for a wide range of behavioural data. Notably, this 'base model' does not include any significant topological knowledge: the insect does not need to recover the information (implicit in their vector memory) about the relationships between salient places; nor to maintain any connectedness or ordering information between view memories; nor to form any associations between views and vectors. However, there remains some experimental evidence not fully explained by this base model that may point towards the existence of a more complex or integrated mental map in insects.

Designing Brains for Pain: Human to Mollusc

There is compelling evidence that the "what it feels like" subjective experience of sensory stimuli arises in the cerebral cortex in both humans as well as mammalian experimental animal models. Humans are alone in their ability to verbally communicate their experience of the external environment. In other species, sensory awareness is extrapolated on the basis of behavioral indicators. For instance, cephalopods have been claimed to be sentient on the basis of their complex behavior and anecdotal reports of human-like intelligence. We have interrogated the findings of avoidance learning behavioral paradigms and classical brain lesion studies and conclude that there is no evidence for cephalopods feeling pain. This analysis highlighted the questionable nature of anthropometric assumptions about sensory experience with increased phylogenetic distance from humans. We contend that understanding whether invertebrates such as molluscs are sentient should first begin with defining the computational processes and neural circuitries underpinning subjective awareness. Using fundamental design principles, we advance the notion that subjective awareness is dependent on observer neural networks (networks that in some sense introspect the neural processing generating neural representations of sensory stimuli). This introspective process allows the observer network to create an internal model that predicts the neural processing taking place in the network being surveyed. Predictions arising from the internal model form the basis of a rudimentary form of awareness. We develop an algorithm built on parallel observer networks that generates multiple levels of sensory awareness. A network of cortical regions in the human brain has the appropriate functional properties and neural interconnectivity that is consistent with the predicted circuitry of the algorithm generating pain awareness. By contrast, the cephalopod brain lacks the necessary neural circuitry to implement such an algorithm. In conclusion, we find no compelling behavioral, functional, or neuroanatomical evidence to indicate that cephalopods feel pain.

How to Navigate in Different Environments and Situations: Lessons From Ants

Ants are a globally distributed insect family whose members have adapted to live in a wide range of different environments and ecological niches. Foraging ants everywhere face the recurring challenge of navigating to find food and to bring it back to the nest. More than a century of research has led to the identification of some key navigational strategies, such as compass navigation, path integration, and route following. Ants have been shown to rely on visual, olfactory, and idiothetic cues for navigational guidance. Here, we summarize recent behavioral work, focusing on how these cues are learned and stored as well as how different navigational cues are integrated, often between strategies and even across sensory modalities. Information can also be communicated between different navigational routines. In this way, a shared toolkit of fundamental navigational strategies can lead to substantial flexibility in behavioral outcomes. This allows individual ants to tune their behavioral repertoire to different tasks (e.g., foraging and homing), lifestyles (e.g., diurnal and nocturnal), or environments, depending on the availability and reliability of different guidance cues. We also review recent anatomical and physiological studies in ants and other insects that have started to reveal neural correlates for specific navigational strategies, and which may provide the beginnings of a truly mechanistic understanding of navigation behavior.

Unraveling the neural basis of insect navigation

One of the defining features of animals is their ability to navigate their environment. Using behavioral experiments this topic has been under intense investigation for nearly a century. In insects, this work has largely focused on the remarkable homing abilities of ants and bees. More recently, the neural basis of navigation shifted into the focus of attention. Starting with revealing the neurons that process the sensory signals used for navigation, in particular polarized skylight, migratory locusts became the key species for delineating navigation-relevant regions of the insect brain. Over the last years, this work was used as a basis for research in the fruit fly Drosophila and extraordinary progress has been made in illuminating the neural underpinnings of navigational processes. With increasingly detailed understanding of navigation circuits, we can begin to ask whether there is a fundamentally shared concept underlying all navigation behavior across insects. This review highlights recent advances and puts them into the context of the behavioral work on ants and bees, as well as the circuits involved in polarized-light processing. A region of the insect brain called the central complex emerges as the common substrate for guiding navigation and its highly organized neuroarchitecture provides a framework for future investigations potentially suited to explain all insect navigation behavior at the level of identified neurons.

Ants' navigation in an unfamiliar environment is influenced by their experience of a familiar route

When displaced experimentally from a food source (feeder) to unfamiliar terrain, ants run off a portion of the homeward vector or its entirety, depending on species and conditions, and then search systematically, turning in loops of ever increasing size. The Australian desert ant Melophorus bagoti runs off a smaller portion of its vector if the test site is more dissimilar to its nest area. Here we manipulated familiarity with the training route between a feeder and the ants' nest to examine its effects when the ants were displaced to a distant site from the feeder. Naïve ants that arrived at an experimentally provided feeder for the first time were compared with experienced ants that had travelled the route for two days. At the unfamiliar test site, naïve ants ran off a longer portion of their vector from path integration than did experienced ants. Naïve ants also spread out in their systematic search slower than did experienced ants. We conclude that as ants learn the views encountered on their familiar route better, they identify more readily unfamiliar views. A scene distant from their nest area may not look as unfamiliar to a naïve ant as it does to an experienced ant.

An Anatomically Constrained Model for Path Integration in the Bee Brain

Path integration is a widespread navigational strategy in which directional changes and distance covered are continuously integrated on an outward journey, enabling a straight-line return to home. Bees use vision for this task-a celestial-cue-based visual compass and an optic-flow-based visual odometer-but the underlying neural integration mechanisms are unknown. Using intracellular electrophysiology, we show that polarized-light-based compass neurons and optic-flow-based speed-encoding neurons converge in the central complex of the bee brain, and through block-face electron microscopy, we identify potential integrator cells. Based on plausible output targets for these cells, we propose a complete circuit for path integration and steering in the central complex, with anatomically identified neurons suggested for each processing step. The resulting model circuit is thus fully constrained biologically and provides a functional interpretation for many previously unexplained architectural features of the central complex. Moreover, we show that the receptive fields of the newly discovered speed neurons can support path integration for the holonomic motion (i.e., a ground velocity that is not precisely aligned with body orientation) typical of bee flight, a feature not captured in any previously proposed model of path integration. In a broader context, the model circuit presented provides a general mechanism for producing steering signals by comparing current and desired headings-suggesting a more basic function for central complex connectivity, from which path integration may have evolved.