Stigmergy: from mathematical modelling to control

Stigmergy, the indirect communication between agents of a swarm through dynamic environmental modifications, is a fundamental self-organization mechanism of animal swarms. Engineers have drawn inspiration from stigmergy to establish strategies for the coordination of swarms of robots and of mixed societies of robots and animals. Currently, all models of stigmergy are algorithmic, in the form of behavioural rules implemented at an individual level. A critical challenge for the understanding of stigmergic behaviour and translation of stigmergy to engineering is the lack of a holistic approach to determine which modifications of the environment are necessary to achieve desired behaviours for the swarm. Here, we propose a mathematical framework that rigorously describes the relationship between environmental modifications and swarm behaviour. Building on recent strides in continuification techniques, we model the swarm and environmental modifications as continua. This approach allows us to design the environmental modifications required for the swarm to behave as desired. Through analytical derivations and numerical simulations of one- and two-dimensional examples, we show that our framework yields the distribution of traces required to achieve a desired formation. Such an approach provides an adaptable framework for different implementation platforms, from robotic swarms to mixed societies of robots and animals.

Reinforcement Learning-Based Continuous Action Space Path Planning Method for Mobile Robots

A reinforcement learning-based continuous action space path planning method for mobile robots is proposed in this article. First, the kinematic model of the mobile robot is analyzed, and on this basis, the optimal state space is constructed according to the minimum depth of the field value in the depth image to characterize the distance between the robot and the obstacle. Then, by setting the reward function of the mobile robot based on the artificial potential field method, the information of the robot’s distance from obstacles is continuous, and a new reinforcement learning training process is proposed. Finally, by introducing a DDPG algorithm, the path planning of a mobile robot in an unknown environment is described as a Markov decision process, and the optimal planning of the mobile robot’s continuous action space path is realized with a high success rate. The results show that compared with other three comparison methods, the final success rates of the proposed method are the highest, which are 97.2%, 99.1%, 98.4%, and 98.6%, respectively.

Towards Fast Plume Source Estimation with a Mobile Robot

The estimation of the parameters of an odour source is of high relevance for multiple applications, but it can be a slow and error prone process. This work proposes a fast particle filter-based method for source term estimation with a mobile robot. Two strategies are implemented in order to reduce the computational cost of the filter and increase its accuracy: firstly, the sampling process is adapted by the mobile robot in order to optimise the quality of the data provided to the estimation process; secondly, the filter is initialised only after collecting preliminary data that allow limiting the solution space and use a shorter number of particles than it would be normally necessary. The method assumes a Gaussian plume model for odour dispersion. This models average odour concentrations, but the particle filter was proved adequate to fit instantaneous concentration measurements to that model, while the environment was being sampled. The method was validated in an obstacle free controlled wind tunnel and the validation results show its ability to quickly converge to accurate estimates of the plume’s parameters after a reduced number of plume crossings.

Decentralized Multi-agent information-theoretic control for target estimation and localization: finding gas leaks

This article presents a new decentralized multi-agent information-theoretic (DeMAIT) control algorithm for mobile sensors (agents). The algorithm leverages Bayesian estimation and information-theoretic motion planning for efficient and effective estimation and localization of a target, such as a chemical gas leak. The algorithm consists of: (1) a non-parametric Bayesian estimator, (2) an information-theoretic trajectory planner that generates “informative trajectories” for agents to follow, and (3) a controller and collision avoidance algorithm to ensure that each agent follows its trajectory as closely as possible in a safe manner. Advances include the use of a new information-gain metric and its analytical gradient, which do not depend on an infinite series like prior information metrics. Dynamic programming and multi-threading techniques are applied to efficiently compute the mutual information to minimize measurement uncertainty. The estimation and motion planning processes also take into account the dynamics of the sensors and agents. Extensive simulations are conducted to compare the performance between the DeMAIT algorithm to a traditional raster-scanning method and a clustering method with coordination. The main hypothesis that the DeMAIT algorithm outperforms the other two methods is validated, specifically where the average localization success rate for the DeMAIT algorithm is (a) higher and (b) more robust to changes in the source location, robot team size, and search area size than the raster-scanning and clustering methods. Finally, outdoor field experiments are conducted using a team of custom-built aerial robots equipped with gas concentration sensors to demonstrate efficacy of the DeMAIT algorithm to estimate and find the source of a propane gas leak.

A Study of Modified Infotaxis Algorithms in 2D and 3D Turbulent Environments

Emergency response to hazardous gases in the environment is an important research field in environmental monitoring. In recent years, with the rapid development of sensor technology and mobile device technology, more autonomous search algorithms for hazardous gas emission sources are proposed in uncertain environment, which can avoid emergency personnel from contacting hazardous gas in a short distance. Infotaxis is an autonomous search strategy without a concentration gradient, which uses scattered sensor data to track the location of the release source in turbulent environment. This paper optimizes the imbalance of exploitation and exploration in the reward function of Infotaxis algorithm and proposes a mobile strategy for the three-dimensional scene. In two-dimensional and three-dimensional scenes, the average steps of search tasks are used as the evaluation criteria to analyze the information trend algorithm combined with different reward functions and mobile strategies. The results show that the balance between the exploitation item and exploration item of the reward function proposed in this paper is better than that of the reward function in the Infotaxis algorithm, no matter in the two-dimensional scenes or in the three-dimensional scenes.