UAVs Maneuver Decision-Making Method Based on Transfer Reinforcement Learning

Aiming at the 1vs1 confrontation problem in a complex environment where obstacles are randomly distributed, the DDPG (deep deterministic policy gradient) algorithm is used to design the maneuver decision-making method of UAVs. Traditional methods generally assume that all obstacles are known globally. In this paper, a UAV airborne lidar detection model is designed, which can effectively solve the problem of obstacle avoidance when facing a large number of unknown obstacles. On the basis of the designed model, the idea of transfer learning is used to transfer the strategy trained by one UAV in a simple task to a new similar task, and the strategy will be used to train the strategy of the other UAV. This method can improve the intelligence of the UAVs in both sides alternately and progressively. The simulation results show that the transfer learning method can speed up the training process and improve the training effect.

Two-agent cooperative search model with Petri nets

Purpose

The purpose of this paper is to propose a model for a two-agent multi-target-searching scenario in a two-dimensional region, where some places of the region have limited resource capacity in terms of the number of agents that can simultaneously pass through those places and few places of the region are unreachable that expand with time. The proposed cooperative search model and Petri net model facilitate the search operation considering the constraints mentioned in the paper. The Petri net model graphically illustrates different scenarios and helps the agents to validate the strategies.

Design/methodology/approach

In this paper, the authors have applied an optimization approach to determine the optimal locations of base stations, a cooperative search model, inclusion–exclusion principle, Cartesian product to optimize the search operation and a Petri net model to validate the search technique.

Findings

The proposed approach finds the optimal locations of the base stations in the region. The proposed cooperative search model allows various constraints such as resource capacity, time-dependent unreachable places/obstacles, fuel capacities of the agents, two types of targets assigned to two agents and limited sortie lengths. On the other hand, a Petri net model graphically represents whether collisions/deadlocks between the two agents are possible or not for a particular combination of paths as well as effect of time-dependent unreachable places for different combination of paths are also illustrated.

Practical implications

The problem addressed in this paper is similar to various real-time problems such as rescue operations during/after flood, landslide, earthquake, accident, patrolling in urban areas, international borders, forests, etc. Thus, the proposed model can benefit various organizations and departments such as rescue operation authorities, defense organizations, police departments, etc.

Originality/value

To the best of the authors’ knowledge, the problem addressed in this paper has not been completely explored, and the proposed cooperative search model to conduct the search operation considering the above-mentioned constraints is new. To the best of the authors’ knowledge, no paper has modeled time-dependent unreachable places with the help of Petri net.

Delay-Dependent Algebraic Riccati Equation to Stabilization of Networked Control Systems: Continuous-Time Case

In this paper, a delay-dependent algebraic Riccati equation (DARE) approach is developed to study the meansquare stabilization problem for continuous-time networked control systems. Different from most previous studies that information transmission can be performed with zero delay and infinite precision, this paper presents a basic constraint that the designed control signal is transmitted over a delayed communication channel, where signal attenuation and transmission delay occur simultaneously. The innovative contributions of this paper are threefold. First, we propose a necessary and sufficient stabilizing condition in terms of a unique positive definite solution to a DARE with Q > 0 and R > 0. In accordance with this result, we derive the Lyapunov/spectrum stabilizing criterion. Second, we apply the operator spectrum theory to study the stabilizing solution to a more general DARE with Q ≥ 0 and R > 0. By defining a delay-dependent Lyapunov operator, we propose the existence theorem of the unique stabilizing solution. It is shown that the stabilizing solution, if it exists, is unique and coincides with a maximal solution. Third, as an application, we derive the explicit maximal allowable delay bound for a scalar system. To confirm the validity of our theoretic results, two illustrative examples are included in this paper.

Reliable confirmation of an object identity by a mobile robot: A mixed appearance/localization-driven motion approach

This paper investigates the problem of confirming the identity of a candidate object (expected to be a target based on some crude visual clues) with a mobile robot equipped with visual sensing capabilities. We present a method whose main novelty is to mix localization of the robot relative to the candidate object and to confirm that it is the sought target. This twofold approach drastically reduces false positives. Identity confirmation with this twofold goal is modeled as a Partially-Observable Markov Decision Process, where the states are the cells of the space decomposition. It is solved using Stochastic Dynamic Programming with imperfect state information. A robotic system using this method has been implemented and tests have been carried out both in simulation and with a real robot. The experiments empirically validate the use of various metrics, and demonstrate their ability to perform well in different settings.

Surveillance Strategies for a Pursuer with Finite Sensor Range

This paper addresses the pursuit-evasion problem of maintaining surveillance by a pursuer of an evader in a world populated by polygonal obstacles. This requires the pursuer to plan colision-free motions that honor distance constraints imposed by sensor capabilities, while avoiding occlusion of the evader by any obstacle. The paper extends the three-dimensional cellular decomposition of Schwartz and Sharir to represent the four-dimensional configuration space of the pursuer-evader system, and derive necessary conditions for surveillance (equivalently, sufficient conditions for escape) in terms of this new representation A game theoretic formulation of the problem is then given, and this formulation is used to characterize optimal escape trajectories for the evader. A shooting algorithm is proposed that finds these trajectories using the minimun principle. Finally, noting the similarities between this surveillance problem and the problem of cooperative manipulation by two robots, several cooperation strategies are presented that maximize system performance for cooperative motions.

Visibility-based Pursuit-evasion with Limited Field of View

We study the visibility-based pursuit-evasion problem, in which one or more searchers must move through a given environment so as to guarantee detection of any and all evaders, which can move arbitrarily fast. Our goal is to develop techniques for coordinating teams of robots to execute this task in application domains such as clearing a building, for reasons of security or safety. To this end, we introduce a new class of searcher, the φ-searcher, which can be readily instantiated as a physical mobile robot. We present a detailed analysis of the pursuit-evasion problem using φ-searchers. We present the first complete search algorithm for a single φ-searcher, show how this algorithm can be extended to handle multiple searchers, and give examples of computed trajectories.

Lion and man with visibility in monotone polygons

In the original version of the lion and man game, a lion tries to capture a man who is trying to escape in a circular arena. The players have equal speeds. They can observe each other at all times. We study a new variant of the game in which the lion has only line-of-sight visibility. That is it can observe the man’s position only if the line segment connecting them does not intersect the boundary. We show that despite this limitation, the lion can capture the man in any monotone polygon in finite time.

Capturing an evader in polygonal environments with obstacles: The full visibility case

Suppose an unpredictable evader is free to move around in a polygonal environment of arbitrary complexity that is under full camera surveillance. How many pursuers, each with the same maximum speed as the evader, are necessary and sufficient to guarantee a successful capture of the evader? The pursuers always know the evader’s current position through a camera network, but need to physically reach the evader to capture it. We allow the evader knowledge of the current positions of all the pursuers as well—this accords with the standard worst-case analysis model, but also models a practical situation where the evader has ‘hacked’ into the surveillance system. Our main result is to prove that three pursuers are always sufficient and sometimes necessary to capture the evader. The bound is independent of the number of vertices or holes in the polygonal environment.

Bioinspired neural network for real-time cooperative hunting by multirobots in unknown environments

Multiple robot cooperation is a challenging and critical issue in robotics. To conduct the cooperative hunting by multirobots in unknown and dynamic environments, the robots not only need to take into account basic problems (such as searching, path planning, and collision avoidance), but also need to cooperate in order to pursue and catch the evaders efficiently. In this paper, a novel approach based on a bioinspired neural network is proposed for the real-time cooperative hunting by multirobots, where the locations of evaders and the environment are unknown and changing. The bioinspired neural network is used for cooperative pursuing by the multirobot team. Some other algorithms are used to enable the robots to catch the evaders efficiently, such as the dynamic alliance and formation construction algorithm. In the proposed approach, the pursuing alliances can dynamically change and the robot motion can be adjusted in real-time to pursue the evader cooperatively, to guarantee that all the evaders can be caught efficiently. The proposed approach can deal with various situations such as when some robots break down, the environment has different boundary shapes, or the obstacles are linked with different shapes. The simulation results show that the proposed approach is capable of guiding the robots to achieve the hunting of multiple evaders in real-time efficiently.

Markovian search games in heterogeneous spaces

In this paper, we consider how to search for a mobile evader in a large heterogeneous region when sensors are used for detection. Sensors are modeled using probability of detection. Due to environmental effects, this probability will not be constant over the entire region. We map this problem to a graph-search problem, and even though deterministic graph search is NP-complete, we derive a tractable optimal probabilistic search strategy. We do this by defining the problem as a dynamic game played on a Markov chain. We prove that this strategy is optimal in the sense of Nash. Simulations of an example problem illustrate our approach and verify our claims.