Hybrid Hierarchical Reinforcement Learning for online guidance and navigation with partial observability

Minmax fuzzy deterministic policy gradient for zero-sum differential game: Take pursuit-evasion problem as example

A novel actor-critic algorithm is introduced and applied to zero-sum differential game. The proposed novel structure consists of two actors and a critic. Different actors represent the control policies of different players, and the critic is used to approximate the state-action utility function. Instead of neural network, the fuzzy inference system is applied as approximators for the actors and critic so that the specific practical meaning can be represented by the linguistic fuzzy rules. Since the goals of the players in the game are completely opposite, the actors for different players are simultaneously updated in opposite directions during the training. One actor is updated updated toward the direction that can minimize the Q value while the other updated toward the direction that can maximize the Q value. A pursuit-evasion problem with two pursuers and one evader is taken as an example to illustrate the validity of our method. In this problem, the two pursuers the same actor and the symmetry in the problem is used to improve the replay buffer. At the end of this paper, some confrontations between the policies with different training episodes are conducted.

Goal-directed autonomous navigation of mobile robot based on the principle of neuromodulation

Autonomous navigation in dynamic environment is aprerequisite of the mobile robot to perform tasks, and numerous approaches have been presented, including the supervised learning. Using supervised learning in robot navigation might meet problems, such as inconsistent and noisy data, and high error in training data. Inspired by the advantages of the reinforcement learning, such as no need for desired outputs, many researchers have applied reinforcement learning to robot navigation. This paper presents anovel method to address the robot navigation in different settings, through integrating supervised learning and analogical reinforcement learning into amotivated developmental network. We focus on the effect of the new learning rate on the robot navigation behavior. Experimentally, we show that the effect of internal neurons on the learning rate allows the agent to approach the target and avoid the obstacle as compounding effects of sequential states in static, dynamic, and complex environments. Further, we compare the performance between the emergent developmental network system and asymbolic system, as well as other four reinforcement learning algorithms. These experiments indicate that the reinforcement learning is beneficial for developing desirable behaviors in this set of robot navigation- staying statistically close to its target and away from obstacle.