An Improved Approach towards Multi-Agent Pursuit-Evasion Game Decision-Making Using Deep Reinforcement Learning

An Ecological and Neural Argument for Developing Pursuit-Based Cognitive Enrichment for Sea Lions in Human Care

While general enrichment strategies for captive animals attempt to elicit variable and species-typical behaviors, approaches to cognitive enrichment have been disappointingly one-size-fits-all. In this commentary, we address the potential benefit of tailoring cognitive enrichment to the "cognitive niche" of the species, with a particular focus on a reasonably well-studied marine carnivore, the sea lion. Sea lions likely share some cognitive evolutionary pressures with primates, including complex social behavior. Their foraging ecology, however, like that of many terrestrial carnivores, is based on the rapid and behaviorally flexible pursuit of avoidant prey. Unlike terrestrial carnivores, sea lions carry out this pursuit in a truly fluid three-dimensional field, computing and executing sensorimotor transformations from any solid angle to any other. The cognitive demands of flexible prey pursuit are unlikely to be fully elicited by typical stationary puzzle box style foraging enrichment devices or screen-based interactive games. With this species, we recommend exploring more water-based movement activities generally, and complex pursuit challenges specifically.

Intelligent maneuver strategy for hypersonic vehicles in three-player pursuit-evasion games via deep reinforcement learning

Aiming at the rapid development of anti-hypersonic collaborative interception technology, this paper designs an intelligent maneuver strategy of hypersonic vehicles (HV) based on deep reinforcement learning (DRL) to evade the collaborative interception by two interceptors. Under the meticulously designed collaborative interception strategy, the uncertainty and difficulty of evasion are significantly increased and the opportunity for maneuvers is further compressed. This paper, accordingly, selects the twin delayed deep deterministic gradient (TD3) strategy acting on the continuous action space and makes targeted improvements combining deep neural networks to grasp the maneuver strategy and achieve successful evasion. Focusing on the time-coordinated interception strategy of two interceptors, the three-player pursuit and evasion (PE) problem is modeled as the Markov decision process, and the double training strategy is proposed to juggle both interceptors. In reward functions of the training process, the energy saving factor is set to achieve the trade-off between miss distance and energy consumption. In addition, the regression neural network is introduced into the deep neural network of TD3 to enhance intelligent maneuver strategies' generalization. Finally, numerical simulations are conducted to verify that the improved TD3 algorithm can effectively evade the collaborative interception of two interceptors under tough situations, and the improvements of the algorithm in terms of convergence speed, generalization, and energy-saving effect are verified.

A discrete particle swarm optimization coalition formation algorithm for multi-pursuer multi-evader game

Multi-Pursuers Multi-Evader Game (MPMEG) is considered as a multi-agent complex problem in which the pursuers must perform the capture of the detected evaders according to the temporal constraints. In this paper, we propose a metaheuristic approach based on a Discrete Particle Swarm Optimization in order to allow a dynamic coalition formation of the pursuers during the pursuit game. A pursuit coalition can be considered as the role definition of each pursuer during the game. In this work, each possible coalition is represented by a feasible particle’s position, which changes the concerned coalition according to its velocity during the pursuit game. With the aim of showcasing the performance of the new approach, we propose a comparison study in relation to recent approaches processing the MPMEG in term of capturing time and payoff acquisition. Moreover, we have studied the pursuit capturing time according to the number of used particles as well as the dynamism of the pursuit coalitions formed during the game. The obtained results note that the proposed approach outperforms the compared approaches in relation to the capturing time by only using eight particles. Moreover, this approach improves the pursuers’ payoff acquisition, which represents the pursuers’ learning rate during the task execution.

Pursuit and Evasion Strategy of a Differential Game Based on Deep Reinforcement Learning

Since the emergence of deep neural network (DNN), it has achieved excellent performance in various research areas. As the combination of DNN and reinforcement learning, deep reinforcement learning (DRL) becomes a new paradigm for solving differential game problems. In this study, we build up a reinforcement learning environment and apply relevant DRL methods to a specific bio-inspired differential game problem: the dog sheep game. The dog sheep game environment is set on a circle where the dog chases down the sheep attempting to escape. According to some presuppositions, we are able to acquire the kinematic pursuit and evasion strategy. Next, this study implements the value-based deep Q network (DQN) model and the deep deterministic policy gradient (DDPG) model to the dog sheep game, attempting to endow the sheep the ability to escape successfully. To enhance the performance of the DQN model, this study brought up the reward mechanism with a time-out strategy and the game environment with an attenuation mechanism of the steering angle of sheep. These modifications effectively increase the probability of escape for the sheep. Furthermore, the DDPG model is adopted due to its continuous action space. Results show the modifications of the DQN model effectively increase the escape probabilities to the same level as the DDPG model. When it comes to the learning ability under various environment difficulties, the refined DQN and the DDPG models have bigger performance enhancement over the naive evasion model in harsh environments than in loose environments.

A Word-Granular Adversarial Attacks Framework for Causal Event Extraction

As a data augmentation method, masking word is commonly used in many natural language processing tasks. However, most mask methods are based on rules and are not related to downstream tasks. In this paper, we propose a novel masking word generator, named Actor-Critic Mask Model (ACMM), which can adaptively adjust the mask strategy according to the performance of downstream tasks. In order to demonstrate the effectiveness of the method, we conducted experiments on two causal event extraction datasets. Experiment results show that, compared with various rule-based masking methods, the masked sentences generated by our proposed method can significantly enhance the generalization of the model and improve the model performance.