Attention-Based Meta-Reinforcement Learning for Tracking Control of AUV With Time-Varying Dynamics

Target-Following Control of a Biomimetic Autonomous System Based on Predictive Reinforcement Learning

Biological fish often swim in a schooling manner, the mechanism of which comes from the fact that these schooling movements can improve the fishes' hydrodynamic efficiency. Inspired by this phenomenon, a target-following control framework for a biomimetic autonomous system is proposed in this paper. Firstly, a following motion model is established based on the mechanism of fish schooling swimming, in which the follower robotic fish keeps a certain distance and orientation from the leader robotic fish. Second, by incorporating a predictive concept into reinforcement learning, a predictive deep deterministic policy gradient-following controller is provided with the normalized state space, action space, reward, and prediction design. It can avoid overshoot to a certain extent. A nonlinear model predictive controller is designed and can be selected for the follower robotic fish, together with the predictive reinforcement learning. Finally, extensive simulations are conducted, including the fix point and dynamic target following for single robotic fish, as well as cooperative following with the leader robotic fish. The obtained results indicate the effectiveness of the proposed methods, providing a valuable sight for the cooperative control of underwater robots to explore the ocean.

MW-MADDPG: a meta-learning based decision-making method for collaborative UAV swarm

Unmanned Aerial Vehicles (UAVs) have gained popularity due to their low lifecycle cost and minimal human risk, resulting in their widespread use in recent years. In the UAV swarm cooperative decision domain, multi-agent deep reinforcement learning has significant potential. However, current approaches are challenged by the multivariate mission environment and mission time constraints. In light of this, the present study proposes a meta-learning based multi-agent deep reinforcement learning approach that provides a viable solution to this problem. This paper presents an improved MAML-based multi-agent deep deterministic policy gradient (MADDPG) algorithm that achieves an unbiased initialization network by automatically assigning weights to meta-learning trajectories. In addition, a Reward-TD prioritized experience replay technique is introduced, which takes into account immediate reward and TD-error to improve the resilience and sample utilization of the algorithm. Experiment results show that the proposed approach effectively accomplishes the task in the new scenario, with significantly improved task success rate, average reward, and robustness compared to existing methods.

Real-Time Ocean Current Compensation for AUV Trajectory Tracking Control Using a Meta-Learning and Self-Adaptation Hybrid Approach

Autonomous underwater vehicles (AUVs) may deviate from their predetermined trajectory in underwater currents due to the complex effects of hydrodynamics on their maneuverability. Model-based control methods are commonly employed to address this problem, but they suffer from issues related to the time-variability of parameters and the inaccuracy of mathematical models. To improve these, a meta-learning and self-adaptation hybrid approach is proposed in this paper to enable an underwater robot to adapt to ocean currents. Instead of using a traditional complex mathematical model, a deep neural network (DNN) serving as the basis function is trained to learn a high-order hydrodynamic model offline; then, a set of linear coefficients is adjusted dynamically by an adaptive law online. By conjoining these two strategies for real-time thrust compensation, the proposed method leverages the potent representational capacity of DNN along with the rapid response of adaptive control. This combination achieves a significant enhancement in tracking performance compared to alternative controllers, as observed in simulations. These findings substantiate that the AUV can adeptly adapt to new speeds of ocean currents.