Distributed zero-sum differential game for multi-agent systems in strict-feedback form with input saturation and output constraint

Dynamic learning from adaptive neural control for full-state constrained strict-feedback nonlinear systems

This study focuses on the learning and control issues of strict-feedback systems with full-state constraints. To achieve learning capability under constraints, transformation mapping is utilized to convert the original system with full-state constraints into a quasi-pure-feedback unconstrained system. Utilizing the system transformation technique, only a single neural network (NN) is required to identify the unknown dynamics within the transformed system. Combining the dynamic surface control design, a novel adaptive neural control scheme is developed to ensure that all closed-loop signals are uniformly bounded, and every system state remains within the predefined constraint range. In addition, the precise convergence of NN weights is further transformed into an exponential stability problem for a category of linear time-varying systems under persistent excitation conditions. Subsequently, the converged NN weights are efficiently stored and utilized to create a learning controller to achieve better control performance while abiding by the full-state constraints. The viability of this control strategy is demonstrated via simulations.

An adaptive neurodynamic approach for solving nonsmooth N-cluster games

In this paper, N-cluster games with coupling and private constraints are studied, where each player's cost function is nonsmooth and depends on the actions of all players. In order to seek the generalized Nash equilibrium (GNE) of the nonsmooth N-cluster games, a distributed seeking neurodynamic approach with two-time-scale structure is proposed. An adaptive leader-following consensus technique is adapted to dynamically adjust parameters according to the degree of consensus violation, so as to quickly obtain accurate estimation information of other players' actions which facilitates the evaluation of its own cost. Benefitting from the unique structure of the approach based on primal dual and adaptive penalty methods, the players' actions enter the constraints while completing the seeking for GNE. As a result, the neurodynamic approach is completely distributed, and prior estimation of penalty parameters is avoided. Finally, two engineering examples of power system game and company capacity allocation verify the effectiveness and feasibility of the neurodynamic approach.

Optimized Backstepping Consensus Control Using Reinforcement Learning for a Class of Nonlinear Strict-Feedback-Dynamic Multi-Agent Systems

In this article, an optimized leader-following consensus control scheme is proposed for the nonlinear strict-feedback-dynamic multi-agent system by learning from the controlling idea of optimized backstepping technique, which designs the virtual and actual controls of backstepping to be the optimized solution of corresponding subsystems so that the entire backstepping control is optimized. Since this control needs to not only ensure the optimizing system performance but also synchronize the multiple system state variables, it is an interesting and challenging topic. In order to achieve this optimized control, the neural network approximation-based reinforcement learning (RL) is performed under critic-actor architecture. In most of the existing RL-based optimal controls, since both the critic and actor RL updating laws are derived from the negative gradient of square of the Hamilton-Jacobi-Bellman (HJB) equation's approximation, which contains multiple nonlinear terms, their algorithm are inevitably intricate. However, the proposed optimized control derives the RL updating laws from the negative gradient of a simple positive function, which is correlated with the HJB equation; hence, it can be significantly simple in the algorithm. Meanwhile, it can also release two general conditions, known dynamic and persistence excitation, which are required in most of the RL-based optimal controls. Therefore, the proposed optimized scheme can be a natural selection for the high-order nonlinear multi-agent control. Finally, the effectiveness is demonstrated by both theory and simulation.

Observer-Based Neuro-Adaptive Optimized Control of Strict-Feedback Nonlinear Systems With State Constraints

This article proposes an adaptive neural network (NN) output feedback optimized control design for a class of strict-feedback nonlinear systems that contain unknown internal dynamics and the states that are immeasurable and constrained within some predefined compact sets. NNs are used to approximate the unknown internal dynamics, and an adaptive NN state observer is developed to estimate the immeasurable states. By constructing a barrier type of optimal cost functions for subsystems and employing an observer and the actor-critic architecture, the virtual and actual optimal controllers are developed under the framework of backstepping technique. In addition to ensuring the boundedness of all closed-loop signals, the proposed strategy can also guarantee that system states are confined within some preselected compact sets all the time. This is achieved by means of barrier Lyapunov functions which have been successfully applied to various kinds of nonlinear systems such as strict-feedback and pure-feedback dynamics. Besides, our developed optimal controller requires less conditions on system dynamics than some existing approaches concerning optimal control. The effectiveness of the proposed optimal control approach is eventually validated by numerical as well as practical examples.

Pattern-Moving-Based Partial Form Dynamic Linearization Model Free Adaptive Control for a Class of Nonlinear Systems

This work addresses a pattern-moving-based partial form dynamic linearization model free adaptive control (P-PFDL-MFAC) scheme and illustrates the bounded convergence of its tracking error for a class of unknown nonaffine nonlinear discrete-time systems. The concept of pattern moving is to take the pattern class of the system output condition as a dynamic operation variable, and the control purpose is to ensure that the system outputs belong to a certain pattern class or some desired pattern classes. The P-PFDL-MFAC scheme mainly includes a modified tracking control law, a deviation estimation algorithm and a pseudo-gradient (PG) vector estimation algorithm. The classification-metric deviation is considered as an external disturbance, which is caused by the process of establishing the pattern-moving-based system dynamics description, and an improved cost function is proposed from the perspective of a two-player zero-sum game (TP-ZSG). The bounded convergence of the tracking error is rigorously proven by the contraction mapping principle, and the validity of the theoretical results is verified by simulation examples.

Event-triggered distributed zero-sum differential game for nonlinear multi-agent systems using adaptive dynamic programming

In this paper, to reduce the computational and communication burden, the event-triggered distributed zero-sum differential game problem for multi-agent systems is investigated. Firstly, based on the Minimax principle, an adaptive event-triggered distributed iterative differential game strategy is derived with an adaptive triggering condition for updating the control scheme aperiodically. Then, to implement this proposed strategy, the solution of coupled Hamilton-Jacobi-Isaacs​ (HJI) equation is approximated by constructing the critic neural network (NN). In order to further relax the restrictive persistent of excitation (PE) condition, a novel PE-free updating law is designed by using the experience replay method. Then, the distributed event-triggered nonlinear system is expressed as an impulsive dynamical system. After analyzing the stability, the developed strategy ensures the uniformly ultimately bounded (UUB) of all the closed-loop signals. Moreover, the minimal intersample time is proved to be lower bounded, which avoids the infamous Zeno behavior. Finally, the simulation results show that the number of controller update is reduced obviously, which saves the computational and communication resources.