Event-Triggered Adaptive Dynamic Programming for Unmatched Uncertain Nonlinear Continuous-Time Systems

Advanced optimal tracking integrating a neural critic technique for asymmetric constrained zero-sum games

This paper investigates the optimal tracking issue for continuous-time (CT) nonlinear asymmetric constrained zero-sum games (ZSGs) by exploiting the neural critic technique. Initially, an improved algorithm is constructed to tackle the tracking control problem of nonlinear CT multiplayer ZSGs. Also, we give a novel nonquadratic function to settle the asymmetric constraints. One thing worth noting is that the method used in this paper to solve asymmetric constraints eliminates the strict restriction on the control matrix compared to the previous ones. Further, the optimal controls, the worst disturbances, and the tracking Hamilton-Jacobi-Isaacs equation are derived. Next, a single critic neural network is built to estimate the optimal cost function, thus obtaining the approximations of the optimal controls and the worst disturbances. The critic network weight is updated by the normalized steepest descent algorithm. Additionally, based on the Lyapunov method, the stability of the tracking error and the weight estimation error of the critic network is analyzed. In the end, two examples are offered to validate the theoretical results.

Neural Q-learning for discrete-time nonlinear zero-sum games with adjustable convergence rate

In this paper, an adjustable Q-learning scheme is developed to solve the discrete-time nonlinear zero-sum game problem, which can accelerate the convergence rate of the iterative Q-function sequence. First, the monotonicity and convergence of the iterative Q-function sequence are analyzed under some conditions. Moreover, by employing neural networks, the model-free tracking control problem can be overcome for zero-sum games. Second, two practical algorithms are designed to guarantee the convergence with accelerated learning. In one algorithm, an adjustable acceleration phase is added to the iteration process of Q-learning, which can be adaptively terminated with convergence guarantee. In another algorithm, a novel acceleration function is developed, which can adjust the relaxation factor to ensure the convergence. Finally, through a simulation example with the practical physical background, the fantastic performance of the developed algorithm is demonstrated with neural networks.

Output information-based intermittent optimal control for continuous-time nonlinear systems with unmatched uncertainties via adaptive dynamic programming

Intermittent control stands as a valuable strategy for resource conservation and cost reduction across diverse systems. Nonetheless, prevailing research is intractable to address the challenges posed by robust optimal intermittent control of nonlinear input-affine systems with unmatched uncertainties. This paper aims to fill this gap. Initially, we introduce an enhanced finite-time intermittent control approach to ensure stability within nonlinear dynamic systems harboring bounded errors. A neural networks (NNs) state observer is constructed to estimate system information. Subsequently, an optimal intermittent controller that operates within a finite time span, guaranteeing system stability by employing the Hamilton-Jacobi-Bellman (HJB) methodology. Furthermore, we devise an output information-based event-triggered intermittent (ETI) approach rooted in the robust adaptive dynamic programming (ADP) algorithm, furnishing an optimal intermittent control law. In this process, a critic NNs is introduced to estimate the cost function and optimal intermittent controller. Simulation results show that our proposed method is superior to existing intermittent control strategies.

Traffic Signal Control With Adaptive Online-Learning Scheme Using Multiple-Model Neural Networks

This article proposes a new traffic signal control algorithm to deal with unknown-traffic-system uncertainties and reduce delays in vehicle travel time. Unknown-traffic-system dynamics are approximated using a recurrent neural network (NN). To accurately identify the traffic system model, an online-learning scheme is developed to switch among a set of candidate NNs (i.e., multiple-model NNs) based on their estimation errors. Then, a bank of optimal signal-timing controllers is designed based on the online identification of the traffic system. Simulation studies have been carried out for the obtained control strategies using multiple-model NNs, and the desired results have been obtained. Moreover, compared with the widely used actuated traffic signal control schemes, it is shown that the proposed method can reduce vehicle travel delays and improve traffic system robustness.

Adaptive dynamic programming-based hierarchical decision-making of non-affine systems

In this paper, the problem of multiplayer hierarchical decision-making problem for non-affine systems is solved by adaptive dynamic programming. Firstly, the control dynamics are obtained according to the theory of dynamic feedback and combined with the original system dynamics to construct the affine augmented system. Thus, the non-affine multiplayer system is transformed into a general affine form. Then, the hierarchical decision problem is modeled as a Stackelberg game. In the Stackelberg game, the leader makes a decision based on the information of all followers, whereas the followers do not know each other's information and only obtain their optimal control strategy based on the leader's decision. Then, the augmented system is reconstructed by a neural network (NN) using input-output data. Moreover, a single critic NN is used to approximate the value function to obtain the optimal control strategy for each player. An extra term added to the weight update law makes the initial admissible control law no longer needed. According to the Lyapunov theory, the state of the system and the error of the weights of the NN are both uniformly ultimately bounded. Finally, the feasibility and validity of the algorithm are confirmed by simulation.

Adaptive Event-Triggered Finite-Time Control for Uncertain Time Delay Nonlinear System

In this article, adaptive event-triggered finite-time control is explored for uncertain nonlinear systems with time delay. First, to handle the time-varying state delays, the Lyapunov-Krasovskii function is used. Fuzzy-logic systems are used to deal with the unknown nonlinearities of the system. Notice that compared to the reporting achievements, our proposed virtual control laws are derivable by using the novel switch function, which avoids "singularity hindrance" problem. Moreover, the dynamic event-triggered controller is designed to reduce the communication pressure and we prove that the controller is Zeno free. Our proposed control strategy ensures that the tracking error is arbitrarily small in finite time and all variables of the closed-loop system remain bounded. Finally, to show the effectiveness of our control strategy, the simulation results are given.

Adaptive Iterative Learning Control for a Class of Nonlinear Strict-Feedback Systems With Unknown State Delays

In this article, an adaptive iterative learning control scheme is presented for a class of nonlinear parametric strict-feedback systems with unknown state delays, aiming to achieve the point-wise tracking of desired trajectory in a finite interval. The appropriate Lyapunov-Krasovskii functions are established to compensate the influence of time-delay uncertainties on the control systems. As the main features, the proposed approach integrates the command filter into the backstepping procedure to avoid the differential explosion problem that may occur with the increase of system order, and introduces the hyperbolic tangent functions into the learning controller to handle the singularity problem thus maintaining the continuity of input signal. The results of theoretical analysis and numerical simulation demonstrate that the tracking errors at the entire period will converge to a compact set along the iteration axis. Compared with the existing works, the proposed control scheme is promising to manifest the better performance and practicability owing to the learning mechanism, the dynamic model, as well as the implementation of controller.

Event-triggered adaptive optimal tracking control for nonlinear stochastic systems with dynamic state constraints

This paper investigates the issue of event-triggered adaptive optimal tracking control for uncertain nonlinear systems with stochastic disturbances and dynamic state constraints. To handle the dynamic state constraints, a novel unified tangent-type nonlinear mapping function is proposed. A neural networks (NNs)-based identifier is designed to cope with the stochastic disturbances. By utilizing adaptive dynamic programming (ADP) of identifier-actor-critic architecture and event triggering mechanism, the adaptive optimized event-triggered control (ETC) approach for the nonlinear stochastic system is first proposed. It is proven that the designed optimized ETC approach guarantees the robustness of the stochastic systems and the semi-globally uniformly ultimately bounded in the mean square of the NNs adaptive estimation error, and the Zeno behavior can be avoided. Simulations are offered to illustrate the effectiveness of the proposed control approach.

Event-Based Robust Optimal Consensus Control for Nonlinear Multiagent System With Local Adaptive Dynamic Programming

This article investigates the robust optimal consensus for nonlinear multiagent systems (MASs) through the local adaptive dynamic programming (ADP) approach and the event-triggered control method. Due to the nonlinearities in dynamics, the first part defines a novel measurement error to construct a distributed integral sliding-mode controller, and the consensus errors can approximately converge to the origin in a fixed time. Then, a modified cost function with augmented control is proposed to deal with the unmatched disturbances for the event-based optimal consensus controller. Specifically, a single network local ADP structure with novel concurrent learning is presented to approximate the optimal consensus policies, which guarantees the robustness of the MASs and the uniform ultimate boundedness (UUB) of the neural network (NN) weights' estimation error and relaxes the requirement of initial admissible control. Finally, an illustrative simulation verifies the effectiveness of the method.

Dynamic Event-Triggered Integral Sliding Mode Adaptive Optimal Tracking Control for Uncertain Nonlinear Systems

In this paper, we study the event-triggered integral sliding mode optimal tracking problem of nonlinear systems with matched and unmatched disturbances. The goal is to design an adaptive dynamic programming-based sliding-mode controller, which stabilizes the closed-loop system and guarantees the optimal performance of the sliding-mode dynamics. First, in order to remove the effects of the matched uncertainties, an event-triggered sliding mode controller is designed to force the state of the systems on the sliding mode surface without Zeno behavior. Second, another event-triggered controller is designed to suppress unmatched disturbances with a nearly optimal performance while also guaranteeing Zeno-free behavior. Finally, the benefits of the proposed algorithm are shown in comparison to several traditional triggering and learning-based mechanisms.

Dynamic Event-Sampled Control of Interconnected Nonlinear Systems Using Reinforcement Learning

We develop a decentralized dynamic event-based control strategy for nonlinear systems subject to matched interconnections. To begin with, we introduce a dynamic event-based sampling mechanism, which relies on the system's states and the variables generated by time-based differential equations. Then, we prove that the decentralized event-based controller for the whole system is composed of all the optimal event-based control policies of nominal subsystems. To derive these optimal event-based control policies, we design a critic-only architecture to solve the related event-based Hamilton-Jacobi-Bellman equations in the reinforcement learning framework. The implementation of such an architecture uses only critic neural networks (NNs) with their weight vectors being updated through the gradient descent method together with concurrent learning. After that, we demonstrate that the asymptotic stability of closed-loop nominal subsystems and the uniformly ultimate boundedness stability of critic NNs' weight estimation errors are guaranteed by using Lyapunov's approach. Finally, we provide simulations of a matched nonlinear-interconnected plant to validate the present theoretical claims.