Neural robust stabilization via event-triggering mechanism and adaptive learning technique

ADP-based fault-tolerant consensus control for multiagent systems with irregular state constraints

This paper investigates the consensus control issue for nonlinear multiagent systems (MASs) subject to irregular state constraints and actuator faults using an adaptive dynamic programming (ADP) algorithm. Unlike the regular state constraints considered in previous studies, this paper addresses irregular state constraints that may exhibit asymmetry, time variation, and can emerge or disappear during operation. By developing a system transformation method based on one-to-one state mapping, equivalent unconstrained MASs can be obtained. Subsequently, a finite-time distributed observer is designed to estimate the state information of the leader, and the consensus control problem is transformed into the tracking control problem for each agent to ensure that actuator faults of any agent cannot affect its neighboring agents. Then, a critic-only ADP-based fault tolerant control strategy, which consists of the optimal control policy for nominal system and online fault compensation for time-varying addictive faults, is proposed to achieve optimal tracking control. To enhance the learning efficiency of critic neural networks (NNs), an improved weight learning law utilizing stored historical data is employed, ensuring the convergence of critic NN weights towards ideal values under a finite excitation condition. Finally, a practical example of multiple manipulator systems is presented to demonstrate the effectiveness of the developed control method.

Event-Driven Off-Policy Reinforcement Learning for Control of Interconnected Systems

In this article, we introduce a novel approximate optimal decentralized control scheme for uncertain input-affine nonlinear-interconnected systems. In the proposed scheme, we design a controller and an event-triggering mechanism (ETM) at each subsystem to optimize a local performance index and reduce redundant control updates, respectively. To this end, we formulate a noncooperative dynamic game at every subsystem in which we collectively model the interconnection inputs and the event-triggering error as adversarial players that deteriorate the subsystem performance and model the control policy as the performance optimizer, competing against these adversarial players. To obtain a solution to this game, one has to solve the associated Hamilton-Jacobi-Isaac (HJI) equation, which does not have a closed-form solution even when the subsystem dynamics are accurately known. In this context, we introduce an event-driven off-policy integral reinforcement learning (OIRL) approach to learn an approximate solution to this HJI equation using artificial neural networks (NNs). We then use this NN approximated solution to design the control policy and event-triggering threshold at each subsystem. In the learning framework, we guarantee the Zeno-free behavior of the ETMs at each subsystem using the exploration policies. Finally, we derive sufficient conditions to guarantee uniform ultimate bounded regulation of the controlled system states and demonstrate the efficacy of the proposed framework with numerical examples.

An Approximate Neuro-Optimal Solution of Discounted Guaranteed Cost Control Design

The adaptive optimal feedback stabilization is investigated in this article for discounted guaranteed cost control of uncertain nonlinear dynamical systems. Via theoretical analysis, the guaranteed cost control problem involving a discounted utility is transformed to the design of a discounted optimal control policy for the nominal plant. The size of the neighborhood with respect to uniformly ultimately bounded stability is discussed. Then, for deriving the approximate optimal solution of the modified Hamilton-Jacobi-Bellman equation, an improved self-learning algorithm under the framework of adaptive critic designs is established. It facilitates the neuro-optimal control implementation without an additional requirement of the initial admissible condition. The simulation verification toward several dynamics is provided, involving the F16 aircraft plant, in order to illustrate the effectiveness of the discounted guaranteed cost control method.

Event-Triggered Adaptive Optimal Control With Output Feedback: An Adaptive Dynamic Programming Approach

This article presents an event-triggered output-feedback adaptive optimal control method for continuous-time linear systems. First, it is shown that the unmeasurable states can be reconstructed by using the measured input and output data. An event-based feedback strategy is then proposed to reduce the number of controller updates and save communication resources. The discrete-time algebraic Riccati equation is iteratively solved through event-triggered adaptive dynamic programming based on both policy iteration (PI) and value iteration (VI) methods. The convergence of the proposed algorithm and the closed-loop stability is carried out by using the Lyapunov techniques. Two numerical examples are employed to verify the effectiveness of the design methodology.

Neural optimal tracking control of constrained nonaffine systems with a wastewater treatment application

In this paper, we aim to solve the optimal tracking control problem for a class of nonaffine discrete-time systems with actuator saturation. First, a data-based neural identifier is constructed to learn the unknown system dynamics. Then, according to the expression of the trained neural identifier, we can obtain the steady control corresponding to the reference trajectory. Next, by involving the iterative dual heuristic dynamic programming algorithm, the new costate function and the tracking control law are developed. Two other neural networks are used to estimate the costate function and approximate the tracking control law. Considering approximation errors of neural networks, the stability analysis of the proposed algorithm for the specific systems is provided by introducing the Lyapunov approach. Finally, via conducting simulation and comparison, the superiority of the developed optimal tracking method is confirmed. Moreover, the trajectory tracking performance of the wastewater treatment application is also involved for further verifying the proposed approach.

Compensator-critic structure-based event-triggered decentralized tracking control of modular robot manipulators: theory and experimental verification

This paper presents a novel compensator-critic structure-based event-triggered decentralized tracking control of modular robot manipulators (MRMs). On the basis of subsystem dynamics under joint torque feedback (JTF) technique, the proposed tracking error fusion function, which includes position error and velocity error, is utilized to construct performance index function. By analyzing the dynamic uncertainties, a local dynamic information-based robust controller is designed to engage the model uncertainty compensation. Based on adaptive dynamic programming (ADP) algorithm and the event-triggered mechanism, the decentralized tracking control is obtained by solving the event-triggered Hamilton–Jacobi–Bellman equation (HJBE) with the critic neural network (NN). The tracking error of the closed-loop manipulators system is proved to be ultimately uniformly bounded (UUB) using the Lyapunov stability theorem. Finally, experimental results illustrate the effectiveness of the developed control method.

Safe Intermittent Reinforcement Learning With Static and Dynamic Event Generators

In this article, we present an intermittent framework for safe reinforcement learning (RL) algorithms. First, we develop a barrier function-based system transformation to impose state constraints while converting the original problem to an unconstrained optimization problem. Second, based on optimal derived policies, two types of intermittent feedback RL algorithms are presented, namely, a static and a dynamic one. We finally leverage an actor/critic structure to solve the problem online while guaranteeing optimality, stability, and safety. Simulation results show the efficacy of the proposed approach.

Event-driven H∞ control with critic learning for nonlinear systems

In this paper, we study an event-driven H_∞ control problem of continuous-time nonlinear systems. Initially, with the introduction of a discounted cost function, we convert the nonlinear H_∞ control problem into an event-driven nonlinear two-player zero-sum game. Then, we develop an event-driven Hamilton-Jacobi-Isaacs equation (HJIE) related to the two-player zero-sum game. After that, we propose a novel event-triggering condition guaranteeing Zeno behavior not to happen. The triggering threshold in the newly proposed event-triggering condition can be kept positive without requiring to properly choose the prescribed level of disturbance attenuation. To solve the event-driven HJIE, we employ an adaptive critic architecture which contains a unique critic neural network (NN). The weight parameters used in the critic NN are tuned via the gradient descent method. After that, we carry out stability analysis of the hybrid closed-loop system based on Lyapunov's direct approach. Finally, we provide two nonlinear plants, including the pendulum system, to validate the proposed event-driven H_∞ control scheme.

Integral reinforcement learning based event-triggered control with input saturation

In this paper, a novel integral reinforcement learning (IRL)-based event-triggered adaptive dynamic programming scheme is developed for input-saturated continuous-time nonlinear systems. By using the IRL technique, the learning system does not require the knowledge of the drift dynamics. Then, a single critic neural network is designed to approximate the unknown value function and its learning is not subjected to the requirement of an initial admissible control. In order to reduce computational and communication costs, the event-triggered control law is designed. The triggering threshold is given to guarantee the asymptotic stability of the control system. Two examples are employed in the simulation studies, and the results verify the effectiveness of the developed IRL-based event-triggered control method.

Adaptive Tracking Control of Wheeled Inverted Pendulums With Periodic Disturbances

This paper reports our study on adaptive tracking control for a mobile-wheeled inverted pendulum with periodic disturbances and parametric uncertainties. With an appropriate reduced dynamic model, incorporating repetitive learning strategies with dynamic decoupling and related adaptive control techniques, a novel controller is successfully constructed to ensure that the output tracking errors of the system will stay within a small neighborhood around zero and all of the other signals are semiglobal uniform bounded. Meanwhile, only one parameter estimation is used for adaptive controller design, which overcomes the problem of over-parametrization. Furthermore, a required condition of period identifier mechanisms is proposed. Finally, detailed simulation results are presented to demonstrate the effectiveness of the proposed control schemes.

Improved value iteration for neural-network-based stochastic optimal control design

In this paper, a novel value iteration adaptive dynamic programming (ADP) algorithm is presented, which is called an improved value iteration ADP algorithm, to obtain the optimal policy for discrete stochastic processes. In the improved value iteration ADP algorithm, for the first time we propose a new criteria to verify whether the obtained policy is stable or not for stochastic processes. By analyzing the convergence properties of the proposed algorithm, it is shown that the iterative value functions can converge to the optimum. In addition, our algorithm allows the initial value function to be an arbitrary positive semi-definite function. Finally, two simulation examples are presented to validate the effectiveness of the developed method.

Adaptive Tracking Control of Surface Vessel Using Optimized Backstepping Technique

In this paper, a tracking control approach for surface vessel is developed based on the new control technique named optimized backstepping (OB), which considers optimization as a backstepping design principle. Since surface vessel systems are modeled by second-order dynamic in strict feedback form, backstepping is an ideal technique for finishing the tracking task. In the backstepping control of surface vessel, the virtual and actual controls are designed to be the optimized solutions of corresponding subsystems, therefore the overall control is optimized. In general, optimization control is designed based on the solution of Hamilton-Jacobi-Bellman equation. However, solving the equation is very difficult or even impossible due to the inherent nonlinearity and complexity. In order to overcome the difficulty, the reinforcement learning (RL) strategy of actor-critic architecture is usually considered, of which the critic and actor are utilized for evaluating the control performance and executing the control behavior, respectively. By employing the actor-critic RL algorithm for both virtual and actual controls of the vessel, it is proven that the desired optimizing and tracking performances can be arrived. Simulation results further demonstrate effectiveness of the proposed surface vessel control.