Event-Triggered Reinforcement Learning-Based Adaptive Tracking Control for Completely Unknown Continuous-Time Nonlinear Systems

Data-Driven Finite-Horizon H∞ Tracking Control With Event-Triggered Mechanism for the Continuous-Time Nonlinear Systems

In this article, the neural network (NN)-based adaptive dynamic programming (ADP) event-triggered control method is presented to obtain the near-optimal control policy for the model-free finite-horizon H∞ optimal tracking control problem with constrained control input. First, using available input-output data, a data-driven model is established by a recurrent NN (RNN) to reconstruct the unknown system. Then, an augmented system with event-triggered mechanism is obtained by a tracking error system and a command generator. We present a novel event-triggering condition without Zeno behavior. On this basis, the relationship between event-triggered Hamilton-Jacobi-Isaacs (HJI) equation and time-triggered HJI equation is given in Theorem 3. Since the solution of the HJI equation is time-dependent for the augmented system, the time-dependent activation functions of NNs are considered. Moreover, an extra error is incorporated to satisfy the terminal constraints of cost function. This adaptive control pattern finds, in real time, approximations of the optimal value while also ensuring the uniform ultimate boundedness of the closed-loop system. Finally, the effectiveness of the proposed near-optimal control pattern is verified by two simulation examples.

Adaptive Multigradient Recursive Reinforcement Learning Event-Triggered Tracking Control for Multiagent Systems

This article proposes a fault-tolerant adaptive multigradient recursive reinforcement learning (RL) event-triggered tracking control scheme for strict-feedback discrete-time multiagent systems. The multigradient recursive RL algorithm is used to avoid the local optimal problem that may exist in the gradient descent scheme. Different from the existing event-triggered control results, a new lemma about the relative threshold event-triggered control strategy is proposed to handle the compensation error, which can improve the utilization of communication resources and weaken the negative impact on tracking accuracy and closed-loop system stability. To overcome the difficulty caused by sensor fault, a distributed control method is introduced by adopting the adaptive compensation technique, which can effectively decrease the number of online estimation parameters. Furthermore, by using the multigradient recursive RL algorithm with less learning parameters, the online estimation time can be effectively reduced. The stability of closed-loop multiagent systems is proved by using the Lyapunov stability theorem, and it is verified that all signals are semiglobally uniformly ultimately bounded. Finally, two simulation examples are given to show the availability of the presented control scheme.

Prescribed Performance Fault-Tolerant Control for Uncertain Nonlinear MIMO System Using Actor-Critic Learning Structure

This article studies the prescribed performance fault-tolerant control problem for a class of uncertain nonlinear multi-input and multioutput systems. A learning-based fault-tolerant controller is proposed to achieve the asymptotic stability, without requiring a priori knowledge of the system dynamics. To deal with the prescribed performance, a new error transformation function is introduced to convert the constrained error dynamics into an equivalent unconstrained one. Under the actor-critic learning structure, a continuous-time long-term performance index is presented to evaluate the current control behavior. Then, a critic network is used to approximate the designed performance index and provide a reinforcement signal to the action network. Based on the robust integral of the sign of error feedback control method, an action network-based controller is developed. It is shown by the Lyapunov approach that the tracking error can converge to zero asymptotically with the prescribed performance guaranteed. Simulation results are provided to validate the feasibility and effectiveness of the proposed control scheme.

Actor-critic learning based coordinated control for a dual-arm robot with prescribed performance and unknown backlash-like hysteresis

In this paper, we focus on the tracking problem of a dual-arm robot (DAR) with prescribed performance and unknown input backlash-like hysteresis. Considering this problem, adaptive coordinated control with actor-critic (AC) design is proposed. Motivated by the increasing control requirements, prescribed performance is imposed on the DAR system to guarantee the tracking performance. In order to improve the self-learning ability and handle the problems caused by the input backlash-like hysteresis and system uncertainty, AC learning (ACL) algorithm is introduced. Through the cost function about tracking errors, a critic network is adopted to judge the control performance. An actor network is adopted to obtain the control input based on the critic result, where the system uncertainty and unknown part of the input backlash-like hysteresis are approximated by neural networks (NNs). In addition, the system stability is proven by the Lyapunov direct method. Numerical simulation is finally conducted to further testify the validity of the proposed coordinated control with AC design for the DAR system.

Dynamic Event-Sampled Control of Interconnected Nonlinear Systems Using Reinforcement Learning

We develop a decentralized dynamic event-based control strategy for nonlinear systems subject to matched interconnections. To begin with, we introduce a dynamic event-based sampling mechanism, which relies on the system's states and the variables generated by time-based differential equations. Then, we prove that the decentralized event-based controller for the whole system is composed of all the optimal event-based control policies of nominal subsystems. To derive these optimal event-based control policies, we design a critic-only architecture to solve the related event-based Hamilton-Jacobi-Bellman equations in the reinforcement learning framework. The implementation of such an architecture uses only critic neural networks (NNs) with their weight vectors being updated through the gradient descent method together with concurrent learning. After that, we demonstrate that the asymptotic stability of closed-loop nominal subsystems and the uniformly ultimate boundedness stability of critic NNs' weight estimation errors are guaranteed by using Lyapunov's approach. Finally, we provide simulations of a matched nonlinear-interconnected plant to validate the present theoretical claims.

Adaptive Sliding Mode Disturbance Observer and Deep Reinforcement Learning Based Motion Control for Micropositioners

The motion control of high-precision electromechanitcal systems, such as micropositioners, is challenging in terms of the inherent high nonlinearity, the sensitivity to external interference, and the complexity of accurate identification of the model parameters. To cope with these problems, this work investigates a disturbance observer-based deep reinforcement learning control strategy to realize high robustness and precise tracking performance. Reinforcement learning has shown great potential as optimal control scheme, however, its application in micropositioning systems is still rare. Therefore, embedded with the integral differential compensator (ID), deep deterministic policy gradient (DDPG) is utilized in this work with the ability to not only decrease the state error but also improve the transient response speed. In addition, an adaptive sliding mode disturbance observer (ASMDO) is proposed to further eliminate the collective effect caused by the lumped disturbances. The micropositioner controlled by the proposed algorithm can track the target path precisely with less than 1 μm error in simulations and actual experiments, which shows the sterling performance and the accuracy improvement of the controller.

Adaptive model-based dynamic event-triggered output feedback control of a robotic manipulator with disturbance

This paper focuses on the stable tracking control of the manipulator with constrained communication, unmeasurable velocity, and nonlinear uncertainties. An NN observer-depended output feedback scheme in the discrete-time domain is developed by virtue of the model-based dynamic event-triggered backstepping technique in the channel of sensor to controller. For generalizing the zero-order-holder (ZOH) implementation, a plant model is built to approximate the triggered states in the time flow, and according to which, the control law is fabricated. Based on model-based error events, we construct a dead-zone triggered condition with a dynamically adjustable threshold, making the threshold evolve with the system performance, to achieve flexible communication scheduling and avoid the accumulation of triggers in small tracking errors. The internal and external nonlinear uncertainties are online compensated by the neural network, and the aperiodic adaptive law is derived in the sense of control stability to save the computation. Finally, the conditions for semi-global ultimate uniform bounded (SGUUB) of all variables are given via impulse Lyapunov analysis, and a positive lower bound in the time interval between consecutive executions to guarantee the Zeno free behavior is obtained. Simulations are conducted on a three-link manipulator to illustrate the effectiveness of our method.

Adaptive neural tracking control of strict-feedback nonlinear systems with event-triggered state measurement

This paper investigates the adaptive neural tracking control of the strict-feedback nonlinear systems, where the states are measured in an event-triggered manner so as to save the communication resources. As the neural networks (NNs) account for the unknown dynamics of the system, the minimum learning parameters (MLPs) are extracted from the weights of the NNs and the upper bounds of the disturbances. The estimates of the MLPs are updated in an event-triggered manner to ensure the approximation ability of the NNs and the stability of the closed-loop system. An adaptive neural model is established to substitute for the original strict-feedback system and direct the design of the backstepping-based control laws. The states of this adaptive model are reset to the measured states of the original system when the triggering condition is violated. The triggering condition is constructed in the compound form and with the adaptive threshold. The dead-zone operator is involved to avoid the accumulation of triggering instants. In this paper, we notice the problem of "jumps of virtual control laws" for the event-triggered control (ETC) in the backstepping frame, and a detailed formulaic definition is given in section 2.2. To solve this problem, the first-order filters are fabricated to provide the continuous substitutes for virtual control laws. In addition, the "complexity explosion" generated by direct differentiating of virtual control laws can be averted. Through the proposed scheme, the closed-loop system can be viewed as an impulsive dynamic system, and the semi-globally uniformly ultimate boundedness (SGUUB) of all the errors is proved. Finally, two examples validate the feasibility of the proposed control scheme.

Model-Based Event-Triggered Tracking Control of Underactuated Surface Vessels With Minimum Learning Parameters

This article studies the model-based event-triggered control (ETC) for the tracking activity of the underactuated surface vessel (USV). Following this ideology, the continuous acquisition of states is no longer needed, and the communication traffic is reduced in the channel of sensor to controller. The control laws are fabricated in the frame of an adaptive model, which is renewed with the states of the original system whenever the triggering condition is violated. In the scheme, both internal and external uncertainties are approximated by the neural networks (NNs). To decrease the computing complexity, the minimum learning parameters (MLPs) are involved both in the adaptive model and the derived controller. The adaptive laws of only two MLPs are devised, and their updating only happens at triggering instants. Using the MLPs, an adaptive triggering condition is further derived. To avoid the "Zeno" phenomenon in small tracking errors, a dead-zone operator is designed for the triggering condition. Furthermore, we incorporate the dynamic surface control (DSC) into the controller design, such that the jumping of virtual control laws at triggering instants is smoothed and the problem of "complexity explosion" is circumvented. Through the techniques of the impulsive dynamic system and the direct Lyapunov function, the parameter setting for the DSC is derived to guarantee the semiglobal uniformly ultimate boundedness (SGUUB) of all the error signals in the closed-loop system. Finally, the effectiveness of the proposed scheme is validated through the simulation.