Event-Triggered Multigradient Recursive Reinforcement Learning Tracking Control for Multiagent Systems

Adaptive Optimal Tracking Control of an Underactuated Surface Vessel Using Actor-Critic Reinforcement Learning

In this article, we present an adaptive reinforcement learning optimal tracking control (RLOTC) algorithm for an underactuated surface vessel subject to modeling uncertainties and time-varying external disturbances. By integrating backstepping technique with the optimized control design, we show that the desired optimal tracking performance of vessel control is guaranteed due to the fact that the virtual and actual control inputs are designed as optimized solutions of every subsystem. To enhance the robustness of vessel control systems, we employ neural network (NN) approximators to approximate uncertain vessel dynamics and present adaptive control technique to estimate the upper boundedness of external disturbances. Under the reinforcement learning framework, we construct actor-critic networks to solve the Hamilton-Jacobi-Bellman equations corresponding to subsystems of surface vessel to achieve the optimized control. The optimized control algorithm can synchronously train the adaptive parameters not only for actor-critic networks but also for NN approximators and adaptive control. By Lyapunov stability theorem, we show that the RLOTC algorithm can ensure the semiglobal uniform ultimate boundedness of the closed-loop systems. Compared with the existing reinforcement learning control results, the presented RLOTC algorithm can compensate for uncertain vessel dynamics and unknown disturbances, and obtain the optimized control performance by considering optimization in every backstepping design. Simulation studies on an underactuated surface vessel are given to illustrate the effectiveness of the RLOTC algorithm.

Neural Network Control of Underactuated Surface Vehicles With Prescribed Trajectory Tracking Performance

This article is concerned with the fast and accurate trajectory tracking control problem for a sort of underactuated surface vehicle under model uncertainties and environmental disturbances. A novel neural networks (NNs)-based prescribed performance control strategy is proposed to solve the problem. In the control design, a new type of performance function is constructed which provides a way to predefine the settling time and accuracy, straightforward. Then, a pair of barrier functions are employed to combat not only the position error but also the virtual control input. This evades the possible singularity or discontinuity of the control solution. Next, an initialization technique is exploited, removing the requirement for the initial condition of the control system. Finally, two NNs are employed to deal with the unknown ship nonlinearities. The performance analysis not only demonstrates the effectiveness of the proposed approach but also reveals its robustness against disturbances and unknown reference trajectory derivatives. There is, thus, no need to acquire such knowledge or employ specialized tools to handle disturbances. The theoretical findings are illustrated by a simulation study.

An Integrated Multitasking Intelligent Bearing Fault Diagnosis Scheme Based on Representation Learning Under Imbalanced Sample Condition

Accurate bearing fault diagnosis is of great significance of the safety and reliability of rotary mechanical system. In practice, the sample proportion between faulty data and healthy data in rotating mechanical system is imbalanced. Furthermore, there are commonalities between the bearing fault detection, classification, and identification tasks. Based on these observations, this article proposes a novel integrated multitasking intelligent bearing fault diagnosis scheme with the aid of representation learning under imbalanced sample condition, which realizes bearing fault detection, classification, and unknown fault identification. Specifically, in the unsupervised condition, a bearing fault detection approach based on modified denoising autoencoder (DAE) with self-attention mechanism for bottleneck layer (MDAE-SAMB) is proposed in the integrated scheme, which only uses the healthy data for training. The self-attention mechanism is introduced into the neurons in the bottleneck layer, which can assign different weights to the neurons in the bottleneck layer. Moreover, the transfer learning based on representation learning is proposed for few-shot fault classification. Only a few fault samples are used for offline training, and high-accuracy online bearing fault classification is achieved. Finally, according to the known fault data, the unknown bearing faults can be effectively identified. A bearing dataset generated by rotor dynamics experiment rig (RDER) and a public bearing dataset demonstrates the applicability of the proposed integrated fault diagnosis scheme.

Time-varying gain extended state observer-based adaptive optimal control for disturbed unmanned helicopter

In this paper, the robust adaptive optimal tracking control problem is addressed for the disturbed unmanned helicopter based on the time-varying gain extended state observer (TVGESO) and adaptive dynamic programming (ADP) methods. Firstly, a novel TVGESO is developed to tackle the unknown disturbance, which can overcome the drawback of initial peaking phenomenon in the traditional linear ESO method. Meanwhile, compared with the nonlinear ESO, the proposed TVGESO possesses easier and rigorous stability analysis process. Subsequently, the optimal tracking control issue for the original unmanned helicopter system is transformed into an optimization stabilization problem. By means of the ADP and neural network techniques, the feedforward controller and optimal feedback controller are skillfully designed. Compared with the conventional backstepping approach, the designed anti-disturbance optimal controller can make the unmanned helicopter accomplish the tracking task with less energy. Finally, simulation comparisons demonstrate the validity of the developed control scheme.

Distributed Dynamic Event-Triggered Control to Leader-Following Consensus of Nonlinear Multi-Agent Systems with Directed Graphs

This paper investigates achieving leader-following consensus in a class of multi-agent systems with nonlinear dynamics. Initially, it introduces a dynamic event-triggered strategy designed to effectively alleviate the strain on the system's communication resources. Subsequently, a distributed control strategy is proposed and implemented in the nonlinear leader-follower system using the dynamic event-triggered mechanism, aiming to ensure synchronization across all nodes at an exponential convergence speed. Thirdly, the research shows that under the dynamic event-triggered strategy the minimum event interval of any two consecutive triggers guarantees the elimination of Zeno behavior. Lastly, the validity of the calculation results is verified by a simulation example.

Dynamic event-triggered controller design for nonlinear systems: Reinforcement learning strategy

The current investigation aims at the optimal control problem for discrete-time nonstrict-feedback nonlinear systems by invoking the reinforcement learning-based backstepping technique and neural networks. The dynamic-event-triggered control strategy introduced in this paper can alleviate the communication frequency between the actuator and controller. Based on the reinforcement learning strategy, actor-critic neural networks are employed to implement the n-order backstepping framework. Then, a neural network weight-updated algorithm is developed to minimize the computational burden and avoid the local optimal problem. Furthermore, a novel dynamic-event-triggered strategy is introduced, which can remarkably outperform the previously studied static-event-triggered strategy. Moreover, combined with the Lyapunov stability theory, all signals in the closed-loop system are strictly proven to be semiglobal uniformly ultimately bounded. Finally, the practicality of the offered control algorithms is further elucidated by the numerical simulation examples.

Optimal Tracking Control of a Nonlinear Multiagent System Using Q-Learning via Event-Triggered Reinforcement Learning

This article offers an optimal control tracking method using an event-triggered technique and the internal reinforcement Q-learning (IrQL) algorithm to address the tracking control issue of unknown nonlinear systems with multiple agents (MASs). Relying on the internal reinforcement reward (IRR) formula, a Q-learning function is calculated, and then the iteration IRQL method is developed. In contrast to mechanisms triggered by time, an event-triggered algorithm reduces the rate of transmission and computational load, since the controller may only be upgraded when the predetermined triggering circumstances are met. In addition, in order to implement the suggested system, a neutral reinforce-critic-actor (RCA) network structure is created that may assess the indices of performance and online learning of the event-triggering mechanism. This strategy is intended to be data-driven without having in-depth knowledge of system dynamics. We must develop the event-triggered weight tuning rule, which only modifies the parameters of the actor neutral network (ANN) in response to triggering cases. In addition, a Lyapunov-based convergence study of the reinforce-critic-actor neutral network (NN) is presented. Lastly, an example demonstrates the accessibility and efficiency of the suggested approach.

Fault-tolerant control for nonlinear systems with a dead zone: Reinforcement learning approach

This paper focuses on the adaptive reinforcement learning-based optimal control problem for standard nonstrict-feedback nonlinear systems with the actuator fault and an unknown dead zone. To simultaneously reduce the computational complexity and eliminate the local optimal problem, a novel neural network weight updated algorithm is presented to replace the classic gradient descent method. By utilizing the backstepping technique, the actor critic-based reinforcement learning control strategy is developed for high-order nonlinear nonstrict-feedback systems. In addition, two auxiliary parameters are presented to deal with the input dead zone and actuator fault respectively. All signals in the system are proven to be semi-globally uniformly ultimately bounded by Lyapunov theory analysis. At the end of the paper, some simulation results are shown to illustrate the remarkable effect of the proposed approach.

Dynamic Event-Sampled Control of Interconnected Nonlinear Systems Using Reinforcement Learning

We develop a decentralized dynamic event-based control strategy for nonlinear systems subject to matched interconnections. To begin with, we introduce a dynamic event-based sampling mechanism, which relies on the system's states and the variables generated by time-based differential equations. Then, we prove that the decentralized event-based controller for the whole system is composed of all the optimal event-based control policies of nominal subsystems. To derive these optimal event-based control policies, we design a critic-only architecture to solve the related event-based Hamilton-Jacobi-Bellman equations in the reinforcement learning framework. The implementation of such an architecture uses only critic neural networks (NNs) with their weight vectors being updated through the gradient descent method together with concurrent learning. After that, we demonstrate that the asymptotic stability of closed-loop nominal subsystems and the uniformly ultimate boundedness stability of critic NNs' weight estimation errors are guaranteed by using Lyapunov's approach. Finally, we provide simulations of a matched nonlinear-interconnected plant to validate the present theoretical claims.