Observer-Based Neural Control of N-Link Flexible-Joint Robots

Adaptive Optimal Tracking Control of an Underactuated Surface Vessel Using Actor-Critic Reinforcement Learning

In this article, we present an adaptive reinforcement learning optimal tracking control (RLOTC) algorithm for an underactuated surface vessel subject to modeling uncertainties and time-varying external disturbances. By integrating backstepping technique with the optimized control design, we show that the desired optimal tracking performance of vessel control is guaranteed due to the fact that the virtual and actual control inputs are designed as optimized solutions of every subsystem. To enhance the robustness of vessel control systems, we employ neural network (NN) approximators to approximate uncertain vessel dynamics and present adaptive control technique to estimate the upper boundedness of external disturbances. Under the reinforcement learning framework, we construct actor-critic networks to solve the Hamilton-Jacobi-Bellman equations corresponding to subsystems of surface vessel to achieve the optimized control. The optimized control algorithm can synchronously train the adaptive parameters not only for actor-critic networks but also for NN approximators and adaptive control. By Lyapunov stability theorem, we show that the RLOTC algorithm can ensure the semiglobal uniform ultimate boundedness of the closed-loop systems. Compared with the existing reinforcement learning control results, the presented RLOTC algorithm can compensate for uncertain vessel dynamics and unknown disturbances, and obtain the optimized control performance by considering optimization in every backstepping design. Simulation studies on an underactuated surface vessel are given to illustrate the effectiveness of the RLOTC algorithm.

Distributed Optimal Attitude Synchronization Control of Multiple QUAVs via Adaptive Dynamic Programming

This article proposes a distributed optimal attitude synchronization control strategy for multiple quadrotor unmanned aerial vehicles (QUAVs) through the adaptive dynamic programming (ADP) algorithm. The attitude systems of QUAVs are modeled as affine nominal systems subject to parameter uncertainties and external disturbances. Considering attitude constraints in complex flying environments, a one-to-one mapping technique is utilized to transform the constrained systems into equivalent unconstrained systems. An improved nonquadratic cost function is constructed for each QUAV, which reflects the requirements of robustness and the constraints of control input simultaneously. To overcome the issue that the persistence of excitation (PE) condition is difficult to meet, a novel tuning rule of critic neural network (NN) weights is developed via the concurrent learning (CL) technique. In terms of the Lyapunov stability theorem, the stability of the closed-loop system and the convergence of critic NN weights are proved. Finally, simulation results on multiple QUAVs show the effectiveness of the proposed control strategy.

Effects of Two-Channel Noise and Packet Loss on Performance of Information Time Delay Systems

The performance limitations of multiple-input multiple-output (MIMO) information time delay system (ITDS) with packet loss, codec and white Gaussian noise (WGN) are investigated in this article. By using the spectrum decomposition technique, inner-outer factorization, and partial factorization techniques, the expression of performance limitations is obtained under the two-degree-of-freedom (2DOF) compensator. The theoretical analysis results demonstrate that the system performance is related to the time delay, non-minimum phase (NMP) zeros, unstable zeros and their directions in a given device. In addition, WGN, packet loss and codec also impact the performance. Finally, the theoretical results are verified by simulation examples. Simulation results show that packet loss rate and encoding and decoding have a greater impact on system performance.

Reinforcement learning-based consensus control for MASs with intermittent constraints

In this article, an adaptive optimal consensus control problem is studied for multiagent systems in the strict-feedback structure with intermittent constraints (the constraints appear intermittently). More specifically, by designing a novel switch-like function and an improved coordinate transformation, the constrained states are converted into unconstrained states, and the problem of intermittent constraints is resolved without requiring "feasibility conditions". In addition, using the composite learning algorithm and neural networks to construct the identifier, a simplified identifier-actor-critic-based reinforcement learning strategy is proposed to obtain the approximate optimal controller under the framework of backstepping. Meanwhile, with the aid of the nonlinear dynamic surface control technique, the issue of "explosion of complexity" in backstepping is removed, and the requirements for filter parameters are loosened. Based on Lyapunov stability theory, it is demonstrated that all signals in the closed-loop system are bounded. Finally, two simulation examples are used to verify the effectiveness of the proposed method.

Optimal tracking performance of networked control systems under communication constraints

This paper investigates the optimal tracking performance (OTP) of multiple-input multiple-output discrete-time communication-constrained systems by thinking about Denial of Service (DoS) attacks, codecs and additive Gaussian white noise under energy constraints. The non-cooperative relationship between DoS attacks and intrusion detection systems (IDS) is analyzed using repeated game theory. A penalty mechanism is constructed to force the attackers to adopt a cooperative strategy, thus improving the system performance. Partial factorization and spectrum decomposition are used to provide the OTP for systems. The results demonstrate that the systems' OTP are linked to intrinsic characteristics like non-minimum phase zeros and unstable poles. Finally, concrete examples are shown that the results are accurate.