Optimal Control-Based Adaptive NN Design for a Class of Nonlinear Discrete-Time Block-Triangular Systems

Dynamic event-triggered controller design for nonlinear systems: Reinforcement learning strategy

The current investigation aims at the optimal control problem for discrete-time nonstrict-feedback nonlinear systems by invoking the reinforcement learning-based backstepping technique and neural networks. The dynamic-event-triggered control strategy introduced in this paper can alleviate the communication frequency between the actuator and controller. Based on the reinforcement learning strategy, actor-critic neural networks are employed to implement the n-order backstepping framework. Then, a neural network weight-updated algorithm is developed to minimize the computational burden and avoid the local optimal problem. Furthermore, a novel dynamic-event-triggered strategy is introduced, which can remarkably outperform the previously studied static-event-triggered strategy. Moreover, combined with the Lyapunov stability theory, all signals in the closed-loop system are strictly proven to be semiglobal uniformly ultimately bounded. Finally, the practicality of the offered control algorithms is further elucidated by the numerical simulation examples.

Secure State Estimation of Nonlinear Cyber-Physical Systems Against DoS Attacks: A Multiobserver Approach

This article focuses on the problem of secure state estimation for cyber-physical systems (CPSs), whose physical plants are modeled as nonlinear strict-feedback systems. The measured output is sent to the designed observer over a wireless communication network subject to denial-of-service (DoS) attacks. Due to the energy constraints of the attackers, the attack duration is upper bounded. Under DoS attacks, the transmission is prevented, which worsens the estimation accuracy of the existing nonlinear observers significantly. To maintain the estimation performance, a novel multiobserver scheme and a switched algorithm are proposed by introducing the hold-input mechanism and the cascade observer technique. In comparison to the existing results, where the estimation error systems may be unstable during the attack time interval, the estimation error of the designed observer converges exponentially, such that the estimation performance is improved effectively. Finally, the theoretical findings are illustrated by simulation results.

Game-Based Backstepping Design for Strict-Feedback Nonlinear Multi-Agent Systems Based on Reinforcement Learning

In this article, the game-based backstepping control method is proposed for the high-order nonlinear multi-agent system with unknown dynamic and input saturation. Reinforcement learning (RL) is employed to get the saddle point solution of the tracking game between each agent and the reference signal for achieving robust control. Specifically, the approximate optimal solution of the established Hamilton-Jacobi-Isaacs (HJI) equation is obtained by policy iteration for each subsystem, and the single network adaptive critic (SNAC) architecture is used to reduce the computational burden. In addition, based on the separation operation of the error term from the derivative of the value function, we achieve the different proportions of the two agents in the game to realize the regulation of the final equilibrium point. Different from the general use of the neural network for system identification, the unknown nonlinear dynamic term is approximated based on the state difference obtained by the command filter. Furthermore, a sufficient condition is established to guarantee that the whole system and each subsystem included are uniformly ultimately bounded. Finally, simulation results are given to show the effectiveness of the proposed method.

Cooperative Optimal Controller and Its Application to Activated Sludge Process

With the increasing complexity and scale of activated sludge process (ASP), it is quite challenging to coordinate the performance indices with different time scales. To address this problem, a cooperative optimal controller (COC) is proposed to improve the operation performance in this paper. First, a cooperative optimal scheme is developed for designing the control system, where the different time-scale performance indices are formulated by two levels. Second, a data-driven surrogate-assisted optimization (DDSAO) algorithm is provided to optimize the cooperative objectives, where a surrogate model is established for evaluating the feasibility of optimal solutions based on the minimum squared error. Third, an adaptive predictive control strategy is investigated to derive the control laws for improving the tracking control performance. Finally, the proposed COC is tested on benchmark simulation model No. 1 (BSM1). The results demonstrate that the proposed COC is able to coordinate the multiple time-scale performance indices and achieve the competitive optimal control performance.

Deep Learning-Based Model Predictive Control for Continuous Stirred-Tank Reactor System

A continuous stirred-tank reactor (CSTR) system is widely applied in wastewater treatment processes. Its control is a challenging industrial-process-control problem due to great difficulty to achieve accurate system identification. This work proposes a deep learning-based model predictive control (DeepMPC) to model and control the CSTR system. The proposed DeepMPC consists of a growing deep belief network (GDBN) and an optimal controller. First, GDBN can automatically determine its size with transfer learning to achieve high performance in system identification, and it serves just as a predictive model of a controlled system. The model can accurately approximate the dynamics of the controlled system with a uniformly ultimately bounded error. Second, quadratic optimization is conducted to obtain an optimal controller. This work analyzes the convergence and stability of DeepMPC. Finally, the DeepMPC is used to model and control a second-order CSTR system. In the experiments, DeepMPC shows a better performance in modeling, tracking, and antidisturbance than the other state-of-the-art methods.

Adaptive Consensus Tracking Control of Uncertain Nonlinear Multiagent Systems With Predefined Accuracy

In this article, we consider the leader-follower consensus control problem of uncertain multiagent systems, aiming to achieve the improvement of system steady state and transient performance. To this end, a new adaptive neural control approach is proposed with a novel design of the Lyapunov function, which is generated with a class of positive functions. Guided by this idea, a series of smooth functions is incorporated into backstepping design and Lyapunov analysis to develop a performance-oriented controller. It is proved that the proposed controller achieves a perfect asymptotic consensus performance and a tunable L₂ transient performance of synchronization errors, whereas most existing results can only ensure the stability. Simulation demonstrates the obtained results.

Adaptive Reinforcement Learning Neural Network Control for Uncertain Nonlinear System With Input Saturation

In this paper, an adaptive neural network (NN) control problem is investigated for discrete-time nonlinear systems with input saturation. Radial-basis-function (RBF) NNs, including critic NNs and action NNs, are employed to approximate the utility functions and system uncertainties, respectively. In the previous works, a gradient descent scheme is applied to update weight vectors, which may lead to local optimal problem. To circumvent this problem, a multigradient recursive (MGR) reinforcement learning scheme is proposed, which utilizes both the current gradient and the past gradients. As a consequence, the MGR scheme not only eliminates the local optimal problem but also guarantees faster convergence rate than the gradient descent scheme. Moreover, the constraint of actuator input saturation is considered. The closed-loop system stability is developed by using the Lyapunov stability theory, and it is proved that all the signals in the closed-loop system are semiglobal uniformly ultimately bounded (SGUUB). Finally, the effectiveness of the proposed approach is further validated via some simulation results.

Event-Triggered Neural Control of Nonlinear Systems With Rate-Dependent Hysteresis Input Based on a New Filter

In controlling nonlinear uncertain systems, compensating for rate-dependent hysteresis nonlinearity is an important, yet challenging problem in adaptive control. In fact, it can be illustrated through simulation examples that instability is observed when existing control methods in canceling hysteresis nonlinearities are applied to the networked control systems (NCSs). One control difficulty that obstructs these methods is the design conflict between the quantized networked control signal and the rate-dependent hysteresis characteristics. So far, there is still no solution to this problem. In this paper, we consider the event-triggered control for NCSs subject to actuator rate-dependent hysteresis and failures. A new second-order filter is proposed to overcome the design conflict and used for control design. With the incorporation of the filter, a novel adaptive control strategy is developed from a neural network technique and a modified backstepping recursive design. It is proved that all the control signals are semiglobally uniformly ultimately bounded and the tracking error will converge to a tunable residual around zero.

A Repeatable Motion Scheme for Kinematic Control of Redundant Manipulators

To achieve closed trajectory motion planning of redundant manipulators, each joint angle has to be returned to its initial position. Most of the repeatable motion schemes have been proposed to solve kinematic problems considering only the initial desired position of each joint at first. Actually, it is very difficult for various joint angles of the robot arms to be positioned in the expected trajectory before moving. To construct an effective kinematic model, a novel optimal programming index based on a recurrent neural network is designed and analyzed in this paper. The repetitiveness and timeliness are presented and analyzed. Combining the kinematic equation constraint of manipulators, a repeatable motion scheme is formulated. In addition, the Lagrange multiplier theorem is introduced to prove that such a repeatable motion scheme can be converted into a time-varying linear equation. A finite-time neural network solver is constructed for the solution of the motion scheme. Simulation results for two different trajectories illustrate the accuracy and timeliness of the proposed motion scheme. Finally, two different repetitive schemes are compared and verified the optimal time for the novelty of the proposed kinematic scheme.

Two New Discrete-Time Neurodynamic Algorithms Applied to Online Future Matrix Inversion With Nonsingular or Sometimes-Singular Coefficient

In this paper, a high-precision general discretization formula using six time instants is first proposed to approximate the first-order derivative. Then, such a formula is studied to discretize two continuous-time neurodynamic models, both of which are derived by applying the neurodynamic approaches based on neural networks (i.e., zeroing neurodynamics and gradient neurodynamics). Originating from the general six-instant discretization (6ID) formula, a specific 6ID formula is further presented. Subsequently, two new discrete-time neurodynamic algorithms, i.e., 6ID-type discrete-time zeroing neurodynamic (DTZN) algorithm and 6ID-type discrete-time gradient neurodynamic (DTGN) algorithm, are proposed and investigated for online future matrix inversion (OFMI). In addition to analyzing the usual nonsingular situation of the coefficient, this paper investigates the sometimes-singular situation of the coefficient for OFMI. Finally, two illustrative numerical examples, including an application to the inverse-kinematic control of a PUMA560 robot manipulator, are provided to show respective characteristics and advantages of the proposed 6ID-type DTZN and DTGN algorithms for OFMI in different situations, where the coefficient matrix to be inverted is always-nonsingular or sometimes-singular during time evolution.

Adaptive -Learning for Data-Based Optimal Output Regulation With Experience Replay

In this paper, the data-based optimal output regulation problem of discrete-time systems is investigated. An off-policy adaptive -learning (QL) method is developed by using real system data without requiring the knowledge of system dynamics and the mathematical model of utility function. By introducing the -function, an off-policy adaptive QL algorithm is developed to learn the optimal -function. An adaptive parameter in the policy evaluation is used to achieve tradeoff between the current and future -functions. The convergence of adaptive QL algorithm is proved and the influence of the adaptive parameter is analyzed. To realize the adaptive QL algorithm with real system data, the actor-critic neural network (NN) structure is developed. The least-squares scheme and the batch gradient descent method are developed to update the critic and actor NN weights, respectively. The experience replay technique is employed in the learning process, which leads to simple and convenient implementation of the adaptive QL method. Finally, the effectiveness of the developed adaptive QL method is verified through numerical simulations.

Neural Network-Based Model-Free Adaptive Near-Optimal Tracking Control for a Class of Nonlinear Systems

In this paper, the receding horizon near-optimal tracking control problem about a class of continuous-time nonlinear systems with fully unknown dynamics is considered. The main challenges of this problem lie in two aspects: 1) most existing systems only restrict their considerations to the state feedback part while the input channel parameters are assumed to be known. This paper considers fully unknown system dynamics in both the state feedback channel and the input channel and 2) the optimal control of nonlinear systems requires the solution of nonlinear Hamilton-Jacobi-Bellman equations. Up to date, there are no systematic approaches in the existing literature to solve it accurately. A novel model-free adaptive near-optimal control method is proposed to solve this problem via utilizing the Taylor expansion-based problem relaxation, the universal approximation property of sigmoid neural networks, and the concept of sliding mode control. By making approximation for the performance index, it is first relaxed to a quadratic program, and then, a linear algebraic equation with unknown terms. An auxiliary system is designed to reconstruct the input-to-output property of the control systems with unknown dynamics, so as to tackle the difficulty caused by the unknown terms. Then, by considering the property of the sliding-mode surface, an explicit adaptive near-optimal control law is derived from the linear algebraic equation. Theoretical analysis shows that the auxiliary system is convergent, the resultant closed-loop system is asymptotically stable, and the performance index asymptomatically converges to optimal. An illustrative example and experimental results are presented, which substantiate the efficacy of the proposed method and verify the theoretical results.

Adaptive Critic Design for Pure-Feedback Discrete-Time MIMO Systems Preceded by Unknown Backlashlike Hysteresis

This paper concentrates on the adaptive critic design (ACD) issue for a class of uncertain multi-input multioutput (MIMO) nonlinear discrete-time systems preceded by unknown backlashlike hysteresis. The considered systems are in a block-triangular pure-feedback form, in which there exist nonaffine functions and couplings between states and inputs. This makes that the ACD-based optimal control becomes very difficult and complicated. To this end, the mean value theorem is employed to transform the original systems into input-output models. Based on the reinforcement learning algorithm, the optimal control strategy is established with an actor-critic structure. Not only the stability of the systems is ensured but also the performance index is minimized. In contrast to the previous results, the main contributions are: 1) it is the first time to build an ACD framework for such MIMO systems with unknown hysteresis and 2) an adaptive auxiliary signal is developed to compensate the influence of hysteresis. In the end, a numerical study is provided to demonstrate the effectiveness of the present method.

Identification of hinging hyperplane autoregressive exogenous model using efficient mixed-integer programming

A computationally efficient algorithm for hinging hyperplane autoregressive exogenous (HHARX) model identification via mixed-integer programming technique is proposed in this paper. The HHARX model is attractive since it accurately approximates a general nonlinear process as a sum of hinge functions and preserves the continuity even in a piecewise affine form. Traditional mixed-integer programming-based method for HHARX model identification can only be applied on small-scale input/output datasets due to its significant computational demands. The contribution of this paper is to develop a sequential optimization approach to build accurate HHARX model more efficiently on a relatively large number of experimental data. Moreover, the proposed framework can handle more difficult and practical cases in piecewise model identification, such as: limited submodel switching, missing output data and specified steady state. Finally, the efficiency and accuracy of the proposed computational scheme are demonstrated through modeling of two simulated examples and a pilot-scale heat exchanger.

Tracking Control of a Continuous Stirred Tank Reactor Using Direct and Tuned Reinforcement Learning Based Controllers

The need for linear model, of the nonlinear system, while tuning controllers limits the use of classic controllers. Also, the tuning procedure involves complex computations. This is further complicated when it is necessary to operate the nonlinear system under different operating constraints. Continues Stirred Tank Reactor (CSTR) is one of those non-linear systems which is studied extensively in control and chemical engineering due to its highly non-linear characteristics and its diverse operating range. This paper proposes two different control schemes based on reinforcement learning algorithm to achieve both servo as well as regulatory control. One approach is the direct application of Reinforcement Learning (RL) with ANN approximation and another is tuning of PID controller parameters using reinforcement learning. The main objective of this paper is to handle multiple set point control for the CSTR system using RL. The temperature of the CSTR system is controlled here for multiple setpoint changes. A comparative study is also done between the two proposed algorithm and from the test result, it is seen that direct RL approach with approximation performs better than tuning a PID using RL as oscillations and overshoot are less for direct RL approach. Also, the learning time for the direct RL based controller is lesser than the later.

Optimal Fault-Tolerant Control for Discrete-Time Nonlinear Strict-Feedback Systems Based on Adaptive Critic Design

This paper investigates the problem of optimal fault-tolerant control (FTC) for a class of unknown nonlinear discrete-time systems with actuator fault in the framework of adaptive critic design (ACD). A pivotal highlight is the adaptive auxiliary signal of the actuator fault, which is designed to offset the effect of the fault. The considered systems are in strict-feedback forms and involve unknown nonlinear functions, which will result in the causal problem. To solve this problem, the original nonlinear systems are transformed into a novel system by employing the diffeomorphism theory. Besides, the action neural networks (ANNs) are utilized to approximate a predefined unknown function in the backstepping design procedure. Combined the strategic utility function and the ACD technique, a reinforcement learning algorithm is proposed to set up an optimal FTC, in which the critic neural networks (CNNs) provide an approximate structure of the cost function. In this case, it not only guarantees the stability of the systems, but also achieves the optimal control performance as well. In the end, two simulation examples are used to show the effectiveness of the proposed optimal FTC strategy.

Approximation-Based Adaptive Neural Tracking Control of Nonlinear MIMO Unknown Time-Varying Delay Systems With Full State Constraints

This paper deals with the tracking control problem for a class of nonlinear multiple input multiple output unknown time-varying delay systems with full state constraints. To overcome the challenges which cause by the appearances of the unknown time-varying delays and full-state constraints simultaneously in the systems, an adaptive control method is presented for such systems for the first time. The appropriate Lyapunov-Krasovskii functions and a separation technique are employed to eliminate the effect of unknown time-varying delays. The barrier Lyapunov functions are employed to prevent the violation of the full state constraints. The singular problems are dealt with by introducing the signal function. Finally, it is proven that the proposed method can both guarantee the good tracking performance of the systems output, all states are remained in the constrained interval and all the closed-loop signals are bounded in the design process based on choosing appropriate design parameters. The practicability of the proposed control technique is demonstrated by a simulation study in this paper.

Command Filter-Based Adaptive Neural Tracking Controller Design for Uncertain Switched Nonlinear Output-Constrained Systems

In this paper, a new adaptive approximation-based tracking controller design approach is developed for a class of uncertain nonlinear switched lower-triangular systems with an output constraint using neural networks (NNs). By introducing a novel barrier Lyapunov function (BLF), the constrained switched system is first transformed into a new system without any constraint, which means the control objectives of the both systems are equivalent. Then command filter technique is applied to solve the so-called "explosion of complexity" problem in traditional backstepping procedure, and radial basis function NNs are directly employed to model the unknown nonlinear functions. The designed controller ensures that all the closed-loop variables are ultimately boundedness, while the output limit is not transgressed and the output tracking error can be reduced arbitrarily small. Furthermore, the use of an asymmetric BLF is also explored to handle the case of asymmetric output constraint as a generalization result. Finally, the control performance of the presented control schemes is illustrated via two examples.

Control Synthesis of Discrete-Time T-S Fuzzy Systems: Reducing the Conservatism Whilst Alleviating the Computational Burden

The augmented multi-indexed matrix approach acts as a powerful tool in reducing the conservatism of control synthesis of discrete-time Takagi-Sugeno fuzzy systems. However, its computational burden is sometimes too heavy as a tradeoff. Nowadays, reducing the conservatism whilst alleviating the computational burden becomes an ideal but very challenging problem. This paper is toward finding an efficient way to achieve one of satisfactory answers. Different from the augmented multi-indexed matrix approach in the literature, we aim to design a more efficient slack variable approach under a general framework of homogenous matrix polynomials. Thanks to the introduction of a new extended representation for homogeneous matrix polynomials, related matrices with the same coefficient are collected together into one sole set and thus those redundant terms of the augmented multi-indexed matrix approach can be removed, i.e., the computational burden can be alleviated in this paper. More importantly, due to the fact that more useful information is involved into control design, the conservatism of the proposed approach as well is less than the counterpart of the augmented multi-indexed matrix approach. Finally, numerical experiments are given to show the effectiveness of the proposed approach.

Adaptive super-twisting sliding mode control for a three-phase single-stage grid-connected differential boost inverter based photovoltaic system

This paper presents an adaptive super-twisting sliding mode control (STC) along with double-loop control for voltage tracking performance of three-phase differential boost inverter and DC-link capacitor voltage regulation in grid-connected PV system. The effectiveness of the proposed control strategies are demonstrated under realistic scenarios such as variations in solar insolation, load power demand, grid voltage, and transition from grid-connected to standalone mode etc. Additional supplementary power quality control functions such as harmonic compensation, and reactive power management are also investigated with the proposed control strategy. The results are compared with conventional proportional-integral controller, and PWM sliding mode controller. The system performance is evaluated in simulation and in real-time.