Data-based robust optimal control of continuous-time affine nonlinear systems with matched uncertainties

H∞ Codesign for Uncertain Nonlinear Control Systems Based on Policy Iteration Method

In this article, the problem of H_∞ codesign for nonlinear control systems with unmatched uncertainties and adjustable parameters is investigated. The main purpose is to solve the adjustable parameters and H_∞ controller simultaneously so that better robust control performance can be achieved. By introducing a bounded function and defining a special cost function, the problem of solving the Hamilton-Jacobi-Isaacs equation is transformed into an optimization problem with nonlinear inequality constraints. Based on the sum of squares technique, a novel policy iteration algorithm is proposed to solve the problem of the H_∞ codesign. Moreover, one modified algorithm for optimizing the robust performance index is given. The convergence and the performance improvement of new iteration policy algorithms are proved. Simulation results are presented to demonstrate the effectiveness of the proposed algorithms.

Model-free nonlinear robust control design via online critic learning

Online critic learning or solving robust control problems of complex systems usually requires knowledge about system dynamics. In order to achieve these goals in data-driven method, a new performance index related to the decreasing rate of the conventional cost is designed. The corresponding optimal control policy can be approximated online using a new actor-critic scheme with three neural networks, without depending on initial stable control and knowledge about system dynamics. The learning process and the learned control policy show excellent robustness. Numerical simulations and an inverted pendulum experiment show that compared with benchmark methods, the proposed method relaxes the dependence on initial admissible control and exhibits better disturbance attenuation performance.

Approximate Optimal Tracking Control of Nondifferentiable Signals for a Class of Continuous-Time Nonlinear Systems

In this article, for a class of continuous-time nonlinear nonaffine systems with unknown dynamics, a robust approximate optimal tracking controller (RAOTC) is proposed in the framework of adaptive dynamic programming (ADP). The distinguishing contribution of this article is that a new Lyapunov function is constructed, by using which the derivative information of tracking errors is not required in computing its time derivative along with the solution of the closed-loop system. Thus, the proposed method can make the system states follow nondifferentiable reference signals, which removes the common assumption that the reference signals have to be continuous for tracking control of continuous-time nonlinear systems in the literature. The theoretical analysis, simulation, and application results well illustrate the effectiveness and superiority of the proposed method.

Observer Design and H∞ Performance for Discrete-Time Uncertain Fuzzy-Logic Systems

The observer design and H_∞ algorithm are proposed for the discrete-time fuzzy-logic systems in this article. The considered fuzzy-logic systems are subject to parameter uncertainty and unmeasurable state variables. To appropriately deal with parameter uncertainty and unmeasurable state variables, we first disintegrate the space of premise variables, then the total partitioned regions will be divided into two kinds: 1) crisp regions and 2) fuzzy regions. With the aid of partitioned regions, piecewise fuzzy H_∞ observers are presented. Availability of the piecewise fuzzy H_∞ observers gives us an accurate picture of the overall evolution of the augmented systems leading therefore to an improved situational awareness for the noncoordination of premise variables and external interference and, hence, the augmented systems achieve the performance conditions: asymptotic convergence and H_∞ performance. The simulation results of the proposed approach show the accuracy of the resulting state estimates.

Adaptive Tracking Control of Surface Vessel Using Optimized Backstepping Technique

In this paper, a tracking control approach for surface vessel is developed based on the new control technique named optimized backstepping (OB), which considers optimization as a backstepping design principle. Since surface vessel systems are modeled by second-order dynamic in strict feedback form, backstepping is an ideal technique for finishing the tracking task. In the backstepping control of surface vessel, the virtual and actual controls are designed to be the optimized solutions of corresponding subsystems, therefore the overall control is optimized. In general, optimization control is designed based on the solution of Hamilton-Jacobi-Bellman equation. However, solving the equation is very difficult or even impossible due to the inherent nonlinearity and complexity. In order to overcome the difficulty, the reinforcement learning (RL) strategy of actor-critic architecture is usually considered, of which the critic and actor are utilized for evaluating the control performance and executing the control behavior, respectively. By employing the actor-critic RL algorithm for both virtual and actual controls of the vessel, it is proven that the desired optimizing and tracking performances can be arrived. Simulation results further demonstrate effectiveness of the proposed surface vessel control.

Optimized Backstepping for Tracking Control of Strict-Feedback Systems

In this paper, a control technique named optimized backstepping is first proposed by implementing tracking control for a class of strict-feedback systems, which considers optimization as a design philosophy of the high-order system control. The basic idea is that designing the actual and virtual controls of backstepping is the optimized solutions of the corresponding subsystems so that overall control of the high-order system is optimized. In general, optimization control is designed based on the solution of Hamilton-Jacobi-Bellman equation, but solving the equation is very difficult due to the inherent nonlinearity and intractability. In order to overcome the difficulty, the neural network (NN)-based reinforcement learning strategy of actor-critic architecture is used. In every backstepping step, the actor and critic NNs are constructed for executing control behavior and evaluating control performance, respectively. According to the Lyapunov stability theorem, it is proven that the desired control performance can be obtained. Finally, a simulation example is carried out to further demonstrate the effectiveness of the proposed control approach.

Improving the Critic Learning for Event-Based Nonlinear $H_{\infty }$ Control Design

In this paper, we aim at improving the critic learning criterion to cope with the event-based nonlinear H_∞ state feedback control design. First of all, the H_∞ control problem is regarded as a two-player zero-sum game and the adaptive critic mechanism is used to achieve the minimax optimization under event-based environment. Then, based on an improved updating rule, the event-based optimal control law and the time-based worst-case disturbance law are obtained approximately by training a single critic neural network. The initial stabilizing control is no longer required during the implementation process of the new algorithm. Next, the closed-loop system is formulated as an impulsive model and its stability issue is handled by incorporating the improved learning criterion. The infamous Zeno behavior of the present event-based design is also avoided through theoretical analysis on the lower bound of the minimal intersample time. Finally, the applications to an aircraft dynamics and a robot arm plant are carried out to verify the efficient performance of the present novel design method.

Adaptive Critic Nonlinear Robust Control: A Survey

Adaptive dynamic programming (ADP) and reinforcement learning are quite relevant to each other when performing intelligent optimization. They are both regarded as promising methods involving important components of evaluation and improvement, at the background of information technology, such as artificial intelligence, big data, and deep learning. Although great progresses have been achieved and surveyed when addressing nonlinear optimal control problems, the research on robustness of ADP-based control strategies under uncertain environment has not been fully summarized. Hence, this survey reviews the recent main results of adaptive-critic-based robust control design of continuous-time nonlinear systems. The ADP-based nonlinear optimal regulation is reviewed, followed by robust stabilization of nonlinear systems with matched uncertainties, guaranteed cost control design of unmatched plants, and decentralized stabilization of interconnected systems. Additionally, further comprehensive discussions are presented, including event-based robust control design, improvement of the critic learning rule, nonlinear H_∞ control design, and several notes on future perspectives. By applying the ADP-based optimal and robust control methods to a practical power system and an overhead crane plant, two typical examples are provided to verify the effectiveness of theoretical results. Overall, this survey is beneficial to promote the development of adaptive critic control methods with robustness guarantee and the construction of higher level intelligent systems.