Adaptive Finite-Time-Based Neural Optimal Control of Time-Delayed Wheeled Mobile Robotics Systems

For nonlinear systems with uncertain state time delays, an adaptive neural optimal tracking control method based on finite time is designed. With the help of the appropriate LKFs, the time-delay problem is handled. A novel nonquadratic Hamilton-Jacobi-Bellman (HJB) function is defined, where finite time is selected as the upper limit of integration. This function contains information on the state time delay, while also maintaining the basic information. To meet specific requirements, the integral reinforcement learning method is employed to solve the ideal HJB function. Then, a tracking controller is designed to ensure finite-time convergence and optimization of the controlled system. This involves the evaluation and execution of gradient descent updates of neural network weights based on a reinforcement learning architecture. The semi-global practical finite-time stability of the controlled system and the finite-time convergence of the tracking error are guaranteed.

Optimized Adaptive Fuzzy Security Control of Nonlinear Systems With Prescribed Tracking Performance

This article studies the optimized fuzzy prescribed performance control problem for nonlinear nonstrict-feedback systems under denial-of-service (DoS) attacks. A fuzzy estimator is delicately designed to model the immeasurable system states in the presence of DoS attacks. To achieve the preset tracking performance, a simper prescribed performance error transformation is constructed considering the characteristics of DoS attacks, which helps obtain a novel Hamilton-Jacobi-Bellman equation to derive the optimized prescribed performance controller. Furthermore, the fuzzy-logic system, combined with the reinforcement learning (RL) technique, is employed to approximate the unknown nonlinearity existing in the prescribed performance controller design process. An optimized adaptive fuzzy security control law is then proposed for the considered nonlinear nonstrict-feedback systems subject to DoS attacks. Through the Lyapunov stability analysis, the tracking error is proved to approach the predefined region by the preset finite time, even in the presence of DoS attacks. Meanwhile, the consumed control resources are minimized due to the RL-based optimized algorithm. Finally, an actual example with comparisons verifies the effectiveness of the proposed control algorithm.

Safe deep reinforcement learning in diesel engine emission control

A deep reinforcement learning application is investigated to control the emissions of a compression ignition diesel engine. The main purpose of this study is to reduce the engine-out nitrogen oxide ( N O x ) emissions and to minimize fuel consumption while tracking a reference engine load. First, a physics-based engine simulation model is developed in GT-Power and calibrated using experimental data. Using this model and a GT-Power/Simulink co-simulation, a deep deterministic policy gradient is developed. To reduce the risk of an unwanted output, a safety filter is added to the deep reinforcement learning. Based on the simulation results, this filter has no effect on the final trained deep reinforcement learning; however, during the training process, it is crucial to enforce constraints on the controller output. The developed safe reinforcement learning is then compared with an iterative learning controller and a deep neural network-based nonlinear model predictive controller. This comparison shows that the safe reinforcement learning is capable of accurately tracking an arbitrary reference input while the iterative learning controller is limited to a repetitive reference. The comparison between the nonlinear model predictive control and reinforcement learning indicates that for this case reinforcement learning is able to learn the optimal control output directly from the experiment without the need for a model. However, to enforce output constraint for safe learning reinforcement learning, a simple model of system is required. In this work, reinforcement learning was able to reduce N O x emissions more than the nonlinear model predictive control; however, it suffered from slightly higher error in load tracking and a higher fuel consumption.

Adaptive Reinforcement Learning Control Based on Neural Approximation for Nonlinear Discrete-Time Systems With Unknown Nonaffine Dead-Zone Input

In this paper, an optimal control algorithm is designed for uncertain nonlinear systems in discrete-time, which are in nonaffine form and with unknown dead-zone. The main contributions of this paper are that an optimal control algorithm is for the first time framed in this paper for nonlinear systems with nonaffine dead-zone, and the adaptive parameter law for dead-zone is calculated by using the gradient rules. The mean value theory is employed to deal with the nonaffine dead-zone input and the implicit function theory based on reinforcement learning is appropriately introduced to find an unknown ideal controller which is approximated by using the action network. Other neural networks are taken as the critic networks to approximate the strategic utility functions. Based on the Lyapunov stability analysis theory, we can prove the stability of systems, i.e., the optimal control laws can guarantee that all the signals in the closed-loop system are bounded and the tracking errors are converged to a small compact set. Finally, two simulation examples demonstrate the effectiveness of the design algorithm.

Neural Approximation-Based Adaptive Control for a Class of Nonlinear Nonstrict Feedback Discrete-Time Systems

In this paper, an adaptive control approach-based neural approximation is developed for a class of uncertain nonlinear discrete-time (DT) systems. The main characteristic of the considered systems is that they can be viewed as a class of multi-input multioutput systems in the nonstrict feedback structure. The similar control problem of this class of systems has been addressed in the past, but it focused on the continuous-time systems. Due to the complicacies of the system structure, it will become more difficult for the controller design and the stability analysis. To stabilize this class of systems, a new recursive procedure is developed, and the effect caused by the noncausal problem in the nonstrict feedback DT structure can be solved using a semirecurrent neural approximation. Based on the Lyapunov difference approach, it is proved that all the signals of the closed-loop system are semiglobal, ultimately uniformly bounded, and a good tracking performance can be guaranteed. The feasibility of the proposed controllers can be validated by setting a simulation example.

Optimal Control-Based Adaptive NN Design for a Class of Nonlinear Discrete-Time Block-Triangular Systems

In this paper, we propose an optimal control scheme-based adaptive neural network design for a class of unknown nonlinear discrete-time systems. The controlled systems are in a block-triangular multi-input-multi-output pure-feedback structure, i.e., there are both state and input couplings and nonaffine functions to be included in every equation of each subsystem. The design objective is to provide a control scheme, which not only guarantees the stability of the systems, but also achieves optimal control performance. The main contribution of this paper is that it is for the first time to achieve the optimal performance for such a class of systems. Owing to the interactions among subsystems, making an optimal control signal is a difficult task. The design ideas are that: 1) the systems are transformed into an output predictor form; 2) for the output predictor, the ideal control signal and the strategic utility function can be approximated by using an action network and a critic network, respectively; and 3) an optimal control signal is constructed with the weight update rules to be designed based on a gradient descent method. The stability of the systems can be proved based on the difference Lyapunov method. Finally, a numerical simulation is given to illustrate the performance of the proposed scheme.

Adaptive Neural Control of MIMO Nonstrict-Feedback Nonlinear Systems With Time Delay

In this paper, an adaptive neural output-feedback tracking controller is designed for a class of multiple-input and multiple-output nonstrict-feedback nonlinear systems with time delay. The system coefficient and uncertain functions of our considered systems are both unknown. By employing neural networks to approximate the unknown function entries, and constructing a new input-driven filter, a backstepping design method of tracking controller is developed for the systems under consideration. The proposed controller can guarantee that all the signals in the closed-loop systems are ultimately bounded, and the time-varying target signal can be tracked within a small error as well. The main contributions of this paper lie in that the systems under consideration are more general, and an effective design procedure of output-feedback controller is developed for the considered systems, which is more applicable in practice. Simulation results demonstrate the efficiency of the proposed algorithm.