Simplified Optimized Backstepping Control for a Class of Nonlinear Strict-Feedback Systems With Unknown Dynamic Functions

Barrier Lyapunov Function-Based Safe Reinforcement Learning for Autonomous Vehicles With Optimized Backstepping

Guaranteed safety and performance under various circumstances remain technically critical and practically challenging for the wide deployment of autonomous vehicles. Safety-critical systems in general, require safe performance even during the reinforcement learning (RL) period. To address this issue, a Barrier Lyapunov Function-based safe RL (BLF-SRL) algorithm is proposed here for the formulated nonlinear system in strict-feedback form. This approach appropriately arranges and incorporates the BLF items into the optimized backstepping control method to constrain the state-variables in the designed safety region during learning. Wherein, thus, the optimal virtual/actual control in every backstepping subsystem is decomposed with BLF items and also with an adaptive uncertain item to be learned, which achieves safe exploration during the learning process. Then, the principle of Bellman optimality of continuous-time Hamilton-Jacobi-Bellman equation in every backstepping subsystem is satisfied with independently approximated actor and critic under the framework of actor-critic through the designed iterative updating. Eventually, the overall system control is optimized with the proposed BLF-SRL method. It is furthermore noteworthy that the variance of the attained control performance under uncertainty is also reduced with the proposed method. The effectiveness of the proposed method is verified with two motion control problems for autonomous vehicles through appropriate comparison simulations.

Optimized Adaptive Fuzzy Security Control of Nonlinear Systems With Prescribed Tracking Performance

This article studies the optimized fuzzy prescribed performance control problem for nonlinear nonstrict-feedback systems under denial-of-service (DoS) attacks. A fuzzy estimator is delicately designed to model the immeasurable system states in the presence of DoS attacks. To achieve the preset tracking performance, a simper prescribed performance error transformation is constructed considering the characteristics of DoS attacks, which helps obtain a novel Hamilton-Jacobi-Bellman equation to derive the optimized prescribed performance controller. Furthermore, the fuzzy-logic system, combined with the reinforcement learning (RL) technique, is employed to approximate the unknown nonlinearity existing in the prescribed performance controller design process. An optimized adaptive fuzzy security control law is then proposed for the considered nonlinear nonstrict-feedback systems subject to DoS attacks. Through the Lyapunov stability analysis, the tracking error is proved to approach the predefined region by the preset finite time, even in the presence of DoS attacks. Meanwhile, the consumed control resources are minimized due to the RL-based optimized algorithm. Finally, an actual example with comparisons verifies the effectiveness of the proposed control algorithm.

Predefined-Time Hierarchical Coordinated Neural Control for Hypersonic Reentry Vehicle

This paper investigates the predefined-time hierarchical coordinated adaptive control on the hypersonic reentry vehicle in presence of low actuator efficiency. In order to compensate for the deficiency of rudder deflection in advantage of channel coupling, the hierarchical design is proposed for coordination of the elevator deflection and aileron deflection. Under the control scheme, the equivalent control law and switching control law are constructed with the predefined-time technology. For the dynamics uncertainty approximation, the composite learning using the tracking error and the prediction error is constructed by designing the serial-parallel estimation model. The closed-loop system stability is analyzed via the Lyapunov approach and the tracking errors are guaranteed to be uniformly ultimately bounded in a predefined time. The tracking performance and the learning accuracy of the proposed algorithm are verified via simulation tests.

Integral BLF-Based Adaptive Neural Constrained Regulation for Switched Systems With Unknown Bounds on Control Gain

In this article, an integral barrier Lyapunov-function (IBLF)-based adaptive tracking controller is proposed for a class of switched nonlinear systems under the arbitrary switching rule, in which the unknown terms are approximated by radial basis function neural networks (RBFNNs). The IBLF method is used to solve the problem of state constraint. This method constrains states directly and avoids the verification of feasibility conditions. In addition, a completely unknown control gain is considered, which makes it impossible to directly apply previous existing methods. To offset the effect of the unknown control gain, the lower bound of the control gain is added into the barrier Lyapunov function, and a regulating term is introduced into the controller. The proposed control strategy realizes three control objectives: 1) all the signals in the resulting system are bounded; 2) the system output tracks the reference signal to a arbitrarily small compact set; and 3) all the constraint conditions for system states are not violated. Finally, a simulation example is used to show the effectiveness of the proposed method.

Decentralized Adaptive Neural Inverse Optimal Control of Nonlinear Interconnected Systems

Existing methods on decentralized optimal control of continuous-time nonlinear interconnected systems require a complicated and time-consuming iteration on finding the solution of Hamilton-Jacobi-Bellman (HJB) equations. In order to overcome this limitation, in this article, a decentralized adaptive neural inverse approach is proposed, which ensures the optimized performance but avoids solving HJB equations. Specifically, a new criterion of inverse optimal practical stabilization is proposed, based on which a new direct adaptive neural strategy and a modified tuning functions method are proposed to design a decentralized inverse optimal controller. It is proven that all the closed-loop signals are bounded and the goal of inverse optimality with respect to the cost functional is achieved. Illustrative examples validate the performance of the methods presented.

Event-Triggered Prescribed Settling Time Consensus Compensation Control for a Class of Uncertain Nonlinear Systems With Actuator Failures

For a class of uncertain nonlinear systems with actuator failures, the event-triggered prescribed settling time consensus adaptive compensation control method is proposed. The unknown form of actuator failures may occur in practical applications, resulting in system instability or even control failure. In order to effectively deal with the above problems, a neural network adaptive control method is developed to ensure that the system states rapidly converge in the event of failure and compensate for the failures of actuator. Meanwhile, a nonlinear transformation function is introduced to make sure that the tracking error converges for the predefined interval within a prescribed settling time, which makes that the convergence time can be preset. Furthermore, a finite-time event-triggered compensation control strategy is established by the backstepping technology. Under this strategy, the system not only can rapidly stabilize in finite time but also can effectively save network bandwidth. In addition, the states of the system are globally uniformly bounded. Finally, the theoretical analysis and simulation experiments validate the effectiveness of the proposed method.

Neuroadaptive Optimal Fixed-Time Synchronization and its Circuit Realization for Unidirectionally Coupled FO Self-Sustained Electromechanical Seismograph Systems

This article investigates the neuroadaptive optimal fixed-time synchronization and its circuit realization along with dynamical analysis for unidirectionally coupled fractional-order (FO) self-sustained electromechanical seismograph systems under subharmonic and superharmonic oscillations. The synchronization model of the coupled FO seismograph system is established based on drive and response seismic detectors. The dynamical analysis reveals this coupled system generating transient chaos and homoclinic/heteroclinic oscillations. The test results of the constructed equivalent analog circuit further testify its complex nonlinear dynamics. Then, a neuroadaptive optimal fixed-time synchronization controller integrated with the FO hyperbolic tangent tracking differentiator (HTTD), interval type-2 fuzzy neural network (IT2FNN) with transformation, and prescribed performance function (PPF) together with the constraint condition is developed in the backstepping recursive design. Furthermore, it is proved that all signals of this closed-loop system are bounded, and the tracking errors fall into a trap of the prescribed constraint along with the minimized cost function. Extensive studies confirm the effectiveness of the proposed scheme.

Adaptive Control of Uncertain Nonlinear Systems via Event-Triggered Communication and NN Learning

This article concentrates on adaptive tracking control of strict-feedback uncertain nonlinear systems with an event-based learning scheme. A novel neural network (NN) learning law is proposed to design the adaptive control scheme. The NN weights information driven by the prediction-error-based control process is intermittently transmitted in the event-triggered context to the NN learning law mainly for signal tracking. The online stored sampled data of NN driven by the tracking error are utilized in the event context to update the learning law. With the adaptive control and NN learning law updated via the event-triggered communication, the improvements of NN learning capability, tracking performance, and system computing resource saving are guaranteed. In addition, it is proved that the minimum time interval for triggering errors of the two types of events is bounded and the Zeno behavior is strictly excluded. Finally, simulation results illustrate the effectiveness and good performance of the proposed control method.

Event-Triggered Near-Optimal Control for Unknown Discrete-Time Nonlinear Systems Using Parallel Control

This article uses parallel control to investigate the problem of event-triggered near-optimal control (ETNOC) for unknown discrete-time (DT) nonlinear systems. First, to achieve parallel control, an augmented nonlinear system (ANS) with an augmented performance index (API) is proposed to introduce the control input into the feedback system. The control stability relationship between the ANS and the original system is analyzed, and it is shown that, by choosing a proper API, optimal control of the ANS with the API can be seen as near-optimal control of the original system with the original performance index (OPI). Second, based on parallel control, a novel event-triggered scheme is proposed, and then a novel ETNOC method is developed using the time-triggered optimal value function of the ANS with the API. The control stability is proved, and an upper bound, which is related to the design parameter, is provided for the actual performance index in advance. Then, to implement the developed ETNOC method for unknown DT nonlinear systems, a novel online learning algorithm is developed without reconstructing unknown systems, and neural network (NN) and adaptive dynamic programming (ADP) techniques are employed in the developed algorithm. The convergence of the signals in the closed-loop system (CLS) is shown using the Lyapunov approach, and the assumption of boundedness of input dynamics is not required. Finally, two simulations justify the theoretical conjectures.

A survey of adaptive optimal control theory

This paper makes a survey about the recent development of optimal control based on adaptive dynamic programming (ADP). First of all, based on DP algorithm and reinforcement learning (RL) algorithm, the origin and development of the optimization idea and its application in the control field are introduced. The second part introduces achievements in the optimal control direction, then we classify and summarize the research results of optimization method, constraint problem, structure design in control algorithm and practical engineering process based on optimal control. Finally, the possible future research topics are discussed. Through a comprehensive and complete investigation of its application in many existing fields, this survey fully demonstrates that the optimal control algorithms via ADP with critic-actor neural network (NN) structure, which also have a broad application prospect, and some developed optimal control design algorithms have been applied to practical engineering fields.

Adaptive NN Controller of Nonlinear State-Dependent Constrained Systems With Unknown Control Direction

Various constraints commonly exist in most physical systems; however, traditional constraint control methods consider the constraint boundaries only relying on constant or time variable, which greatly restricts applying constraint control to practical systems. To avoid such conservatism, this study develops a new adaptive neural controller for the nonlinear strict-feedback systems subject to state-dependent constraint boundaries. The nonlinear state-dependent mapping is employed in each step of backstepping procedure, and the prescribed transient performance on tracking error and the constraints on system states are ensured without repeatedly verifying the feasibility conditions on virtual controllers. The radial basis function neural network (NN) with less parameters approach is introduced as an identifier to estimate the unknown system dynamics and reduce computation burden. For removing the effect of unknown control direction, the Nussbaum gain technique is integrated into controller design. Based on the Lyapunov analysis, the developed control strategy can ensure that all the closed-loop signals are bounded, and the constraints on full system states and tracking error are achieved. The simulation examples are used to illustrate the effectiveness of the developed control strategy.

Game-Based Backstepping Design for Strict-Feedback Nonlinear Multi-Agent Systems Based on Reinforcement Learning

In this article, the game-based backstepping control method is proposed for the high-order nonlinear multi-agent system with unknown dynamic and input saturation. Reinforcement learning (RL) is employed to get the saddle point solution of the tracking game between each agent and the reference signal for achieving robust control. Specifically, the approximate optimal solution of the established Hamilton-Jacobi-Isaacs (HJI) equation is obtained by policy iteration for each subsystem, and the single network adaptive critic (SNAC) architecture is used to reduce the computational burden. In addition, based on the separation operation of the error term from the derivative of the value function, we achieve the different proportions of the two agents in the game to realize the regulation of the final equilibrium point. Different from the general use of the neural network for system identification, the unknown nonlinear dynamic term is approximated based on the state difference obtained by the command filter. Furthermore, a sufficient condition is established to guarantee that the whole system and each subsystem included are uniformly ultimately bounded. Finally, simulation results are given to show the effectiveness of the proposed method.

Event-based adaptive fuzzy control design for nonstrict-feedback nonlinear time-delay systems with state constraints

This article considers the issue of event-triggered adaptive fuzzy control for state-constrained nonstrict-feedback nonlinear time-delay systems. The adverse effect of time-delay is effectively overcome by choosing the approximate Lyapunov-Krasovskii functional. The fuzzy logic systems are utilized to address unknown dynamics. The computation complexity is reduced by taking the norm of fuzzy weight vector as estimation. The barrier Lyapunov function is employed to ensure the prescribed constraints. To decrease the update frequency of control signal, event-triggered mechanism is fused into backstepping design process. The semi-globally uniformly ultimately bounded (SGUUB) of the closed-loop system is proved by virtue of Lyapunov stability analysis. Two simulation examples are given to account for the usefulness of the developed method.

Adaptive Neural Network Sliding Mode Control for a Class of SISO Nonlinear Systems

In this article, a sliding mode control (SMC) is proposed on the basis of an adaptive neural network (NN) for a class of Single-Input–Single-Output (SISO) nonlinear systems containing unknown dynamic functions. Since the control objective is to steer the system states to track the given reference signals, the SMC method is considered by employing the adaptive neural network (NN) strategy for dealing with the unknown dynamic problem. In order to compress the impaction coming from chattering phenomenon (which inherently exists in most SMC methods because of the discontinuous switching term), the boundary layer technique is considered. The basic design idea is to introduce a continuous proportional function to replace the discontinuous switching control term inside the boundary layer so that the chattering can be effectively alleviated. Finally, both Lyapunov theoretical analysis and computer numerical simulation are used to verify the effectiveness of the proposed SMC method.

Adaptive neural network asymptotic tracking control for nonstrict feedback stochastic nonlinear systems

The adaptive neural network asymptotic tracking control issue of nonstrict feedback stochastic nonlinear systems is studied in our article by adopting backstepping algorithm. Compared with the existing research, the hypothesis about unknown virtual control coefficients (UVCC) is overcome in the control design. By using the bound estimation scheme and some smooth functions, associating with approximation-based neural network, the asymptotic tracking controller is recursively constructed. With the aid of Lyapunov function and beneficial inequalities, the asymptotic convergence character and stability with stochastic disturbance and unknown UVCC can be ensured. Finally, the theoretical finding is verified via a simulation example.