Data-driven model-free slip control of anti-lock braking systems using reinforcement Q-learning

PSO Optimized Active Disturbance Rejection Control for Aircraft Anti-Skid Braking System

A high-quality and secure touchdown run for an aircraft is essential for economic, operational, and strategic reasons. The shortest viable touchdown run without any skidding requires variable braking pressure to manage the friction between the road surface and braking tire at all times. Therefore, the manipulation and regulation of the anti-skid braking system (ABS) should be able to handle steady nonlinearity and undetectable disturbances and to regulate the wheel slip ratio to make sure that the braking system operates securely. This work proposes an active disturbance rejection control technique for the anti-skid braking system. The control law ensures action that is bounded and manageable, and the manipulating algorithm can ensure that the closed-loop machine works around the height factor of the secure area of the friction curve, thereby improving overall braking performance and safety. The stability of the proposed algorithm is proven primarily by means of Lyapunov-based strategies, and its effectiveness is assessed by means of simulations on a semi-physical aircraft brake simulation platform.

Robust Self-Learning PID Control of an Aircraft Anti-Skid Braking System

In order to deal with strong nonlinearity and external interference in the braking process, this paper proposes a robust self-learning PID algorithm based on particle swarm optimization, which does not depend on a precise mathematical model of the controlled object. The self-learning function is used to adapt to the diversity of the runway road surface friction, the particle swarm algorithm is used to optimize the rate of self-learning, and robust control is used to deal with the modeling uncertainty and external disturbance of the system. The convergence of the control strategy is proved by theoretical analysis and simulation experiments. The superiority and accuracy of the method are verified by NASA ground test results. The simulation results shows that the adverse effect of the external disturbance is suppressed, and the ideal trajectory is tracked.

Knowledge Implementation and Transfer With an Adaptive Learning Network for Real-Time Power Management of the Plug-in Hybrid Vehicle

Essential decision-making tasks such as power management in future vehicles will benefit from the development of artificial intelligence technology for safe and energy-efficient operations. To develop the technique of using neural network and deep learning in energy management of the plug-in hybrid vehicle and evaluate its advantage, this article proposes a new adaptive learning network that incorporates a deep deterministic policy gradient (DDPG) network with an adaptive neuro-fuzzy inference system (ANFIS) network. First, the ANFIS network is built using a new global K-fold fuzzy learning (GKFL) method for real-time implementation of the offline dynamic programming result. Then, the DDPG network is developed to regulate the input of the ANFIS network with the real-world reinforcement signal. The ANFIS and DDPG networks are integrated to maximize the control utility (CU), which is a function of the vehicle's energy efficiency and the battery state-of-charge. Experimental studies are conducted to testify the performance and robustness of the DDPG-ANFIS network. It has shown that the studied vehicle with the DDPG-ANFIS network achieves 8% higher CU than using the MATLAB ANFIS toolbox on the studied vehicle. In five simulated real-world driving conditions, the DDPG-ANFIS network increased the maximum mean CU value by 138% over the ANFIS-only network and 5% over the DDPG-only network.

Prescribed Performance Active Braking Control with Reference Adaptation for High-Speed Trains

Active braking control systems are vital for the safety of high-speed trains by leading the train operation at its maximum adhesion state. The train adhesion is a nonlinear function of the slip ratio and varies with the uncertain wheel-rail contact conditions. A nonlinear active braking control with rapid and accurate tracking performance is highly required for train braking systems. This paper proposes a novel prescribed performance active braking control with reference adaptation to obtain the maximum adhesion force. The developed feedback linearization controller employs a prescribed performance function that specifies the convergence rate, steady-state error, and maximum overshoot to ensure the transient and steady-state control performance. Furthermore, in the designed control approach, a continuous-time unscented Kalman filter is introduced to estimate the uncertainty of wheel-rail adhesion. The estimation is utilized to represent uncertainty and compensate for the prescribed performance control law. Finally, based on the estimated wheel-rail adhesion, an on-line optimal slip ratio generation algorithm is proposed for the adaptation of the reference wheel slip. The stability of the system is provided, and experiment results validate the effectiveness of the proposed method.

Adaptive slip ratio estimation for active braking control of high-speed trains

Active braking control systems in high-speed trains are vital to ensure safety and are intended to reduce brake distances and prevent the wheels from locking. The slip ratio, which represents the relative difference between the wheel speed and vehicle velocity, is crucial to the design and successful implementation of active braking control systems. Slip ratio estimation and active braking control are challenging owing to the uncertainties of wheel-rail adhesion and system nonlinearities. Therefore, this paper proposes a novel adaptive slip ratio estimation approach for the active braking control based on an improved extended state observer. The extended state observer is developed through the augmentation of the system state-space to estimate the unmeasured train states as well as the model uncertainty. The accurate slip ratio is estimated using the observed extended states. Furthermore, the adaptability of the observer is improved by introducing the beetle antennae search algorithm to determine the optimal observer parameters. Finally, a feedback linearization braking control law is established to stabilize the closed-loop system due to its potential in coping with nonlinearities, which benefits the proven theoretical bounded stability. Experimental results validate the effectiveness of the proposed method.

Virtual State Feedback Reference Tuning and Value Iteration Reinforcement Learning for Unknown Observable Systems Control

In this paper, a novel Virtual State-feedback Reference Feedback Tuning (VSFRT) and Approximate Iterative Value Iteration Reinforcement Learning (AI-VIRL) are applied for learning linear reference model output (LRMO) tracking control of observable systems with unknown dynamics. For the observable system, a new state representation in terms of input/output (IO) data is derived. Consequently, the Virtual State Feedback Tuning (VRFT)-based solution is redefined to accommodate virtual state feedback control, leading to an original stability-certified Virtual State-Feedback Reference Tuning (VSFRT) concept. Both VSFRT and AI-VIRL use neural networks controllers. We find that AI-VIRL is significantly more computationally demanding and more sensitive to the exploration settings, while leading to inferior LRMO tracking performance when compared to VSFRT. It is not helped either by transfer learning the VSFRT control as initialization for AI-VIRL. State dimensionality reduction using machine learning techniques such as principal component analysis and autoencoders does not improve on the best learned tracking performance however it trades off the learning complexity. Surprisingly, unlike AI-VIRL, the VSFRT control is one-shot (non-iterative) and learns stabilizing controllers even in poorly, open-loop explored environments, proving to be superior in learning LRMO tracking control. Validation on two nonlinear coupled multivariable complex systems serves as a comprehensive case study.

Sliding Mode Control Algorithms for Anti-Lock Braking Systems with Performance Comparisons

This paper presents the performance of two sliding mode control algorithms, based on the Lyapunov-based sliding mode controller (LSMC) and reaching-law-based sliding mode controller (RSMC), with their novel variants designed and applied to the anti-lock braking system (ABS), which is known to be a strongly nonlinear system. The goal is to prove their superior performance over existing control approaches, in the sense that the LSMC and RSMC do not bring additional computational complexity, as they rely on a reduced number of tuning parameters. The performance of LSMC and RSMC solves the uncertainty in the process model which comes from unmodeled dynamics and a simplification of the actuator dynamics, leading to a reduced second order process. The contribution adds complete design details and stability analysis is provided. Additionally, performance comparisons with several adaptive, neural networks-based and model-free sliding mode control algorithms reveal the robustness of the proposed LSMC and RSMC controllers, in spite of the reduced number of tuning parameters. The robustness and reduced computational burden of the controllers validated on the real-world complex ABS make it an attractive solution for practical industrial implementations.

Learning Output Reference Model Tracking for Higher-Order Nonlinear Systems with Unknown Dynamics

This work suggests a solution for the output reference model (ORM) tracking control problem, based on approximate dynamic programming. General nonlinear systems are included in a control system (CS) and subjected to state feedback. By linear ORM selection, indirect CS feedback linearization is obtained, leading to favorable linear behavior of the CS. The Value Iteration (VI) algorithm ensures model-free nonlinear state feedback controller learning, without relying on the process dynamics. From linear to nonlinear parameterizations, a reliable approximate VI implementation in continuous state-action spaces depends on several key parameters such as problem dimension, exploration of the state-action space, the state-transitions dataset size, and a suitable selection of the function approximators. Herein, we find that, given a transition sample dataset and a general linear parameterization of the Q-function, the ORM tracking performance obtained with an approximate VI scheme can reach the performance level of a more general implementation using neural networks (NNs). Although the NN-based implementation takes more time to learn due to its higher complexity (more parameters), it is less sensitive to exploration settings, number of transition samples, and to the selected hyper-parameters, hence it is recommending as the de facto practical implementation. Contributions of this work include the following: VI convergence is guaranteed under general function approximators; a case study for a low-order linear system in order to generalize the more complex ORM tracking validation on a real-world nonlinear multivariable aerodynamic process; comparisons with an offline deep deterministic policy gradient solution; implementation details and further discussions on the obtained results.

Data-Driven Model-Free Tracking Reinforcement Learning Control with VRFT-based Adaptive Actor-Critic

This paper proposes a neural network (NN)-based control scheme in an Adaptive Actor-Critic (AAC) learning framework designed for output reference model tracking, as a representative deep-learning application. The control learning scheme is model-free with respect to the process model. AAC designs usually require an initial controller to start the learning process; however, systematic guidelines for choosing the initial controller are not offered in the literature, especially in a model-free manner. Virtual Reference Feedback Tuning (VRFT) is proposed for obtaining an initially stabilizing NN nonlinear state-feedback controller, designed from input-state-output data collected from the process in open-loop setting. The solution offers systematic design guidelines for initial controller design. The resulting suboptimal state-feedback controller is next improved under the AAC learning framework by online adaptation of a critic NN and a controller NN. The mixed VRFT-AAC approach is validated on a multi-input multi-output nonlinear constrained coupled vertical two-tank system. Discussions on the control system behavior are offered together with comparisons with similar approaches.

Active Control of Regenerative Brake for Electric Vehicles

Looking at new trends in global policies, electric vehicles (EVs) are expected to increasingly replace gasoline vehicles in the near future. For current electric vehicles, the motor current driving system and the braking control system are two independent issues with separate design. If a self-induced back-EMF voltage from the motor is a short circuit, then short-circuiting the motor will result in braking. The higher the speed of the motor, the stronger the braking effect. However, the effect is deficient quickly once the motor speed drops quickly. Traditional kinetic brake (i.e., in the short circuit is replaced by a resistor) and dynamic brake (the short circuit brake is replaced by a capacitor) rely on the back EMF alone to generate braking toque. The braking torque generated is usually not enough to effectively stop a rotating motor in a short period of time. In this research task, an integrated driving and braking control system is considered for EVs with an active regenerative braking control system where back electromagnetic field (EMF), controlled by the pulse-width modulation (PWM) technique, is used to charge a pump capacitor. The capacitor is used as an extra energy source cascaded with the battery as a charge pump. This is used to boost braking torque to stop the rotating motor in an efficient way while braking. Experiments are conducted to verify the proposed design. Compared to the traditional kinetic brake and dynamic brake, the proposed active regenerative control system shows better braking performance in terms of stopping time and stopping distance.

A Q-Learning-Based Approach for Deploying Dynamic Service Function Chains

As the size and service requirements of today’s networks gradually increase, large numbers of proprietary devices are deployed, which leads to network complexity, information security crises and makes network service and service provider management increasingly difficult. Network function virtualization (NFV) technology is one solution to this problem. NFV separates network functions from hardware and deploys them as software on a common server. NFV can be used to improve service flexibility and isolate the services provided for each user, thus guaranteeing the security of user data. Therefore, the use of NFV technology includes many problems worth studying. For example, when there is a free choice of network path, one problem is how to choose a service function chain (SFC) that both meets the requirements and offers the service provider maximum profit. Most existing solutions are heuristic algorithms with high time efficiency, or integer linear programming (ILP) algorithms with high accuracy. It’s necessary to design an algorithm that symmetrically considers both time efficiency and accuracy. In this paper, we propose the Q-learning Framework Hybrid Module algorithm (QLFHM), which includes reinforcement learning to solve this SFC deployment problem in dynamic networks. The reinforcement learning module in QLFHM is responsible for the output of alternative paths, while the load balancing module in QLFHM is responsible for picking the optimal solution from them. The results of a comparison simulation experiment on a dynamic network topology show that the proposed algorithm can output the approximate optimal solution in a relatively short time while also considering the network load balance. Thus, it achieves the goal of maximizing the benefit to the service provider.

System Identification Algorithm for Computing the Modal Parameters of Linear Mechanical Systems

The goal of this investigation is to construct a computational procedure for identifying the modal parameters of linear mechanical systems. The methodology employed in the paper is based on the Eigensystem Realization Algorithm implemented in conjunction with the Observer/Kalman Filter Identification method (ERA/OKID). This method represents an effective and efficient system identification numerical procedure based on the time domain. The algorithm developed in this work is tested by means of numerical experiments on a full-car vehicle model. To this end, the modal parameters necessary for the design of active and semi-active suspension systems are obtained for the vehicle system considered as an illustrative example. In order to analyze the performance of the methodology developed in this investigation, the system identification numerical procedure was tested considering two case studies, namely a full state measurement and an incomplete state measurement. As expected, the numerical results found for the identified dynamical model showed a good agreement with the modal parameters of the mechanical system model. Furthermore, numerical results demonstrated that the proposed method has good performance considering a scenario in which the signal-to-noise ratio of the input and output measurements is relatively high. The method developed in this paper can be effectively used for solving important engineering problems such as the design of control systems for road vehicles.

Data-driven model reference control of MIMO vertical tank systems with model-free VRFT and Q-Learning

This paper proposes a combined Virtual Reference Feedback Tuning-Q-learning model-free control approach, which tunes nonlinear static state feedback controllers to achieve output model reference tracking in an optimal control framework. The novel iterative Batch Fitted Q-learning strategy uses two neural networks to represent the value function (critic) and the controller (actor), and it is referred to as a mixed Virtual Reference Feedback Tuning-Batch Fitted Q-learning approach. Learning convergence of the Q-learning schemes generally depends, among other settings, on the efficient exploration of the state-action space. Handcrafting test signals for efficient exploration is difficult even for input-output stable unknown processes. Virtual Reference Feedback Tuning can ensure an initial stabilizing controller to be learned from few input-output data and it can be next used to collect substantially more input-state data in a controlled mode, in a constrained environment, by compensating the process dynamics. This data is used to learn significantly superior nonlinear state feedback neural networks controllers for model reference tracking, using the proposed Batch Fitted Q-learning iterative tuning strategy, motivating the original combination of the two techniques. The mixed Virtual Reference Feedback Tuning-Batch Fitted Q-learning approach is experimentally validated for water level control of a multi input-multi output nonlinear constrained coupled two-tank system. Discussions on the observed control behavior are offered.