An Improved N-Step Value Gradient Learning Adaptive Dynamic Programming Algorithm for Online Learning

In problems with complex dynamics and challenging state spaces, the dual heuristic programming (DHP) algorithm has been shown theoretically and experimentally to perform well. This was recently extended by an approach called value gradient learning (VGL). VGL was inspired by a version of temporal difference (TD) learning that uses eligibility traces. The eligibility traces create an exponential decay of older observations with a decay parameter ( λ ). This approach is known as TD( λ ), and its DHP extension is known as VGL( λ ), where VGL(0) is identical to DHP. VGL has presented convergence and other desirable properties, but it is primarily useful for batch learning. Online learning requires an eligibility-trace-work-space matrix, which is not required for the batch learning version of VGL. Since online learning is desirable for many applications, it is important to remove this computational and memory impediment. This paper introduces a dual-critic version of VGL, called N -step VGL (NSVGL), that does not need the eligibility-trace-work-space matrix, thereby allowing online learning. Furthermore, this combination of critic networks allows an NSVGL algorithm to learn faster. The first critic is similar to DHP, which is adapted based on TD(0) learning, while the second critic is adapted based on a gradient of n -step TD( λ ) learning. Both networks are combined to train an actor network. The combination of feedback signals from both critic networks provides an optimal decision faster than traditional adaptive dynamic programming (ADP) via mixing current information and event history. Convergence proofs are provided. Gradients of one- and n -step value functions are monotonically nondecreasing and converge to the optimum. Two simulation case studies are presented for NSVGL to show their superior performance.

Integral terminal sliding mode formation control of non-holonomic robots using leader follower approach

Multi-robot formation control has become an important area of research due to its advantages and applications. This paper presents multi-robot formation control using a leader–follower approach without considering the leader's velocity information or estimation. The leader–follower formation is formulated by incorporating the model uncertainties and disturbances. A novel formation controller is presented using integral terminal sliding mode (ITSM) control, which drives the formation tracking error convergence to zero in finite-time. The stability of the close-loop control scheme is verified by using Lyapunov theory. Furthermore, obstacle detection and avoidance are incorporated to avoid collision while maintaining the formation. The effectiveness of the proposed controller is verified and validated using sine and lamniscate curve trajectories. Moreover, the performance of the proposed ITSM formation controller is compared with the standard linear sliding mode (LSM) control.

Learning from adaptive neural network output feedback control of a unicycle-type mobile robot

This paper studies learning from adaptive neural network (NN) output feedback control of nonholonomic unicycle-type mobile robots. The major difficulties are caused by the unknown robot system dynamics and the unmeasurable states. To overcome these difficulties, a new adaptive control scheme is proposed including designing a new adaptive NN output feedback controller and two high-gain observers. It is shown that the stability of the closed-loop robot system and the convergence of tracking errors are guaranteed. The unknown robot system dynamics can be approximated by radial basis function NNs. When repeating same or similar control tasks, the learned knowledge can be recalled and reused to achieve guaranteed stability and better control performance, thereby avoiding the tremendous repeated training process of NNs.

Optimal Formation of Multirobot Systems Based on a Recurrent Neural Network

The optimal formation problem of multirobot systems is solved by a recurrent neural network in this paper. The desired formation is described by the shape theory. This theory can generate a set of feasible formations that share the same relative relation among robots. An optimal formation means that finding one formation from the feasible formation set, which has the minimum distance to the initial formation of the multirobot system. Then, the formation problem is transformed into an optimization problem. In addition, the orientation, scale, and admissible range of the formation can also be considered as the constraints in the optimization problem. Furthermore, if all robots are identical, their positions in the system are exchangeable. Then, each robot does not necessarily move to one specific position in the formation. In this case, the optimal formation problem becomes a combinational optimization problem, whose optimal solution is very hard to obtain. Inspired by the penalty method, this combinational optimization problem can be approximately transformed into a convex optimization problem. Due to the involvement of the Euclidean norm in the distance, the objective function of these optimization problems are nonsmooth. To solve these nonsmooth optimization problems efficiently, a recurrent neural network approach is employed, owing to its parallel computation ability. Finally, some simulations and experiments are given to validate the effectiveness and efficiency of the proposed optimal formation approach.

Adaptive dynamic surface control for uncertain nonlinear systems with interval type-2 fuzzy neural networks

This paper presents a new robust adaptive control method for a class of nonlinear systems subject to uncertainties. The proposed approach is based on an adaptive dynamic surface control, where the system uncertainties are approximately modeled by interval type-2 fuzzy neural networks. In this paper, the robust stability of the closed-loop system is guaranteed by the Lyapunov theorem, and all error signals are shown to be uniformly ultimately bounded. In addition to simulations, the proposed method is applied to a real ball-and-beam system for performance evaluations. To highlight the system robustness, different initial settings of ball-and-beam parameters are considered. Simulation and experimental results indicate that the proposed control scheme has superior responses, compared to conventional dynamic surface control.

Neural PID Control of Robot Manipulators With Application to an Upper Limb Exoskeleton

In order to minimize steady-state error with respect to uncertainties in robot control, proportional-integral-derivative (PID) control needs a big integral gain, or a neural compensator is added to the classical proportional-derivative (PD) control with a large derivative gain. Both of them deteriorate transient performances of the robot control. In this paper, we extend the popular neural PD control into neural PID control. This novel control is a natural combination of industrial linear PID control and neural compensation. The main contributions of this paper are semiglobal asymptotic stability of the neural PID control and local asymptotic stability of the neural PID control with a velocity observer which are proved with standard weight training algorithms. These conditions give explicit selection methods for the gains of the linear PID control. An experimental study on an upper limb exoskeleton with this neural PID control is addressed.

Fuzzy sliding-mode formation control for multirobot systems: design and implementation

This paper mainly addresses the decentralized formation problems for multiple robots, where a fuzzy sliding-mode formation controller (FSMFC) is proposed. The directed networks of dynamic agents with external disturbances and system uncertainties are discussed in consensus problems. To perform a formation control and to guarantee system robustness, a novel formation algorithm combining the concepts of graph theory and fuzzy sliding-model control is presented. According to the communication topology, formation stability conditions can be determined so that an FSMFC can be derived. By Lyapunov stability theorem, not only the system stability can be guaranteed, but the desired formation pattern of a multirobot system can be also achieved. Simulation results are provided to demonstrate the effectiveness of the provided control scheme. Finally, an experimental setup for the e-puck multirobot system is built. Compared to first-order formation algorithm and fuzzy neural network formation algorithm, it shows that real-time experimental results empirically support the promising performance of desire.

Adaptive Formation Control of Electrically Driven Nonholonomic Mobile Robots With Limited Information

We present a leader-follower-based adaptive formation control method for electrically driven nonholonomic mobile robots with limited information. First, an adaptive observer is developed under the condition that the velocity measurement is not available. With the proposed adaptive observer, the formation control part is designed to achieve the desired formation and guarantee the collision avoidance. In addition, neural network is employed to compensate the actuator saturation, and the projection algorithm is used to estimate the velocity information of the leader. It is shown, by using the Lyapunov theory, that all errors of the closed-loop system are uniformly ultimately bounded. Simulation results are presented to illustrate the performance of the proposed control system.