Decentralized Event-Driven Constrained Control Using Adaptive Critic Designs

Adaptive sampling artificial-actual control for non-zero-sum games of constrained systems

Considering physical constraints encountered by actuators, this paper addresses the non-zero-sum game of continuous nonlinear systems with symmetric and asymmetric input constraints through aperiodic sampling artificial-actual control. Initially, the artificial system built by the improved Elman dynamic neural networks (EDNNs) has artificial-actual interaction with the physical system, which provides a new perspective for predicting the system state. By constantly learning and adjusting parameters, EDNNs can gradually approximate the dynamic behavior of the real system to achieve more effective control. Aiming at accommodating diverse input constraints, the non-quadratic value function constructed from a smoothly bounded function is devised. Then, the polynomial parameterized adaptive dynamic programming (ADP) is employed to approximate the solution of the coupled Hamilton-Jacobi equation (HJE), deriving optimal control laws for two players. To improve the efficiency of data communication, three adaptive sampling mechanisms including event-triggered mechanism (ETM) with relative threshold, dynamic ETM (DETM) and self-triggered mechanism (STM) are introduced in turn during the iterative learning process of control sequences. DETM further extends sampling intervals by incorporating internal dynamic variables, while STM determines the next trigger time through soft calculation without hardware monitoring. All three trigger modes can ensure the system stability while avoiding the Zeno phenomenon, and relevant proofs are given. Finally, the simulation validates the effectiveness of the designed algorithm and highlights the unique characteristics of each trigger mode.

Approximate optimal and safe coordination of nonlinear second-order multirobot systems with model uncertainties

This paper investigates the approximate optimal coordination for nonlinear uncertain second-order multi-robot systems with guaranteed safety (collision avoidance) Through constructing novel local error signals, the collision-free control objective is formulated into an coordination optimization problem for nominal multi-robot systems. Based on approximate dynamic programming technique, the optimal value functions and control policies are learned by simplified critic-only neural networks (NNs). Then, the approximated optimal controllers are redesigned using adaptive law to handle the effects of robots' uncertain dynamics. It is shown that the NN weights estimation errors are uniformly ultimately bounded under proper conditions, and safe coordination of multiple robots can be achieved regardless of model uncertainties. Numerical simulations finally illustrate the effectiveness of the proposed controller.

Trigger-Based K-Band Microwave Ranging System Thermal Control with Model-Free Learning Process

Micron-level accuracy K-band microwave ranging in space relies on the stability of the payload thermal control on-board; however, large quantities of thermal sensors and heating devices around the deployed instruments consume the precious inner communication resources of the central computer. Another problem arises, which is that the payload thermal protection environment can deteriorate gradually through years operating. In this paper, a new trigger-based thermal system controller design is proposed, with consideration of spaceborne communication burden reduction and actuator saturation, which guarantees stable temperature fluctuations of microwave payloads in space missions. The controller combines a nominal constant sampling PID inner loop and a trigger-based outer loop structure under constraints of heating device saturation. Moreover, an iterative model-free reinforcement learning process is adopted that can approximate the estimation of thermal dynamic modeling uncertainty online. Via extensive experiment in a laboratory environment, the performance of the proposed trigger thermal control is verified, with smaller temperature fluctuations compared to the nominal control, and obvious efficiency in system communications. The online learning algorithm is also tested with deliberate thermal conditions that deviate from the original system—the results can quickly converge to normal when the thermal disturbance is removed. Finally, the ranging accuracy is tested for the whole system, and a 25% (RMS) performance improvement can be realized by using a trigger-based control strategy—about 2.2 µm, compared to the nominal control method.

Dynamic Event-Sampled Control of Interconnected Nonlinear Systems Using Reinforcement Learning

We develop a decentralized dynamic event-based control strategy for nonlinear systems subject to matched interconnections. To begin with, we introduce a dynamic event-based sampling mechanism, which relies on the system's states and the variables generated by time-based differential equations. Then, we prove that the decentralized event-based controller for the whole system is composed of all the optimal event-based control policies of nominal subsystems. To derive these optimal event-based control policies, we design a critic-only architecture to solve the related event-based Hamilton-Jacobi-Bellman equations in the reinforcement learning framework. The implementation of such an architecture uses only critic neural networks (NNs) with their weight vectors being updated through the gradient descent method together with concurrent learning. After that, we demonstrate that the asymptotic stability of closed-loop nominal subsystems and the uniformly ultimate boundedness stability of critic NNs' weight estimation errors are guaranteed by using Lyapunov's approach. Finally, we provide simulations of a matched nonlinear-interconnected plant to validate the present theoretical claims.