Dynamic event-triggered controller design for nonlinear systems: Reinforcement learning strategy

Egoism, utilitarianism and egalitarianism in multi-agent reinforcement learning

In multi-agent partially observable sequential decision problems with general-sum rewards, it is necessary to account for the egoism (individual rewards), utilitarianism (social welfare), and egalitarianism (fairness) criteria simultaneously. However, achieving a balance between these criteria poses a challenge for current multi-agent reinforcement learning methods. Specifically, fully decentralized methods without global information of all agents' rewards, observations and actions fail to learn a balanced policy, while agents in centralized training (with decentralized execution) methods are reluctant to share private information due to concerns of exploitation by others. To address these issues, this paper proposes a Decentralized and Federated (D&F) paradigm, where decentralized agents train egoistic policies utilizing solely local information to attain self-interest, and the federation controller primarily considers utilitarianism and egalitarianism. Meanwhile, the parameters of decentralized and federated policies are optimized with discrepancy constraints mutually, akin to a server and client pattern, which ensures the balance between egoism, utilitarianism, and egalitarianism. Furthermore, theoretical evidence demonstrates that the federated model, as well as the discrepancy between decentralized egoistic policies and federated utilitarian policies, obtains an O(1/T) convergence rate. Extensive experiments show that our D&F approach outperforms multiple baselines, in terms of both utilitarianism and egalitarianism.

Reinforcement learning-based consensus control for MASs with intermittent constraints

In this article, an adaptive optimal consensus control problem is studied for multiagent systems in the strict-feedback structure with intermittent constraints (the constraints appear intermittently). More specifically, by designing a novel switch-like function and an improved coordinate transformation, the constrained states are converted into unconstrained states, and the problem of intermittent constraints is resolved without requiring "feasibility conditions". In addition, using the composite learning algorithm and neural networks to construct the identifier, a simplified identifier-actor-critic-based reinforcement learning strategy is proposed to obtain the approximate optimal controller under the framework of backstepping. Meanwhile, with the aid of the nonlinear dynamic surface control technique, the issue of "explosion of complexity" in backstepping is removed, and the requirements for filter parameters are loosened. Based on Lyapunov stability theory, it is demonstrated that all signals in the closed-loop system are bounded. Finally, two simulation examples are used to verify the effectiveness of the proposed method.

Discrete-time robust event-triggered actuator fault-tolerant control based on adaptive networks and reinforcement learning

This paper focuses on the topic of fault-tolerant control for discrete-time systems with nonlinear uncertainties and actuator faults. It considers both passive and active faults as part of the analysis and design. The proposed adaptive controller, based on a nonlinear electronic circuit, handles offset-biasing, sensitivity variation, and dead-zone effects. An event-triggered mechanism, utilizing a sliding surface, enhances robustness and reduces data transmission. Adaptive networks called MiFRENs are employed, trained using reinforcement learning. Theoretical analysis guarantees boundedness of internal signals and tracking error. Experimental results validate the scheme, demonstrating required conditions, reduced data transmission, and robust performance. Comparative evaluations confirm its superiority.