1
|
Chen L, Hu B, Guan ZH, Zhao L, Shen X. Multiagent Meta-Reinforcement Learning for Adaptive Multipath Routing Optimization. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:5374-5386. [PMID: 33881997 DOI: 10.1109/tnnls.2021.3070584] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
In this article, we investigate the routing problem of packet networks through multiagent reinforcement learning (RL), which is a very challenging topic in distributed and autonomous networked systems. In specific, the routing problem is modeled as a networked multiagent partially observable Markov decision process (MDP). Since the MDP of a network node is not only affected by its neighboring nodes' policies but also the network traffic demand, it becomes a multitask learning problem. Inspired by recent success of RL and metalearning, we propose two novel model-free multiagent RL algorithms, named multiagent proximal policy optimization (MAPPO) and multiagent metaproximal policy optimization (meta-MAPPO), to optimize the network performances under fixed and time-varying traffic demand, respectively. A practicable distributed implementation framework is designed based on the separability of exploration and exploitation in training MAPPO. Compared with the existing routing optimization policies, our simulation results demonstrate the excellent performances of the proposed algorithms.
Collapse
|
2
|
Hu B, Guan ZH, Chen G, Lewis FL. Multistability of Delayed Hybrid Impulsive Neural Networks With Application to Associative Memories. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2019; 30:1537-1551. [PMID: 30296243 DOI: 10.1109/tnnls.2018.2870553] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
The important topic of multistability of continuous-and discrete-time neural network (NN) models has been investigated rather extensively. Concerning the design of associative memories, multistability of delayed hybrid NNs is studied in this paper with an emphasis on the impulse effects. Arising from the spiking phenomenon in biological networks, impulsive NNs provide an efficient model for synaptic interconnections among neurons. Using state-space decomposition, the coexistence of multiple equilibria of hybrid impulsive NNs is analyzed. Multistability criteria are then established regrading delayed hybrid impulsive neurodynamics, for which both the impulse effects on the convergence rate and the basins of attraction of the equilibria are discussed. Illustrative examples are given to verify the theoretical results and demonstrate an application to the design of associative memories. It is shown by an experimental example that delayed hybrid impulsive NNs have the advantages of high storage capacity and high fault tolerance when used for associative memories.
Collapse
|
3
|
Zhou Y, Li C, Wang H. Stability analysis on state-dependent impulsive Hopfield neural networks via fixed-time impulsive comparison system method. Neurocomputing 2018. [DOI: 10.1016/j.neucom.2018.07.047] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
|
4
|
Hu B, Guan ZH, Qian TH, Chen G. Dynamic Analysis of Hybrid Impulsive Delayed Neural Networks With Uncertainties. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2018; 29:4370-4384. [PMID: 29990176 DOI: 10.1109/tnnls.2017.2764003] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Neural networks (NNs) have emerged as a powerful illustrative diagram for the brain. Unveiling the mechanism of neural-dynamic evolution is one of the crucial steps toward understanding how the brain works and evolves. Inspired by the universal existence of impulses in many real systems, this paper formulates a type of hybrid NNs (HNNs) with impulses, time delays, and interval uncertainties, and studies its global dynamic evolution by a robust interval analysis. The HNNs incorporate both continuous-time implementation and impulsive jump in mutual activations, where time delays and interval uncertainties are represented simultaneously. By constructing a Banach contraction mapping, the existence and uniqueness of the equilibrium of the HNN model are proved and analyzed in detail. Based on nonsmooth Lyapunov functions and delayed impulsive differential equations, new criteria are derived for ensuring the global robust exponential stability of the HNNs. Convergence analysis together with illustrative examples show the effectiveness of the theoretical results.
Collapse
|
5
|
Zhang X, Li C, Huang T, Ahmad HG. Effects of variable-time impulses on global exponential stability of Cohen–Grossberg neural networks. INT J BIOMATH 2017. [DOI: 10.1142/s1793524517501170] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
We investigate the global exponential stability of Cohen–Grossberg neural networks (CGNNs) with variable moments of impulses using B-equivalence method. Under certain conditions, we show that each solution of the considered system intersects each surface of discontinuity exactly once, and that the variable-time impulsive systems can be reduced to the fixed-time impulsive ones. The obtained results imply that impulsive CGNN will remain stability property of continuous subsystem even if the impulses are of somewhat destabilizing, and that stabilizing impulses can stabilize the unstable continuous subsystem at its equilibrium points. Moreover, two stability criteria for the considered CGNN by use of proposed comparison system are obtained. Finally, the theoretical results are illustrated by two examples.
Collapse
Affiliation(s)
- Xianxiu Zhang
- College of Electronic and Information Engineering, Southwest University, Chongqing 400715, P. R. China
- Department of Mathematics, Liupanshui Normal University, Guizhou, Liupanshui 553001, P. R. China
| | - Chuandong Li
- College of Electronic and Information Engineering, Southwest University, Chongqing 400715, P. R. China
| | - Tingwen Huang
- Department of Mathematics, Texas A&M University at Qatar, Doha 23874, Qatar
| | - Hafiz Gulfam Ahmad
- Department of Computer Science and IT, Ghazi University, D. G. Khan 32260, Pakistan
| |
Collapse
|
6
|
Zhang X, Li C, Huang T. Hybrid impulsive and switching Hopfield neural networks with state-dependent impulses. Neural Netw 2017. [PMID: 28646762 DOI: 10.1016/j.neunet.2017.04.009] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
We discuss the global stability of switching Hopfield neural networks (HNN) with state-dependent impulses using B-equivalence method. Under certain conditions, we show that the state-dependent impulsive switching systems can be reduced to the fixed-time ones, and that the global stability of corresponding comparison system implies the same stability of the considered system. On this basis, a novel stability criterion for the considered HNN is established. Finally, two numerical examples are given to demonstrate the effectiveness of our results.
Collapse
Affiliation(s)
- Xianxiu Zhang
- Chongqing Key Laboratory of Nonlinear Circuits and Intelligent Information Processing, College of Electronic and Information Engineering, Southwest University, Chongqing 400715, China; Department of Mathematics, Liupanshui Normal University, Guizhou Liupanshui 553001, China.
| | - Chuandong Li
- Chongqing Key Laboratory of Nonlinear Circuits and Intelligent Information Processing, College of Electronic and Information Engineering, Southwest University, Chongqing 400715, China.
| | - Tingwen Huang
- Department of Mathematics, Texas A&M University at Qatar, Doha, Qatar
| |
Collapse
|
7
|
|
8
|
Stability analysis of delayed Hopfield Neural Networks with impulses via inequality techniques. Neurocomputing 2015. [DOI: 10.1016/j.neucom.2014.10.036] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
9
|
Wei L, Chen WH. Global exponential stability of a class of impulsive neural networks with unstable continuous and discrete dynamics. Neurocomputing 2015. [DOI: 10.1016/j.neucom.2014.06.072] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
10
|
|
11
|
Zhang W, Tang Y, Fang JA, Wu X. Stability of delayed neural networks with time-varying impulses. Neural Netw 2012; 36:59-63. [DOI: 10.1016/j.neunet.2012.08.014] [Citation(s) in RCA: 52] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2012] [Revised: 07/28/2012] [Accepted: 08/26/2012] [Indexed: 10/27/2022]
|
12
|
Akhmet M, Yılmaz E. Global exponential stability of neural networks with non-smooth and impact activations. Neural Netw 2012; 34:18-27. [DOI: 10.1016/j.neunet.2012.06.004] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2011] [Revised: 02/13/2012] [Accepted: 06/17/2012] [Indexed: 11/16/2022]
|
13
|
GU HAIBO, JIANG HAIJUN, TENG ZHIDONG. PERIODICITY AND STABILITY IN RECURRENT CELLULAR NEURAL NETWORKS WITH IMPULSIVE EFFECTS. INT J BIOMATH 2012. [DOI: 10.1142/s1793524511001295] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
In this paper, the exponential stability analysis problem is considered for a class of impulsive recurrent cellular neural networks (IRCNNs) with time-varying delays. Without assuming the boundedness on the activation functions, some sufficient conditions are derived for checking the existence and exponential stability of periodic solution for this system by using Mawhin's continuation theorem of coincidence degree theory and constructing suitable Lyapunov functional. It is believed that these results are significant and useful for the design and applications of IRCNNs. Finally, an example with numerical simulation is given to show the effectiveness of the proposed method and results.
Collapse
Affiliation(s)
- HAIBO GU
- College of Mathematics Science, Xinjiang Normal University, 102, Xinyi Road, Urumqi 830054, P. R. China
| | - HAIJUN JIANG
- College of Mathematics and System Sciences, Xinjiang University, Urumqi 830046, P. R. China
| | - ZHIDONG TENG
- College of Mathematics and System Sciences, Xinjiang University, Urumqi 830046, P. R. China
| |
Collapse
|
14
|
LI YONGKUN, ZHANG TIANWEI. GLOBAL EXPONENTIAL STABILITY OF FUZZY INTERVAL DELAYED NEURAL NETWORKS WITH IMPULSES ON TIME SCALES. Int J Neural Syst 2011; 19:449-56. [DOI: 10.1142/s0129065709002142] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
In this paper, we investigate the existence and uniqueness of equilibrium point for fuzzy interval delayed neural networks with impulses on time scales. And we give the criteria of the global exponential stability of the unique equilibrium point for the neural networks under consideration using Lyapunov method. Finally, we present an example to illustrate that our results are effective.
Collapse
Affiliation(s)
- YONGKUN LI
- Department of Mathematics, Yunnan University, Kunming, Yunnan 650091, Pepole's Republic of China
| | - TIANWEI ZHANG
- Department of Mathematics, Yunnan University, Kunming, Yunnan 650091, Pepole's Republic of China
| |
Collapse
|
15
|
Wu Q, Zhou J, Xiang L. Impulses-induced exponential stability in recurrent delayed neural networks. Neurocomputing 2011. [DOI: 10.1016/j.neucom.2011.05.001] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
16
|
Li Y, Zhao L, Zhang T. Global Exponential Stability and Existence of Periodic Solution of Impulsive Cohen–Grossberg Neural Networks with Distributed Delays on Time Scales. Neural Process Lett 2011. [DOI: 10.1007/s11063-010-9166-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
17
|
Chuandong Li, Gang Feng, Tingwen Huang. On Hybrid Impulsive and Switching Neural Networks. ACTA ACUST UNITED AC 2008. [DOI: 10.1109/tsmcb.2008.928233] [Citation(s) in RCA: 81] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
18
|
Exponential stability of impulsive Cohen–Grossberg neural networks with time-varying delays and reaction–diffusion terms. Neurocomputing 2008. [DOI: 10.1016/j.neucom.2008.01.009] [Citation(s) in RCA: 59] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
19
|
Exponential synchronization of a class of neural networks with mixed time-varying delays and impulsive effects. Neurocomputing 2008. [DOI: 10.1016/j.neucom.2008.03.004] [Citation(s) in RCA: 55] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
20
|
Li C, Chen L, Aihara K. Impulsive control of stochastic systems with applications in chaos control, chaos synchronization, and neural networks. CHAOS (WOODBURY, N.Y.) 2008; 18:023132. [PMID: 18601498 DOI: 10.1063/1.2939483] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
Real systems are often subject to both noise perturbations and impulsive effects. In this paper, we study the stability and stabilization of systems with both noise perturbations and impulsive effects. In other words, we generalize the impulsive control theory from the deterministic case to the stochastic case. The method is based on extending the comparison method to the stochastic case. The method presented in this paper is general and easy to apply. Theoretical results on both stability in the pth mean and stability with disturbance attenuation are derived. To show the effectiveness of the basic theory, we apply it to the impulsive control and synchronization of chaotic systems with noise perturbations, and to the stability of impulsive stochastic neural networks. Several numerical examples are also presented to verify the theoretical results.
Collapse
Affiliation(s)
- Chunguang Li
- Centre for Nonlinear and Complex Systems, School of Electronic Engineering, University of Electronic Science and Technology of China, Chengdu 610054, People's Republic of China
| | | | | |
Collapse
|
21
|
|
22
|
Zhang H, Chen L. Asymptotic behavior of discrete solutions to delayed neural networks with impulses. Neurocomputing 2008. [DOI: 10.1016/j.neucom.2006.11.022] [Citation(s) in RCA: 25] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
23
|
Gu H, Jiang H, Teng Z. Existence and globally exponential stability of periodic solution of BAM neural networks with impulses and recent-history distributed delays. Neurocomputing 2008. [DOI: 10.1016/j.neucom.2007.03.007] [Citation(s) in RCA: 25] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
24
|
Sun J. Stationary oscillation for chaotic shunting inhibitory cellular neural networks with impulses. CHAOS (WOODBURY, N.Y.) 2007; 17:043123. [PMID: 18163787 DOI: 10.1063/1.2816944] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
In this paper, we study stationary oscillation for general shunting inhibitory cellular neural networks with impulses which are complex nonlinear neural networks. In a recent paper [Z. J. Gui and W. G. Ge, Chaos 16, 033116 (2006)], the authors claimed that they obtained a criterion of existence, uniqueness, and global exponential stability of periodic solution (i.e., stationary oscillation) for shunting inhibitory cellular neural networks with impulses. We point out in this paper that the main result of their paper is incorrect, and presents a sufficient condition of ensuring existence, uniqueness, and global stability of periodic solution for general shunting inhibitory cellular neural networks with impulses. The result is derived by using a new method which is different from those of previous literature. An illustrative example is given to demonstrate the effectiveness.
Collapse
Affiliation(s)
- Jitao Sun
- Department of Mathematics, Tongji University, Shanghai 200092, China.
| |
Collapse
|
25
|
Xia Y, Cao J, Sun Cheng S. Global exponential stability of delayed cellular neural networks with impulses. Neurocomputing 2007. [DOI: 10.1016/j.neucom.2006.08.005] [Citation(s) in RCA: 51] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
26
|
Gui Z, Yang XS, Ge W. Periodic solution for nonautonomous bidirectional associative memory neural networks with impulses. Neurocomputing 2007. [DOI: 10.1016/j.neucom.2006.08.004] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
27
|
Song Q, Cao J. Impulsive Effects on Stability of Fuzzy Cohen–Grossberg Neural Networks With Time-Varying Delays. ACTA ACUST UNITED AC 2007; 37:733-41. [PMID: 17550127 DOI: 10.1109/tsmcb.2006.887951] [Citation(s) in RCA: 120] [Impact Index Per Article: 7.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
In this correspondence, the impulsive effects on the stability of fuzzy Cohen-Grossberg neural networks (FCGNNs) with time-varying delays are considered. Several sufficient conditions are obtained ensuring global exponential stability of equilibrium point for the neural networks by the idea of vector Lyapunov function, M-matrix theory, and analytic methods. Moreover, the estimation for exponential convergence rate index is proposed. The obtained results not only show that the stability still remains under certain impulsive perturbations for the continuous stable FCGNNs with time-varying delays, but also present an approach to stabilize the unstable FCGNNs with time-varying delays by utilizing impulsive effects. An example with simulations is given to show the effectiveness of the obtained results.
Collapse
|
28
|
Ho DWC, Liang J, Lam J. Global exponential stability of impulsive high-order BAM neural networks with time-varying delays. Neural Netw 2006; 19:1581-90. [PMID: 16580174 DOI: 10.1016/j.neunet.2006.02.006] [Citation(s) in RCA: 100] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
In this paper, global exponential stability and exponential convergence are studied for a class of impulsive high-order bidirectional associative memory (BAM) neural networks with time-varying delays. By employing linear matrix inequalities (LMIs) and differential inequalities with delays and impulses, several sufficient conditions are obtained for ensuring the system to be globally exponentially stable. Three illustrative examples are also given at the end of this paper to show the effectiveness of our results.
Collapse
Affiliation(s)
- Daniel W C Ho
- Department of Mathematics, City University of Hong Kong, 83 Tat Chee Ave., Hong Kong
| | | | | |
Collapse
|
29
|
Global exponential stability of BAM neural networks with recent-history distributed delays and impulses. Neurocomputing 2006. [DOI: 10.1016/j.neucom.2005.09.014] [Citation(s) in RCA: 42] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
30
|
Liu X, Teo KL, Xu B. Exponential Stability of Impulsive High-Order Hopfield-Type Neural Networks With Time-Varying Delays. ACTA ACUST UNITED AC 2005; 16:1329-39. [PMID: 16342478 DOI: 10.1109/tnn.2005.857949] [Citation(s) in RCA: 118] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This paper considers the problems of global exponential stability and exponential convergence rate for impulsive high-order Hopfield-type neural networks with time-varying delays. By using the method of Lyapunov functions, some sufficient conditions for ensuring global exponential stability of these networks are derived, and the estimated exponential convergence rate is also obtained. As an illustration, an numerical example is worked out using the results obtained.
Collapse
Affiliation(s)
- Xinzhi Liu
- Department of Applied Mathematics, University of Waterloo, Waterloo, Ontario, Canada
| | | | | |
Collapse
|
31
|
Zhichun Yang, Daoyi Xu. Stability analysis of delay neural networks with impulsive effects. ACTA ACUST UNITED AC 2005. [DOI: 10.1109/tcsii.2005.849032] [Citation(s) in RCA: 134] [Impact Index Per Article: 7.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|