1
|
Yan J, Jin L, Luo X, Li S. Modified RNN for Solving Comprehensive Sylvester Equation With TDOA Application. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:12553-12563. [PMID: 37037242 DOI: 10.1109/tnnls.2023.3263565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
The augmented Sylvester equation, as a comprehensive equation, is of great significance and its special cases (e.g., Lyapunov equation, Sylvester equation, Stein equation) are frequently encountered in various fields. It is worth pointing out that the current research on simultaneously eliminating the lagging error and handling noises in the nonstationary complex-valued field is rather rare. Therefore, this article focuses on solving a nonstationary complex-valued augmented Sylvester equation (NCASE) in real time and proposes two modified recurrent neural network (RNN) models. The first proposed modified RNN model possesses gradient search and velocity compensation, termed as RNN-GV model. The superiority of the proposed RNN-GV model to traditional algorithms including the complex-valued gradient-based RNN (GRNN) model lies in completely eliminating the lagging error when employed in the nonstationary problem. The second model named complex-valued integration enhanced RNN-GV with the nonlinear acceleration (IERNN-GVN) model is proposed to adapt to a noisy environment and accelerate the convergence process. Besides, the convergence and robustness of these two proposed models are proved via theoretical analysis. Simulative results on an illustrative example and an application to the moving source localization coincide with the theoretical analysis and illustrate the excellent performance of the proposed models.
Collapse
|
2
|
Zhang Z, Song Y, Zheng L, Luo Y. A Jump-Gain Integral Recurrent Neural Network for Solving Noise-Disturbed Time-Variant Nonlinear Inequality Problems. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:5793-5806. [PMID: 37022813 DOI: 10.1109/tnnls.2023.3241207] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Nonlinear inequalities are widely used in science and engineering areas, attracting the attention of many researchers. In this article, a novel jump-gain integral recurrent (JGIR) neural network is proposed to solve noise-disturbed time-variant nonlinear inequality problems. To do so, an integral error function is first designed. Then, a neural dynamic method is adopted and the corresponding dynamic differential equation is obtained. Third, a jump gain is exploited and applied to the dynamic differential equation. Fourth, the derivatives of errors are substituted into the jump-gain dynamic differential equation, and the corresponding JGIR neural network is set up. Global convergence and robustness theorems are proposed and proved theoretically. Computer simulations verify that the proposed JGIR neural network can solve noise-disturbed time-variant nonlinear inequality problems effectively. Compared with some advanced methods, such as modified zeroing neural network (ZNN), noise-tolerant ZNN, and varying-parameter convergent-differential neural network, the proposed JGIR method has smaller computational errors, faster convergence speed, and no overshoot when disturbance exists. In addition, physical experiments on manipulator control have verified the effectiveness and superiority of the proposed JGIR neural network.
Collapse
|
3
|
Liu M, Chen X, Shang M, Li H. A Pseudoinversion-Free Method for Weight Updating in Broad Learning System. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:2378-2389. [PMID: 35839197 DOI: 10.1109/tnnls.2022.3190043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Neural networks have evolved into one of the most critical tools in the field of artificial intelligence. As a kind of shallow feedforward neural network, the broad learning system (BLS) uses a training process based on random and pseudoinverse methods, and it does not need to go through a complete training cycle to obtain new parameters when adding nodes. Instead, it performs rapid update iterations on the basis of existing parameters through a series of dynamic update algorithms, which enables BLS to combine high efficiency and accuracy flexibly. The training strategy of BLS is completely different from the existing mainstream neural network training strategy based on the gradient descent algorithm, and the superiority of the former has been proven in many experiments. This article applies an ingenious method of pseudoinversion to the weight updating process in BLS and employs it as an alternative strategy for the dynamic update algorithms in the original BLS. Theoretical analyses and numerical experiments demonstrate the efficiency and effectiveness of BLS aided with this method. The research presented in this article can be regarded as an extended study of the BLS theory, providing an innovative idea and direction for future research on BLS.
Collapse
|
4
|
Chen X, Luo X, Jin L, Li S, Liu M. Growing Echo State Network With an Inverse-Free Weight Update Strategy. IEEE TRANSACTIONS ON CYBERNETICS 2023; 53:753-764. [PMID: 35316203 DOI: 10.1109/tcyb.2022.3155901] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
An echo state network (ESN) draws widespread attention and is applied in many scenarios. As the most typical approach for solving the ESN, the matrix inverse operation of high computational complexity is involved. However, in the modern big data era, addressing the heavy computational burden problem is necessary. In order to reduce the computational load, an inverse-free ESN (IFESN) is proposed for the first time in this article. Besides, an incremental IFESN is constructed to attain the network topology with theoretical proof on the training error's monotone decline property. Simulations and experiments are conducted on several numerical and real-world time-series benchmarks, and corresponding results indicate that the proposed model is superior to some existing models and possesses excellent practical application potential. The source code is publicly available at https://github.com/LongJin-lab/the-supplementary-file-for-CYB-E-2021-04-0944.
Collapse
|
5
|
Xiao X, Jiang C, Mei Q, Zhang Y. Noise‐tolerate and adaptive coefficient zeroing neural network for solving dynamic matrix square root. CAAI TRANSACTIONS ON INTELLIGENCE TECHNOLOGY 2023. [DOI: 10.1049/cit2.12183] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/28/2023] Open
Affiliation(s)
- Xiuchun Xiao
- School of Electronics and Information Engineering Guangdong Ocean University Zhanjiang China
| | - Chengze Jiang
- School of Cyber Science and Engineering Southeast University Nanjing China
| | - Qixiang Mei
- School of Electronics and Information Engineering Guangdong Ocean University Zhanjiang China
| | - Yudong Zhang
- School of Computing and Mathematical Sciences University of Leicester Leicester UK
| |
Collapse
|
6
|
Wang G, Liu Y, Sun Y, Yu J, Sun Z. Generalized zeroing neural dynamics model for online solving time-varying cube roots problem with various external disturbances in different domains. Inf Sci (N Y) 2022. [DOI: 10.1016/j.ins.2022.12.058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
7
|
Different discrete-time noise-suppression Z-type models for online solving time-varying and time-invariant cube roots in real and complex domains: Application to fractals. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.05.089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
8
|
Wu D, Lisser A. A dynamical neural network approach for solving stochastic two-player zero-sum games. Neural Netw 2022; 152:140-149. [DOI: 10.1016/j.neunet.2022.04.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Revised: 02/02/2022] [Accepted: 04/06/2022] [Indexed: 10/18/2022]
|
9
|
Qi Y, Jin L, Luo X, Zhou M. Recurrent Neural Dynamics Models for Perturbed Nonstationary Quadratic Programs: A Control-Theoretical Perspective. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:1216-1227. [PMID: 33449881 DOI: 10.1109/tnnls.2020.3041364] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Recent decades have witnessed a trend that control-theoretical techniques are widely leveraged in various areas, e.g., design and analysis of computational models. Computational methods can be modeled as a controller and searching the equilibrium point of a dynamical system is identical to solving an algebraic equation. Thus, absorbing mature technologies in control theory and integrating it with neural dynamics models can lead to new achievements. This work makes progress along this direction by applying control-theoretical techniques to construct new recurrent neural dynamics for manipulating a perturbed nonstationary quadratic program (QP) with time-varying parameters considered. Specifically, to break the limitations of existing continuous-time models in handling nonstationary problems, a discrete recurrent neural dynamics model is proposed to robustly deal with noise. This work shows how iterative computational methods for solving nonstationary QP can be revisited, designed, and analyzed in a control framework. A modified Newton iteration model and an improved gradient-based neural dynamics are established by referring to the superior structural technology of the presented recurrent neural dynamics, where the chief breakthrough is their excellent convergence and robustness over the traditional models. Numerical experiments are conducted to show the eminence of the proposed models in solving perturbed nonstationary QP.
Collapse
|
10
|
Zhang Z, Du X, Jin L, Wang S, Wang L, Liu X. Large-scale underwater fish recognition via deep adversarial learning. Knowl Inf Syst 2022. [DOI: 10.1007/s10115-021-01643-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
11
|
Real-time cooperative kinematic control for multiple robots in distributed scenarios with dynamic neural networks. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.12.038] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
12
|
Liu M, Ma D, Li S. Neural dynamics for adaptive attitude tracking control of a flapping wing micro aerial vehicle. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.05.088] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
13
|
A generalized approach to solve perfect Bayesian Nash equilibrium for practical network attack and defense. Inf Sci (N Y) 2021. [DOI: 10.1016/j.ins.2021.06.078] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
|
14
|
Peng B, Jin L, Shang M. Multi-robot competitive tracking based on k-WTA neural network with one single neuron. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.07.020] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
15
|
Liu B, Fu D, Qi Y, Huang H, Jin L. Noise-tolerant gradient-oriented neurodynamic model for solving the Sylvester equation. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2021.107514] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
|
16
|
Liu M, He L, Hu B, Li S. Recurrent neural network with noise rejection for cyclic motion generation of robotic manipulators. Neural Netw 2021; 138:164-178. [PMID: 33667935 DOI: 10.1016/j.neunet.2021.02.002] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2020] [Revised: 10/07/2020] [Accepted: 02/04/2021] [Indexed: 11/19/2022]
Abstract
Recurrent neural network (RNN), as a kind of neural network with outstanding computing capability, improvability, and hardware realizability, has been widely used in various fields, especially in robotics. In this paper, an RNN with noise rejection is deliberately constructed to remedy the issue of joint-angle drift frequently occurring during the cyclic motion generation (CMG) of a manipulator in a noisy environment. Different from general RNNs, the proposed RNN possesses inherent noise immunity, especially for time-varying polynomial noises. Besides, proofs on the convergence of the proposed RNN in the absence and presence of noises are given. Furthermore, we carry out simulations on manipulators PUMA 560 and UR5 to demonstrate the reliability of the proposed RNN in remedying joint-angle drift, and comparison simulations under different noisy conditions further verify its superiority. In addition, experiments are conducted on manipulator FRANKA Panda to elucidate the realizability of the proposed RNN.
Collapse
Affiliation(s)
- Mei Liu
- School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China; University of Chinese Academy of Sciences, Beijing 100049, China
| | - Li He
- School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China; University of Chinese Academy of Sciences, Beijing 100049, China
| | - Bin Hu
- School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China
| | - Shuai Li
- School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China.
| |
Collapse
|
17
|
Shi T, Tian Y, Sun Z, Zhang B, Pang Z, Yu J, Zhang X. A New Projected Active Set Conjugate Gradient Approach for Taylor-Type Model Predictive Control: Application to Lower Limb Rehabilitation Robots With Passive and Active Rehabilitation. Front Neurorobot 2020; 14:559048. [PMID: 33343324 PMCID: PMC7744727 DOI: 10.3389/fnbot.2020.559048] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2020] [Accepted: 10/29/2020] [Indexed: 11/25/2022] Open
Abstract
In this paper, a three-order Taylor-type numerical differentiation formula is firstly utilized to linearize and discretize constrained conditions of model predictive control (MPC), which can be generalized from lower limb rehabilitation robots. Meanwhile, a new numerical approach that projected an active set conjugate gradient approach is proposed, analyzed, and investigated to solve MPC. This numerical approach not only incorporates both the active set and conjugate gradient approach but also utilizes a projective operator, which can guarantee that the equality constraints are always satisfied. Furthermore, rigorous proof of feasibility and global convergence also shows that the proposed approach can effectively solve MPC with equality and bound constraints. Finally, an echo state network (ESN) is established in simulations to realize intention recognition for human–machine interactive control and active rehabilitation training of lower-limb rehabilitation robots; simulation results are also reported and analyzed to substantiate that ESN can accurately identify motion intention, and the projected active set conjugate gradient approach is feasible and effective for lower-limb rehabilitation robot of MPC with passive and active rehabilitation training. This approach also ensures computational when disturbed by uncertainties in system.
Collapse
Affiliation(s)
- Tian Shi
- College of Communication Engineering, Jilin University, Changchun, China
| | - Yantao Tian
- College of Communication Engineering, Jilin University, Changchun, China
| | - Zhongbo Sun
- Department of Control Engineering, Changchun University of Technology, Changchun, China.,Key Laboratory of Bionic Engineering of Ministry of Education, Jilin University, Changchun, China
| | - Bangcheng Zhang
- School of Mechatronical Engineering, Changchun University of Technology, Changchun, China
| | - Zaixiang Pang
- School of Mechatronical Engineering, Changchun University of Technology, Changchun, China
| | - Junzhi Yu
- State Key Laboratory for Turbulence and Complex Systems, Department of Advanced Manufacturing and Robotics, College of Engineering, Peking University, Beijing, China
| | - Xin Zhang
- Department of Control Engineering, Changchun University of Technology, Changchun, China
| |
Collapse
|