Constrained Quaternion-Variable Convex Optimization: A Quaternion-Valued Recurrent Neural Network Approach

A Dynamic Parameter Noise-Tolerant Zeroing Neural Network for Time-Varying Quaternion Matrix Equation With Applications

As a common and significant problem in the field of industrial information, the time-varying quaternion matrix equation (TV-QME) is considered in this article and addressed by an improved zeroing neural network (ZNN) method based on the real representation of the quaternion. In the light of an improved dynamic parameter (IDP) and an innovative activation function (IAF), a dynamic parameter noise-tolerant ZNN (DPNTZNN) model is put forward for solving the TV-QME. The presented IDP with the character of changing with the residual error and the proposed IAF with the remarkable performance can strongly enhance the convergence and robustness of the DPNTZNN model. Therefore, the DPNTZNN model possesses fast predefined-time convergence and superior robustness under different noise environments, which are theoretically analyzed in detail. Besides, the provided simulative experiments verify the advantages of the DPNTZNN model for solving the TV-QME, especially compared with other ZNN models. Finally, the DPNTZNN model is applied to image restoration, which further illustrates the practicality of the DPNTZNN model.

Exponential Convergence of Primal-Dual Dynamics Under General Conditions and Its Application to Distributed Optimization

In this article, we establish the local and global exponential convergence of a primal-dual dynamics (PDD) for solving equality-constrained optimization problems without strong convexity and full row rank assumption on the equality constraint matrix. Under the metric subregularity of Karush-Kuhn-Tucker (KKT) mapping, we prove the local exponential convergence of the dynamics. Moreover, we establish the global exponential convergence of the dynamics in an invariant subspace under a technically designed condition which is weaker than strong convexity. As an application, the obtained theoretical results are used to show the exponential convergence of several existing state-of-the-art primal-dual algorithms for solving distributed optimization without strong convexity. Finally, we provide some experiments to demonstrate the effectiveness of our results.

A novel fixed-time error-monitoring neural network for solving dynamic quaternion-valued Sylvester equations

This paper addresses the dynamic quaternion-valued Sylvester equation (DQSE) using the quaternion real representation and the neural network method. To transform the Sylvester equation in the quaternion field into an equivalent equation in the real field, three different real representation modes for the quaternion are adopted by considering the non-commutativity of quaternion multiplication. Based on the equivalent Sylvester equation in the real field, a novel recurrent neural network model with an integral design formula is proposed to solve the DQSE. The proposed model, referred to as the fixed-time error-monitoring neural network (FTEMNN), achieves fixed-time convergence through the action of a state-of-the-art nonlinear activation function. The fixed-time convergence of the FTEMNN model is theoretically analyzed. Two examples are presented to verify the performance of the FTEMNN model with a specific focus on fixed-time convergence. Furthermore, the chattering phenomenon of the FTEMNN model is discussed, and a saturation function scheme is designed. Finally, the practical value of the FTEMNN model is demonstrated through its application to image fusion denoising.

Clifford-Valued Distributed Optimization Based on Recurrent Neural Networks

In this paper, we address the Clifford-valued distributed optimization subject to linear equality and inequality constraints. The objective function of the optimization problems is composed of the sum of convex functions defined in the Clifford domain. Based on the generalized Clifford gradient, a system of multiple Clifford-valued recurrent neural networks (RNNs) is proposed for solving the distributed optimization problems. Each Clifford-valued RNN minimizes a local objective function individually, with local interactions with others. The convergence of the neural system is rigorously proved based on the Lyapunov theory. Two illustrative examples are delineated to demonstrate the viability of the results in this article.

Decentralized ADMM with compressed and event-triggered communication

This paper considers the decentralized optimization problem, where agents in a network cooperate to minimize the sum of their local objective functions by communication and local computation. We propose a decentralized second-order communication-efficient algorithm called communication-censored and communication-compressed quadratically approximated alternating direction method of multipliers (ADMM), termed as CC-DQM, by combining event-triggered communication with compressed communication. In CC-DQM, agents are allowed to transmit the compressed message only when the current primal variables have changed greatly compared to its last estimate. Moreover, to relieve the computation cost, the update of Hessian is also scheduled by the trigger condition. Theoretical analysis shows that the proposed algorithm can still maintain an exact linear convergence, despite the existence of compression error and intermittent communication, if the local objective functions are strongly convex and smooth. Finally, numerical experiments demonstrate its satisfactory communication efficiency.

Fast Quaternion Product Units for Learning Disentangled Representations in [Formula: see text]

Real-world 3D structured data like point clouds and skeletons often can be represented as data in a 3D rotation group (denoted as [Formula: see text]). However, most existing neural networks are tailored for the data in the euclidean space, which makes the 3D rotation data not closed under their algebraic operations and leads to sub-optimal performance in 3D-related learning tasks. To resolve the issues caused by the above mismatching between data and model, we propose a novel non-real neuron model called quaternion product unit (QPU) to represent data on 3D rotation groups. The proposed QPU leverages quaternion algebra and the law of the 3D rotation group, representing 3D rotation data as quaternions and merging them via a weighted chain of Hamilton products. We demonstrate that the QPU mathematically maintains the [Formula: see text] structure of the 3D rotation data during the inference process and disentangles the 3D representations into "rotation-invariant" features and "rotation-equivariant" features, respectively. Moreover, we design a fast QPU to accelerate the computation of QPU. The fast QPU applies a tree-structured data indexing process, and accordingly, leverages the power of parallel computing, which reduces the computational complexity of QPU in a single thread from O(N) to O(logN). Taking the fast QPU as a basic module, we develop a series of quaternion neural networks (QNNs), including quaternion multi-layer perceptron (QMLP), quaternion message passing (QMP), and so on. In addition, we make the QNNs compatible with conventional real-valued neural networks and applicable for both skeletons and point clouds. Experiments on synthetic and real-world 3D tasks show that the QNNs based on our fast QPUs are superior to state-of-the-art real-valued models, especially in the scenarios requiring the robustness to random rotations. The code of this work is available at https://github.com/SuferQin/Fast-QPU.

An RNN-Based Algorithm for Decentralized-Partial-Consensus Constrained Optimization

This technical note proposes a decentralized-partial-consensus optimization (DPCO) problem with inequality constraints. The partial-consensus matrix originating from the Laplacian matrix is constructed to tackle the partial-consensus constraints. A continuous-time algorithm based on multiple interconnected recurrent neural networks (RNNs) is derived to solve the optimization problem. In addition, based on nonsmooth analysis and Lyapunov theory, the convergence of continuous-time algorithm is further proved. Finally, several examples demonstrate the effectiveness of main results.

Fast Algorithms for Deep Octonion Networks

This brief presents the results of a study of the possibilities of reducing the arithmetic complexity of computing basic operations in octonionic neural networks and also proposes new algorithmic solutions for efficiently performing these operations. Here, we primarily mean the operation of multiplying octonions, the operation of computing the dot product of two octonion-valued vectors, and the operation of multiple multiplications of an octonion by several other octonions. In order to reduce the computational complexity of these operations, it is proposed to use the fast Walsh-Hadamard transform, which is well known in digital signal processing. Using this transform reduces the number of multiplications and additions of real numbers required to perform computations. Thus, the use of the proposed algorithms will speed up computations in octonion-valued neural networks.

Dynamic Event-Triggered State Estimation for Markov Jump Neural Networks With Partially Unknown Probabilities

This article focuses on the investigation of finite-time dissipative state estimation for Markov jump neural networks. First, in view of the subsistent phenomenon that the state estimator cannot capture the system modes synchronously, the hidden Markov model with partly unknown probabilities is introduced in this article to describe such asynchronization constraint. For the upper limit of network bandwidth and computing resources, a novel dynamic event-triggered transmission mechanism, whose threshold parameter is constructed as an adjustable diagonal matrix, is set between the estimator and the original system to avoid data collision and save energy. Then, with the assistance of Lyapunov techniques, an event-based asynchronous state estimator is designed to ensure that the resulting system is finite-time bounded with a prescribed dissipation performance index. Ultimately, the effectiveness of the proposed estimator design approach combining with a dynamic event-triggered transmission mechanism is demonstrated by a numerical example.

Finite-Time and Fixed-Time Synchronization of Quaternion-Valued Neural Networks With/Without Mixed Delays: An Improved One-Norm Method

In this article, the finite-time synchronization (FTSYN) of a class of quaternion-valued neural networks (QVNNs) with discrete and distributed time delays is studied. Furthermore, the FTSYN and fixed-time synchronization (FIXSYN) of the QVNNs without time delay are investigated. Different from the existing results, which used decomposition techniques, by introducing an improved one-norm, we use a direct analytical method to study the synchronization problems. Incidentally, several properties of one-norm of the quaternion are analyzed, and then, three effective controllers are proposed to synchronize the drive and response QVNNs within a finite time or fixed time. Moreover, efficient criteria are proposed to guarantee that the synchronization of QVNNs with or without mixed time delays can be realized within a finite and fixed time interval, respectively. In addition, the settling times are reckoned. Compared with the existing work, our advantages are mainly reflected in the simpler Lyapunov analytical process and more general activation function. Finally, the validity and practicability of the conclusions are illustrated via four numerical examples.

A Distributed Optimization Problem Subject to Partial-Impact Cost Functions

This article focuses on a distributed optimization problem subject to partial-impact cost functions that relates to two decision variable vectors. To this end, two algorithms are presented with the aim of solving the considered optimization problem in a structure fashion and in a gradient fashion, respectively. Furthermore, a connection between the equilibrium of the induced algorithm and the involved optimization problem is established, with the aid of the tools from nonsmooth analysis and change of coordinate theorem. Two numerical examples with practical significance are given to demonstrate the efficiency of the designed algorithm.

Matrix-Form Neural Networks for Complex-Variable Basis Pursuit Problem With Application to Sparse Signal Reconstruction

In this article, a continuous-time complex-valued projection neural network (CCPNN) in a matrix state space is first proposed for a general complex-variable basis pursuit problem. The proposed CCPNN is proved to be stable in the sense of Lyapunov and to be globally convergent to the optimal solution under the condition that the sensing matrix is not row full rank. Furthermore, an improved discrete-time complex projection neural network (IDCPNN) is proposed by discretizing the CCPNN model. The proposed IDCPNN consists of a two-step stop strategy to reduce the calculational cost. The proposed IDCPNN is theoretically guaranteed to be global convergent to the optimal solution. Finally, the proposed IDCPNN is applied to the reconstruction of sparse signals based on compressed sensing. Computed results show that the proposed IDCPNN is superior to related complex-valued neural networks and conventional basis pursuit algorithms in terms of solution quality and computation time.

Recurrent Neural Dynamics Models for Perturbed Nonstationary Quadratic Programs: A Control-Theoretical Perspective

Recent decades have witnessed a trend that control-theoretical techniques are widely leveraged in various areas, e.g., design and analysis of computational models. Computational methods can be modeled as a controller and searching the equilibrium point of a dynamical system is identical to solving an algebraic equation. Thus, absorbing mature technologies in control theory and integrating it with neural dynamics models can lead to new achievements. This work makes progress along this direction by applying control-theoretical techniques to construct new recurrent neural dynamics for manipulating a perturbed nonstationary quadratic program (QP) with time-varying parameters considered. Specifically, to break the limitations of existing continuous-time models in handling nonstationary problems, a discrete recurrent neural dynamics model is proposed to robustly deal with noise. This work shows how iterative computational methods for solving nonstationary QP can be revisited, designed, and analyzed in a control framework. A modified Newton iteration model and an improved gradient-based neural dynamics are established by referring to the superior structural technology of the presented recurrent neural dynamics, where the chief breakthrough is their excellent convergence and robustness over the traditional models. Numerical experiments are conducted to show the eminence of the proposed models in solving perturbed nonstationary QP.

Fixed/Preassigned-time synchronization of quaternion-valued neural networks via pure power-law control

The fixed-time synchronization and preassigned-time synchronization of quaternion-valued neural networks are concerned in this article. By developing fixed-time stability and proposing a pure power-law control scheme, some simple conditions are obtained to realize fixed-time synchronization of quaternion-valued neural networks and the upper bound of the synchronized time is provided. Furthermore, the preassigned-time synchronization of quaternion-valued neural networks is investigated based on pure power-law control design, where the synchronization time is preassigned in advance and the control gains are finite. Note that the designed controllers in this paper are the pure power-law forms, which are simpler and more effective compared with the traditional design composed of the linear part and power-law part. Eventually, an example is given to illustrate the feasibility and validity of the results obtained.

Penalty Method for Constrained Distributed Quaternion-Variable Optimization

This article studies the constrained optimization problems in the quaternion regime via a distributed fashion. We begin with presenting some differences for the generalized gradient between the real and quaternion domains. Then, an algorithm for the considered optimization problem is given, by which the desired optimization problem is transformed into an unconstrained setup. Using the tools from the Lyapunov-based technique and nonsmooth analysis, the convergence property associated with the devised algorithm is further guaranteed. In addition, the designed algorithm has the potential for solving distributed neurodynamic optimization problems as a recurrent neural network. Finally, a numerical example involving machine learning is given to illustrate the efficiency of the obtained results.