Stability of Solutions to Systems of Nonlinear Differential Equations with Discontinuous Right-Hand Sides: Applications to Hopfield Artificial Neural Networks

In this paper, we study the stability of solutions to systems of differential equations with discontinuous right-hand sides. We have investigated nonlinear and linear equations. Stability sufficient conditions for linear equations are expressed as a logarithmic norm for coefficients of systems of equations. Stability sufficient conditions for nonlinear equations are expressed as the logarithmic norm of the Jacobian of the right-hand side of the system of equations. Sufficient conditions for the stability of solutions of systems of differential equations expressed in terms of logarithmic norms of the right-hand sides of equations (for systems of linear equations) and the Jacobian of right-hand sides (for nonlinear equations) have the following advantages: (1) in investigating stability in different metrics from the same standpoints, we have obtained a set of sufficient conditions; (2) sufficient conditions are easily expressed; (3) robustness areas of systems are easily determined with respect to the variation of their parameters; (4) in case of impulse action, information on moments of impact distribution is not required; (5) a method to obtain sufficient conditions of stability is extended to other definitions of stability (in particular, to p-moment stability). The obtained sufficient conditions are used to study Hopfield neural networks with discontinuous synapses and discontinuous activation functions.

Standard representation and unified stability analysis for dynamic artificial neural network models

An overview is provided of dynamic artificial neural network models (DANNs) for nonlinear dynamical system identification and control problems, and convex stability conditions are proposed that are less conservative than past results. The three most popular classes of dynamic artificial neural network models are described, with their mathematical representations and architectures followed by transformations based on their block diagrams that are convenient for stability and performance analyses. Classes of nonlinear dynamical systems that are universally approximated by such models are characterized, which include rigorous upper bounds on the approximation errors. A unified framework and linear matrix inequality-based stability conditions are described for different classes of dynamic artificial neural network models that take additional information into account such as local slope restrictions and whether the nonlinearities within the DANNs are odd. A theoretical example shows reduced conservatism obtained by the conditions.

Dissipativity and stability analysis of fractional-order complex-valued neural networks with time delay

As we know, the notion of dissipativity is an important dynamical property of neural networks. Thus, the analysis of dissipativity of neural networks with time delay is becoming more and more important in the research field. In this paper, the authors establish a class of fractional-order complex-valued neural networks (FCVNNs) with time delay, and intensively study the problem of dissipativity, as well as global asymptotic stability of the considered FCVNNs with time delay. Based on the fractional Halanay inequality and suitable Lyapunov functions, some new sufficient conditions are obtained that guarantee the dissipativity of FCVNNs with time delay. Moreover, some sufficient conditions are derived in order to ensure the global asymptotic stability of the addressed FCVNNs with time delay. Finally, two numerical simulations are posed to ensure that the attention of our main results are valuable.

Timescale separation in recurrent neural networks

Supervised learning in recurrent neural networks involves two processes: the neuron activity from which gradients are estimated and the process on connection parameters induced by these measurements. A problem such algorithms must address is how to balance the relative rates of these activities so that accurate sensitivity estimates are obtained while still allowing synaptic modification to take place at a rate sufficient for learning. We show how to calculate a sufficient timescale separation between these two processes for a class of contracting neural networks.

Existence and stability of periodic solution of impulsive neural systems with complex deviating arguments

This paper discusses a class of impulsive neural networks with the variable delay and complex deviating arguments. By using Mawhin's continuation theorem of coincidence degree and the Halanay-type inequalities, several sufficient conditions for impulsive neural networks are established for the existence and globally exponential stability of periodic solutions, respectively. Furthermore, the obtained results are applied to some typical impulsive neural network systems as special cases, with a real-life example to show feasibility of our results.

Improved delay-dependent stability of neutral type neural networks with distributed delays

This paper deals with the problem of improved delay-dependent stability analysis of neutral type neural networks with distributed delays. These conditions are in terms of linear matrix inequality (LMI), easily checked by recently developed algorithms in solving linear matrix inequalities (LMIs). Finally, numerical examples demonstrate effectiveness of the proposed method.

STABILITY ANALYSIS OF A CLASS OF GENERAL PERIODIC NEURAL NETWORKS WITH DELAYS AND IMPULSES

Based on the inequality analysis, matrix theory and spectral theory, a class of general periodic neural networks with delays and impulses is studied. Some sufficient conditions are established for the existence and globally exponential stability of a unique periodic solution. Furthermore, the results are applied to some typical impulsive neural network systems as special cases, with a real-life example to show feasibility of our results.

Stability analysis of Hopfield-type neural networks

The paper applies several concepts in robust control research such as linear matrix inequalities, edge theorem, parameter-dependent Lyapunov function, and Popov criteria to investigate the stability property of Hopfield-type neural networks. The existence and uniqueness of an equilibrium is formulated as a matrix determinant problem. An induction scheme is used to find the equilibrium. To verify whether the determinant is nozero for a class of matrix, a numerical range test is proposed. Several robust control techniques in particular linear matrix inequalities are used to characterize the local stability of the neural networks around the equilibrium. The global stability of the Hopfield neural networks is then addressed using a parameter-dependent Lyapunov function technique. All these results are shown to generalize existing results in verifying the existence/uniqueness of the equilibrium and local/global stability of Hopfield-type neural networks.

Equivalence between local exponential stability of the unique equilibrium point and global stability for Hopfield-type neural networks with two neurons

In a recent paper, Fang and Kincaid proposed an open problem about the relationship between the local stability of the unique equilibrium point and the global stability for a Hopfield-type neural network with continuously differentiable and monotonically increasing activation functions. As a partial answer to the question, in the two-neuron case it is proved that for each given specific interconnection weight matrix, a Hopfield-type neural network has a unique equilibrium point which is also locally exponentially stable for any activation functions and for any other network parameters if and only if the network is globally asymptotically stable for any activation functions and for any other network parameters. If the derivatives of the activation functions of the network are bounded, then the network is globally exponentially stable for any activation functions and for any other network parameters.

A reference model approach to stability analysis of neural networks

In this paper, a novel methodology called a reference model approach to stability analysis of neural networks is proposed. The core of the new approach is to study a neural network model with reference to other related models, so that different modeling approaches can be combinatively used and powerfully cross-fertilized. Focused on two representative neural network modeling approaches (the neuron state modeling approach and the local field modeling approach), we establish a rigorous theoretical basis on the feasibility and efficiency of the reference model approach. The new approach has been used to develop a series of new, generic stability theories for various neural network models. These results have been applied to several typical neural network systems including the Hopfield-type neural networks, the recurrent back-propagation neural networks, the BSB-type neural networks, the bound-constraints optimization neural networks, and the cellular neural networks. The results obtained unify, sharpen or generalize most of the existing stability assertions, and illustrate the feasibility and power of the new method.

Stability Analysis and the Stabilization of a Class of Discrete-Time Dynamic Neural Networks

This paper deals with problems of stability and the stabilization of discrete-time neural networks. Neural structures under consideration belong to the class of the so-called locally recurrent globally feedforward networks. The single processing unit possesses dynamic behavior. It is realized by introducing into the neuron structure a linear dynamic system in the form of an infinite impulse response filter. In this way, a dynamic neural network is obtained. It is well known that the crucial problem with neural networks of the dynamic type is stability as well as stabilization in learning problems. The paper formulates stability conditions for the analyzed class of neural networks. Moreover, a stabilization problem is defined and solved as a constrained optimization task. In order to tackle this problem two methods are proposed. The first one is based on a gradient projection (GP) and the second one on a minimum distance projection (MDP). It is worth noting that these methods can be easily introduced into the existing learning algorithm as an additional step, and suitable convergence conditions can be developed for them. The efficiency and usefulness of the proposed approaches are justified by using a number of experiments including numerical complexity analysis, stabilization effectiveness, and the identification of an industrial process.

Some generalized sufficient convergence criteria for nonlinear continuous neural networks

A reason for applying the direct method of Lyapunov to artificial neural networks (ANNs) is to design dynamical neural networks so that they exhibit global asymptotic stability. Lyapunov functions that frequently appear in the ANN literature include the quadratic function, the Persidskii function, and the Luré-Postnikov function. This contribution revisits the quadratic function and shows that via Krasovskii-like stability criteria, it is possible to have a very simple and systematic procedure to obtain not only new and generalized results but also well-known sufficient conditions for convergence established recently by non-Lyapunov methods, such as the matrix measure and nonlinear measure.

Analyzing stability of equilibrium points in neural networks: a general approach

Networks of coupled neural systems represent an important class of models in computational neuroscience. In some applications it is required that equilibrium points in these networks remain stable under parameter variations. Here we present a general methodology to yield explicit constraints on the coupling strengths to ensure the stability of the equilibrium point. Two models of coupled excitatory-inhibitory oscillators are used to illustrate the approach.

A comparative study of two modeling approaches in neural networks

The neuron state modeling and the local field modeling provides two fundamental modeling approaches to neural network research, based on which a neural network system can be called either as a static neural network model or as a local field neural network model. These two models are theoretically compared in terms of their trajectory transformation property, equilibrium correspondence property, nontrivial attractive manifold property, global convergence as well as stability in many different senses. The comparison reveals an important stability invariance property of the two models in the sense that the stability (in any sense) of the static model is equivalent to that of a subsystem deduced from the local field model when restricted to a specific manifold. Such stability invariance property lays a sound theoretical foundation of validity of a useful, cross-fertilization type stability analysis methodology for various neural network models.

Global dissipativity of continuous-time recurrent neural networks with time delay

This paper addresses the global dissipativity of a general class of continuous-time recurrent neural networks. First, the concepts of global dissipation and global exponential dissipation are defined and elaborated. Next, the sets of global dissipativity and global exponentially dissipativity are characterized using the parameters of recurrent neural network models. In particular, it is shown that the Hopfield network and cellular neural networks with or without time delays are dissipative systems.

New conditions on global stability of Cohen-Grossberg neural networks

In this letter, we discuss the dynamics of the Cohen-Grossberg neural networks. We provide a new and relaxed set of sufficient conditions for the Cohen-Grossberg networks to be absolutely stable and exponentially stable globally. We also provide an estimate of the rate of convergence.

Global convergence rate of recurrently connected neural networks

We discuss recurrently connected neural networks, investigating their global exponential stability (GES). Some sufficient conditions for a class of recurrent neural networks belonging to GES are given. Sharp convergence rate is given too.

Global exponential stability of continuous-time interval neural networks

This paper addresses global robust stability of a class of continuous-time interval neural networks that contain time-invariant uncertain parameters with their values being unknown but bounded in given compact sets. We first introduce the concept of diagonally constrained interval neural networks and present a necessary and sufficient condition for global exponential stability of these interval neural networks irregardless of any bounds of non-diagonal uncertain parameters in connection weight matrices. Then we extend the robust stability result to general interval neural networks by giving a sufficient condition. Simulation results illustrate the characteristics of the main results.

A new approach to stability of neural networks with time-varying delays

The stability of neural networks is a prerequisite for successful applications of the networks as either associative memories or optimization solvers. Because the integration and communication delays are ubiquitous, the stability of neural networks with delays has received extensive attention. However, the approach used in the previous investigation is mainly based on Liapunov's direct method. Since the construction of Liapunov function is very skilful, there is little compatibility among the existing results. In this paper, we develop a new approach to stability analysis of Hopfield-type neural networks with time-varying delays by defining two novel quantities of nonlinear function similar to the matrix norm and the matrix measure, respectively. With the new approach, we present sufficient conditions of the stability, which are either the generalization of those existing or new. The developed approach may be also applied for any general system with time delays rather than Hopfield-type neural networks.

On the stability analysis of delayed neural networks systems

In this paper, the problems of stability of delayed neural networks are investigated, including the stability of discrete and distributed delayed neural networks. Under the generalization of dropping the Lipschitzian hypotheses for output functions, some stability criteria are obtained by using the Liapunov functional method. We do not assume the symmetry of the connection matrix and we establish that the system admits a unique equilibrium point in which the output functions do not satisfy the Lipschitz conditions and do not require them to be differential or strictly monotonously increasing. These criteria can be used to analyze the dynamics of biological neural systems or to design globally stable artificial neural networks.

New theorems on global convergence of some dynamical systems

In this paper, we discuss dynamical behaviors of recurrently asymmetrically connected neural networks. We propose a new approach to study global convergence of the networks. Better test conditions for global convergence are given.

Exponential convergence of delayed dynamical systems

We discuss some delayed dynamical systems, investigating their stability and convergence. We prove that under mild conditions, these delayed systems are global exponential convergent.

On impulsive autoassociative neural networks

Many systems existing in physics, chemistry, biology, engineering, and information science can be characterized by impulsive dynamics caused by abrupt jumps at certain instants during the process. These complex dynamical behaviors can be modeled by impulsive differential systems or impulsive neural networks. This paper formulates and studies a new model of impulsive autoassociative neural networks. Several fundamental issues, such as global exponential stability and existence and uniqueness of equilibria of such neural networks, are established.

Asymptotic hyperstability of a class of neural networks

This paper is concerned with the asymptotic hyperstability of recurrent neural networks. We derive based on the stability results necessary and sufficient conditions for the network parameters. The results we achieve are more general than those based on Lyapunov methods, since they provide milder constraints on the connection weights than the conventional results and do not suppose symmetry of the weights.

On delayed impulsive Hopfield neural networks(1)

Many evolutionary processes, particularly some biological systems, exhibit impulsive dynamical behaviors, which can be well described by impulsive Hopfield neural networks. This paper formulates and studies a model of delayed impulsive Hopfield neural networks. Several fundamental issues such as global exponential stability, existence and uniqueness of the equilibrium of such networks are established. A numerical example is given for illustration and interpretation of the theoretical results.

Analysis and optimal design of continuous neural networks with applications to associative memory

The asymptotic stability of a continuous neural network is analyzed for associative memory. An optimal design method is proposed which ensures the highest associative memory speed and guarantees the storage of each desired memory with attractivity. The network asymptotic stability is analyzed by means of a new energy function, and four theorems are obtained. By comparing these theorems with existing ones, it can be shown that in some cases they are consistent, while in others they are not equivalent but complementary to each other. Further study results in two more generalized conclusions, of which the existing conclusions are special cases. The network optimal design method is proposed in terms of an optimal associative memory theorem. Two application examples are presented to demonstrate the defeffectiveness of the optimal design method, which can be used to design the network for many applications.

Stability analysis of a class of noise perturbed neural networks

We establish robustness stability results for a specific type of artificial neural networks for associative memories under parameter perturbations and determine conditions that ensure the existence of asymptotically stable equilibria of the perturbed neural system that are the asymptotically stable equilibria of the original unperturbed neural network. The proposed stability analysis tool is the sliding mode control and it facilitates the analysis by considering only a reduced-order system instead of the original one and time-dependent external stimuli.