Synchronization and State Estimation of a Class of Hierarchical Hybrid Neural Networks With Time-Varying Delays

A Hardware-Friendly Low-Bit Power-of-Two Quantization Method for CNNs and Its FPGA Implementation

To address the problems of convolutional neural networks (CNNs) consuming more hardware resources (such as DSPs and RAMs on FPGAs) and their accuracy, efficiency, and resources being difficult to balance, meaning they cannot meet the requirements of industrial applications, we proposed an innovative low-bit power-of-two quantization method: the global sign-based network quantization (GSNQ). This method involves designing different quantization ranges according to the sign of the weights, which can provide a larger quantization-value range. Combined with the fine-grained and multi-scale global retraining method proposed in this paper, the accuracy loss of low-bit quantization can be effectively reduced. We also proposed a novel convolutional algorithm using shift operations to replace multiplication to help to deploy the GSNQ quantized models on FPGAs. Quantization comparison experiments performed on LeNet-5, AlexNet, VGG-Net, ResNet, and GoogLeNet showed that GSNQ has higher accuracy than most existing methods and achieves "lossless" quantization (i.e., the accuracy of the quantized CNN model is higher than the baseline) at low-bit quantization in most cases. FPGA comparison experiments showed that our convolutional algorithm does not occupy on-chip DSPs, and it also has a low comprehensive occupancy in terms of on-chip LUTs and FFs, which can effectively improve the computational parallelism, and this proves that GSNQ has good hardware-adaptation capability. This study provides theoretical and experimental support for the industrial application of CNNs.

Synchronization of Complex Dynamical Networks Subject to Noisy Sampling Interval and Packet Loss

This article focuses on the sampled-data synchronization issue for a class of complex dynamical networks (CDNs) subject to noisy sampling intervals and successive packet losses. The sampling intervals are subject to noisy perturbations, and categorical distribution is used to characterize the sampling errors of noisy sampling intervals. By means of the input delay approach, the CDN under consideration is first converted into a delay system with delayed input subject to dual randomness and probability distribution characteristic. To verify the probability distribution characteristic of the delayed input, a novel characterization method is proposed, which is not the same as that of some existing literature. Based on this, a unified framework is then established. By recurring to the techniques of stochastic analysis, a probability-distribution-dependent controller is designed to guarantee the mean-square exponential synchronization of the error dynamical network. Subsequently, a special model is considered where only the lower and upper bounds of delayed input are utilized. Finally, to verify the analysis results and testify the effectiveness and superiority of the designed synchronization algorithm, a numerical example and an example using Chua's circuit are given.

Synchronization of the neurons coupled with sequential developing electrical and chemical synapses

There is some evidence representing the sequential formation and elimination of electrical and chemical synapses in particular brain regions. Relying on this feature, this paper presents a purely mathematical modeling study on the synchronization among neurons connected by transient electrical synapses transformed to chemical synapses over time. This deletion and development of synapses are considered consecutive. The results represent that the transient synapses lead to burst synchronization of the neurons while the neurons are resting when both synapses exist constantly. The period of the transitions and also the time of presence of electrical synapses to chemical ones are effective on the synchronization. The larger synchronization error is obtained by increasing the transition period and the time of chemical synapses' existence.

Consensus Tracking Control of Uncertain Multiagent Systems With Sampled Data and Time-Varying Delay

In this article, the adaptive consensus tracking control is developed for uncertain multiagent systems with time-varying state delay in the case that leader's state is accessible at sampling instants. By proposing a distributed sampled observer with hybrid form, adaptive tracking controller with the complementary term is designed for first-order multiagent systems, and then is extended to high-order multiagent systems with the aid of dynamic surface control. Through the complementary term, the effects of parameter estimation error as well as dynamical terms with time-varying delays are eliminated and thus less conservative condition on time delays is required. It is proved that, under criteria in terms of linear matrix inequalities (LMIs), tracking error and estimation error exponentially converge to zero for first-order systems, and to a sufficiently small neighborhood of zero for high-order systems.

H∞ State Estimation for Neural Networks With General Activation Function and Mixed Time-Varying Delays

This article deals with H_∞ state estimation of neural networks with mixed delays. In order to make full use of delay information, novel delay-product Lyapunov-Krasovskii functional (LKF) by using parameterized delay interval is first constructed. Then, generalized free-weighting-matrix integral inequality is used to estimate the derivative of LKF to reduce the conservatism. Also, a more general activation function is further applied by combining with parameterized delay interval in order to obtain a more accurate estimator model. Finally, sufficient conditions are derived to confirm that the estimation error system is asymptotically stable with a prescribed H_∞ performance. Numerical examples are simulated to show the benefits of our proposed method.

Synchronization of Coupled Time-Delay Neural Networks With Mode-Dependent Average Dwell Time Switching

In the literature, the effects of switching with average dwell time (ADT), Markovian switching, and intermittent coupling on stability and synchronization of dynamic systems have been extensively investigated. However, all of them are considered separately because it seems that the three kinds of switching are different from each other. This article proposes a new concept to unify these switchings and considers global exponential synchronization almost surely (GES a.s.) in an array of neural networks (NNs) with mixed delays (including time-varying delay and unbounded distributed delay), switching topology, and stochastic perturbations. A general switching mechanism with transition probability (TP) and mode-dependent ADT (MDADT) (i.e., TP-based MDADT switching in this article) is introduced. By designing a multiple Lyapunov-Krasovskii functional and developing a set of new analytical techniques, sufficient conditions are obtained to ensure that the coupled NNs with the general switching topology achieve GES a.s., even in the case that there are both synchronizing and nonsynchronizing modes. Our results have removed the restrictive condition that the increment coefficients of the multiple Lyapunov-Krasovskii functional at switching instants are larger than one. As applications, the coupled NNs with Markovian switching topology and intermittent coupling are employed. Numerical examples are provided to demonstrate the effectiveness and the merits of the theoretical analysis.

Stubborn State Estimation for Delayed Neural Networks Using Saturating Output Errors

This paper is concerned with the stubborn state estimation of delayed neural networks that subject to a general class of disturbances in measurements, including outliers and impulsive disturbances as its special cases. This class of disturbances may be unbounded, irregular, and assorted; therefore, they can hardly be suppressed by existing identification-based estimation approaches. In this paper, a stubborn state estimator is constructed by intentionally devising a saturation scheme on the injection of output estimation error. The embedded saturation can effectively resist the influences from these measurement disturbances by saturating them. Moreover, the saturation threshold in the designed scheme is not constant but governed by a dynamic equation with parameters to be designed. Benefiting from this adaptiveness, the estimator obtains more freedom in dealing with various disturbances. By combining a novel Lyapunov functional, the generalized sector condition and two latest integral inequalities, a delay-dependent criterion is derived in a less conservative way to check whether the estimation error system with this dynamic saturation is globally stable. A sufficient condition with two tuning scalars is further provided to codesign the gain of the state estimator and the evolution law of the saturation threshold. Finally, two numerical examples are used to illustrate the stubbornness of this state estimator in the presence of measurement outliers or impulsive disturbances.

Estimation of Domain of Attraction for Aperiodic Sampled-Data Switched Delayed Neural Networks Subject to Actuator Saturation

In this paper, for the case of the asynchronous switching caused by that subsystem's switching occuring during a sampling interval, the domain of attraction estimation problem is investigated for aperiodic sampled-data switched delayed neural networks (ASDSDNNs) subject to actuator saturation. A parameters-dependent time-scheduled Lyapunov functional consisting of a novel looped-functional is constructed using segmentation technology and linear interpolation. By employing this novel functional and using an average dwell time (ADT) approach, exponential stability criteria are proposed for polytopic uncertain ASDSDNNs subject to actuator saturation. And a relationship between ADT and sampling period is revealed for ASDSDNNs. As a corollary, exponential stability criteria are proposed for nominal ASDSDNNs subject to actuator saturation. Furthermore, by describing the domain of attraction as a time-varying ellipsoid determined by the time-scheduled Lyapunov matrix, the proposed theoretical conditions are transformed into a linear matrix inequality (LMI)-based multi-objective optimization problem. The dynamic estimates of the domain of attraction for ASDSDNNs are solved. Numerical simulation examples are provided to illustrate the effectiveness of the proposed method.

Stability Analysis for Delayed Neural Networks via Improved Auxiliary Polynomial-Based Functions

This brief is concerned with stability analysis for delayed neural networks (DNNs). By establishing polynomials and introducing slack variables reasonably, some improved delay-product type of auxiliary polynomial-based functions (APFs) is developed to exploit additional degrees of freedom and more information on extra states. Then, by constructing Lyapunov-Krasovskii functional using APFs and integrals of quadratic forms with high order scalar functions, a novel stability criterion is derived for DNNs, in which the benefits of the improved inequalities are fully integrated and the information on delay and its derivative is well reflected. By virtue of the advantages of APFs, more desirable performance is achieved through the proposed approach, which is demonstrated by the numerical examples.

Finite-Horizon l2-l∞ Synchronization for Time-Varying Markovian Jump Neural Networks Under Mixed-Type Attacks: Observer-Based Case

This paper studies the synchronization issue of time-varying Markovian jump neural networks (NNs). The denial-of-service (DoS) attack is considered in the communication channel connecting master NNs and slave NNs. An observer is designed based on the measurements of master NNs transmitted over this unreliable channel to estimate their states. The deception attack is used to destroy the controller by changing the sign of the control signal. Then, the mixed-type attacks are expressed uniformly, and a synchronization error system is established using this function. A finite-horizon l₂-l_∞ performance is proposed, and sufficient conditions are derived to ensure that the synchronization error system satisfies this performance. The controllers are then obtained by a recursive linear matrix inequality algorithm. At last, a simulation result to show the feasibility of the developed results is given.

Dissipativity-based asynchronous control for discrete-time singular Markov jump systems with multiplicative noises

The singular systems, which could widely describe more general systems and present traits of physical features, are discussed in this study. Taking the fact that noises always exist in the state and output measurement of one singular system into consideration, which may cause some errors and decrease system performance, this article devotes itself to the dissipative control for discrete-time singular Markov jump systems (SMJSs) with multiplicative noises. To deal with the asynchronous phenomena between the system modes and the controller modes, a set of Markov chains are constructed. To make sure the closed-loop singular system is dissipative, a set of sufficient conditions are derived based on the linear matrix inequalities, and then the asynchronous controller is designed to ensure that SMJSs are stochastically admissible and strictly dissipative. Finally, a simulation example is carried out to verify the correctness of the derived theorem. The designed asynchronous controller improves the robustness of the controller and overcomes the asynchronous phenomenon. This control method can be applied in the fields of robot control system.

Synchronization of Coupled Markovian Reaction-Diffusion Neural Networks With Proportional Delays Via Quantized Control

The asymptotic synchronization of coupled reaction-diffusion neural networks with proportional delay and Markovian switching topologies is considered in this brief where the diffusion space does not need to contain the origin. The main objectives of this brief are to save communication resources and to reduce the conservativeness of the obtained synchronization criteria, which are carried out from the following two aspects: 1) mode-dependent quantized control technique is designed to reduce control cost and save communication channels and 2) Wirtinger inequality is utilized to deal with the reaction-diffusion terms in a matrix form and reciprocally convex technique combined with new Lyapunov-Krasovskii functional is used to derive delay-dependent synchronization criteria. The obtained results are general and formulated by linear matrix inequalities. Moreover, combined with an optimal algorithm, control gains with the least magnitude are designed.

Nonfragile State Estimation of Quantized Complex Networks With Switching Topologies

This paper considers the nonfragile $H_\infty $ estimation problem for a class of complex networks with switching topologies and quantization effects. The network architecture is assumed to be dynamic and evolves with time according to a random process subject to a sojourn probability. The coupled signal is to be quantized before transmission due to power and bandwidth constraints, and the quantization errors are transformed into sector-bounded uncertainties. The concept of nonfragility is introduced by inserting randomly occurred uncertainties into the estimator parameters to cope with the unavoidable small gain variations emerging from the implementations of estimators. Both the quantizers and the estimators have several operation modes depending on the switching signal of the underlying network structure. A sufficient condition is provided via a linear matrix inequality approach to ensure the estimation error dynamic to be stochastically stable in the absence of external disturbances, and the $H_\infty $ performance with a prescribed index is also satisfied. Finally, a numerical example is presented to clarify the validity of the proposed method.

Event-Triggered Asynchronous Guaranteed Cost Control for Markov Jump Discrete-Time Neural Networks With Distributed Delay and Channel Fading

This paper is concerned with the guaranteed cost control problem for a class of Markov jump discrete-time neural networks (NNs) with event-triggered mechanism, asynchronous jumping, and fading channels. The Markov jump NNs are introduced to be close to reality, where the modes of the NNs and guaranteed cost controller are determined by two mutually independent Markov chains. The asynchronous phenomenon is considered, which increases the difficulty of designing required mode-dependent controller. The event-triggered mechanism is designed by comparing the relative measurement error with the last triggered state at the process of data transmission, which is used to eliminate dispensable transmission and reduce the networked energy consumption. In addition, the signal fading is considered for the effect of signal reflection and shadow in wireless networks, which is modeled by the novel Rice fading models. Some novel sufficient conditions are obtained to guarantee that the closed-loop system reaches a specified cost value under the designed jumping state feedback control law in terms of linear matrix inequalities. Finally, some simulation results are provided to illustrate the effectiveness of the proposed method.

A Flexible Terminal Approach to Sampled-Data Exponentially Synchronization of Markovian Neural Networks With Time-Varying Delayed Signals

This paper investigates the problem of sampled-data (SD) exponentially synchronization for a class of Markovian neural networks with time-varying delayed signals. Based on the tunable parameter and convex combination computational method, a new approach named flexible terminal approach is proposed to reduce the conservatism of delay-dependent synchronization criteria. The SD subject to stochastic sampling period is introduced to exhibit the general phenomena of reality. Novel exponential synchronization criterion are derived by utilizing uniform Lyapunov-Krasovskii functional and suitable integral inequality. Finally, numerical examples are provided to show the usefulness and advantages of the proposed design procedure.

Energy-to-Peak State Estimation for Static Neural Networks With Interval Time-Varying Delays

This paper is concerned with energy-to-peak state estimation on static neural networks (SNNs) with interval time-varying delays. The objective is to design suitable delay-dependent state estimators such that the peak value of the estimation error state can be minimized for all disturbances with bounded energy. Note that the Lyapunov-Krasovskii functional (LKF) method plus proper integral inequalities provides a powerful tool in stability analysis and state estimation of delayed NNs. The main contribution of this paper lies in three points: 1) the relationship between two integral inequalities based on orthogonal and nonorthogonal polynomial sequences is disclosed. It is proven that the second-order Bessel-Legendre inequality (BLI), which is based on an orthogonal polynomial sequence, outperforms the second-order integral inequality recently established based on a nonorthogonal polynomial sequence; 2) the LKF method together with the second-order BLI is employed to derive some novel sufficient conditions such that the resulting estimation error system is globally asymptotically stable with desirable energy-to-peak performance, in which two types of time-varying delays are considered, allowing its derivative information is partly known or totally unknown; and 3) a linear-matrix-inequality-based approach is presented to design energy-to-peak state estimators for SNNs with two types of time-varying delays, whose efficiency is demonstrated via two widely studied numerical examples.

Asynchronous State Estimation for Discrete-Time Switched Complex Networks With Communication Constraints

This paper is concerned with the asynchronous state estimation for a class of discrete-time switched complex networks with communication constraints. An asynchronous estimator is designed to overcome the difficulty that each node cannot access to the topology/coupling information. Also, the event-based communication, signal quantization, and the random packet dropout problems are studied due to the limited communication resource. With the help of switched system theory and by resorting to some stochastic system analysis method, a sufficient condition is proposed to guarantee the exponential stability of estimation error system in the mean-square sense and a prescribed performance level is also ensured. The characterization of the desired estimator gains is derived in terms of the solution to a convex optimization problem. Finally, the effectiveness of the proposed design approach is demonstrated by a simulation example.

Nonfragile Exponential Synchronization of Delayed Complex Dynamical Networks With Memory Sampled-Data Control

This paper considers nonfragile exponential synchronization for complex dynamical networks (CDNs) with time-varying coupling delay. The sampled-data feedback control, which is assumed to allow norm-bounded uncertainty and involves a constant signal transmission delay, is constructed for the first time in this paper. By constructing a suitable augmented Lyapunov function, and with the help of introduced integral inequalities and employing the convex combination technique, a sufficient condition is developed, such that the nonfragile exponential stability of the error system is guaranteed. As a result, for the case of sampled-data control free of norm-bound uncertainties, some sufficient conditions of sampled-data synchronization criteria for the CDNs with time-varying coupling delay are presented. As the formulations are in the framework of linear matrix inequality, these conditions can be easily solved and implemented. Two illustrative examples are presented to demonstrate the effectiveness and merits of the proposed feedback control.

Finite-Time Synchronization of Coupled Hierarchical Hybrid Neural Networks With Time-Varying Delays

This paper is concerned with the finite-time synchronization problem of coupled hierarchical hybrid delayed neural networks. This coupled hierarchical hybrid neural networks consist of a higher level switching and a lower level Markovian jumping. The time-varying delays are dependent on not only switching signal but also jumping mode. By using a less conservative weighted integral inequality and stochastic multiple Lyapunov-Krasovskii functional, new finite-time synchronization criteria are obtained, which makes the state trajectories be kept within the prescribed bound in a time interval. Finally, an example is proposed to demonstrate the effectiveness of the obtained results.

Reachable Set Estimation for Markovian Jump Neural Networks With Time-Varying Delays

In this paper, the reachable set estimation problem is investigated for Markovian jump neural networks (NNs) with time-varying delays and bounded peak disturbances. Our goal is to find a set as small as possible which bounds all the state trajectories of the NNs under zero initial conditions. In the framework of Lyapunov-Krasovskii theorem, a newly-found summation inequality combined with the reciprocally convex approach is used to bound the difference of the proposed Lyapunov functional. A new less conservative condition dependent on the upper bound, the lower bound and the delay range of the time delay is established to guarantee that the state trajectories are bounded within an ellipsoid-like set. Then the result is extended to the case with incomplete transition probabilities and a more general condition is derived. Finally, examples including a genetic regulatory network are given to demonstrate the usefulness and the effectiveness of the results obtained in this paper.