Markovian Architectural Bias of Recurrent Neural Networks

Reachable set estimation and stochastic sampled-data exponential synchronization of Markovian jump neural networks with time-varying delays

In this paper, the stochastic sampled-data exponential synchronization problem for Markovian jump neural networks (MJNNs) with time-varying delays and the reachable set estimation (RSE) problem for MJNNs subjected to external disturbances are investigated. Firstly, assuming that two sampled-data periods satisfy Bernoulli distribution, and introducing two stochastic variables to represent the unknown input delay and the sampled-data period respectively, the mode-dependent two-sided loop-based Lyapunov functional (TSLBLF) is constructed, and the conditions for the mean square exponential stability of the error system are derived. Furthermore, a mode-dependent stochastic sampled-data controller is designed. Secondly, by analyzing the unit-energy bounded disturbance of MJNNs, a sufficient condition is proved that all states of MJNNs are confined to an ellipsoid under zero initial condition. In order to make the target ellipsoid contain the reachable set of the system, a stochastic sampled-data controller with RSE is designed. Eventually, two numerical examples and an analog resistor-capacitor network circuit are provided to show that the textual approach can obtain a larger sampled-data period than the existing approach.

Dynamic Event-Triggered State Estimation for Markov Jump Neural Networks With Partially Unknown Probabilities

This article focuses on the investigation of finite-time dissipative state estimation for Markov jump neural networks. First, in view of the subsistent phenomenon that the state estimator cannot capture the system modes synchronously, the hidden Markov model with partly unknown probabilities is introduced in this article to describe such asynchronization constraint. For the upper limit of network bandwidth and computing resources, a novel dynamic event-triggered transmission mechanism, whose threshold parameter is constructed as an adjustable diagonal matrix, is set between the estimator and the original system to avoid data collision and save energy. Then, with the assistance of Lyapunov techniques, an event-based asynchronous state estimator is designed to ensure that the resulting system is finite-time bounded with a prescribed dissipation performance index. Ultimately, the effectiveness of the proposed estimator design approach combining with a dynamic event-triggered transmission mechanism is demonstrated by a numerical example.

Integer Echo State Networks: Efficient Reservoir Computing for Digital Hardware

We propose an approximation of echo state networks (ESNs) that can be efficiently implemented on digital hardware based on the mathematics of hyperdimensional computing. The reservoir of the proposed integer ESN (intESN) is a vector containing only n -bits integers (where is normally sufficient for a satisfactory performance). The recurrent matrix multiplication is replaced with an efficient cyclic shift operation. The proposed intESN approach is verified with typical tasks in reservoir computing: memorizing of a sequence of inputs, classifying time series, and learning dynamic processes. Such architecture results in dramatic improvements in memory footprint and computational efficiency, with minimal performance loss. The experiments on a field-programmable gate array confirm that the proposed intESN approach is much more energy efficient than the conventional ESN.

l₂-l∞ State Estimation for Persistent Dwell-Time Switched Coupled Networks Subject to Round-Robin Protocol

This article is concerned with the issue of l₂ - l_∞ state estimation for nonlinear coupled networks, where the variation of coupling mode is governed by a set of switching signals satisfying a persistent dwell-time property. To solve the problem of data collisions in a constrained communication network, the round-robin protocol, as an important scheduling strategy for orchestrating the transmission order of sensor nodes, is introduced. Redundant channels with signal quantization are used to improve the reliability of data transmission. The main purpose is to determine an estimator that can guarantee the exponential stability in mean square sense and an l₂ - l_∞ performance level of the estimation error system. Based on the Lyapunov method, sufficient conditions for the addressed problem are established. The desired estimator gains can be obtained by addressing a convex optimization case. The correctness and availability of the developed approach are finally explained via two illustrative examples.

Finite-time and fixed-time anti-synchronization of Markovian neural networks with stochastic disturbances via switching control

This paper proposes a unified theoretical framework to study the problem of finite/fixed-time drive-response anti-synchronization for a class of Markovian stochastic neural networks. State feedback switching controllers without the sign function are designed to achieve the finite/fixed-time anti-synchronization of the addressed systems. Compared with the existing synchronization criteria, our results indicate that the controllers via the switching control without the sign function are given with less conservativeness, and the controllers without any sign function can deal with the chattering problem. By employing Lyapunov functional method and properties of the Weiner process, several finite/fixed-time synchronization criteria are presented and the corresponding settling times are calculated as well. Finally, three numerical examples are provided to illustrate the effectiveness of the theoretical results.

Nonfragile Dissipative Synchronization for Markovian Memristive Neural Networks: A Gain-Scheduled Control Scheme

In this paper, the dissipative synchronization control problem for Markovian jump memristive neural networks (MNNs) is addressed with fully considering the time-varying delays and the fragility problem in the process of implementing the gain-scheduled controller. A Markov jump model is introduced to describe the stochastic changing among the connection of MNNs and it makes the networks under consideration suitable for some actual circumstances. By utilizing some improved integral inequalities and constructing a proper Lyapunov-Krasovskii functional, several delay-dependent synchronization criteria with less conservatism are established to ensure the dynamic error system is strictly stochastically dissipative. Based on these criteria, the procedure of designing the desired nonfragile gain-scheduled controller is established, which can well handle the fragility problem in the process of implementing the controller. Finally, an illustrated example is employed to explain that the developed method is efficient and available.

Reduced-Order Filtering of Delayed Static Neural Networks With Markovian Jumping Parameters

The reduced-order filtering problems are investigated in this paper for static neural networks with Markovian jumping parameters and mode-dependent time-varying delays. By fully making use of integral inequalities, the designs of reduced-order and filters are discussed. The proper gain matrices of filters and the optimal performance indices are efficiently obtained by resolving corresponding convex optimization problems with the constraints of linear matrix inequalities. It is verified that the computational complexity for the reduced-order filter design is significantly reduced when compared with the full-order one. Furthermore, the nonfragile reduced-order filtering problems are also resolved in this paper. Two examples with simulation results are presented to demonstrate the feasibility and application of the established results.

Protocol-based state estimation for delayed Markovian jumping neural networks

This paper is concerned with the state estimation problem for a class of Markovian jumping neural networks (MJNNs) with sensor nonlinearities, mode-dependent time delays and stochastic disturbances subject to the Round-Robin (RR) scheduling mechanism. The system parameters experience switches among finite modes according to a Markov chain. As an equal allocation scheme, the RR communication protocol is introduced for efficient usage of limited bandwidth and energy saving. The update matrix method is adopted to deal with the periodic time-delays resulting from the RR protocol. The objective of the addressed problem is to construct a state estimator for the MJNNs such that the dynamics of the estimation error is exponentially ultimately bounded in the mean square with a certain upper bound. Sufficient conditions are established for the existence of the desired state estimator by resorting to a combination of the Lyapunov stability theory and the stochastic analysis technique. Furthermore, the estimator gain matrices are characterized in terms of the solution to a convex optimization problem. Finally, a numerical simulation example is exploited to demonstrate the effectiveness of the proposed estimator design strategy.

Extended Dissipative State Estimation for Markov Jump Neural Networks With Unreliable Links

This paper is concerned with the problem of extended dissipativity-based state estimation for discrete-time Markov jump neural networks (NNs), where the variation of the piecewise time-varying transition probabilities of Markov chain is subject to a set of switching signals satisfying an average dwell-time property. The communication links between the NNs and the estimator are assumed to be imperfect, where the phenomena of signal quantization and data packet dropouts occur simultaneously. The aim of this paper is to contribute with a Markov switching estimator design method, which ensures that the resulting error system is extended stochastically dissipative, in the simultaneous presences of packet dropouts and signal quantization stemmed from unreliable communication links. Sufficient conditions for the solvability of such a problem are established. Based on the derived conditions, an explicit expression of the desired Markov switching estimator is presented. Finally, two illustrated examples are given to show the effectiveness of the proposed design method.

Synchronization and State Estimation of a Class of Hierarchical Hybrid Neural Networks With Time-Varying Delays

This paper addresses the problems of synchronization and state estimation for a class of discrete-time hierarchical hybrid neural networks (NNs) with time-varying delays. The hierarchical hybrid feature consists of a higher level nondeterministic switching and a lower level stochastic switching. The latter is used to describe the NNs subject to Markovian modes transitions, whereas the former is of the average dwell-time switching regularity to model the supervisory orchestrating mechanism among these Markov jump NNs. The considered time delays are not only time-varying but also dependent on the mode of NNs on the lower layer in the hierarchical structure. Despite quantization and random data missing, the synchronized controllers and state estimators are designed such that the resulting error system is exponentially stable with an expected decay rate and has a prescribed H∞ disturbance attenuation level. Two numerical examples are provided to show the validity and potential of the developed results.

Resilient Asynchronous H∞ Filtering for Markov Jump Neural Networks With Unideal Measurements and Multiplicative Noises

This paper is concerned with the resilient H∞ filtering problem for a class of discrete-time Markov jump neural networks (NNs) with time-varying delays, unideal measurements, and multiplicative noises. The transitions of NNs modes and desired mode-dependent filters are considered to be asynchronous, and a nonhomogeneous mode transition matrix of filters is used to model the asynchronous jumps to different degrees that are also mode-dependent. The unknown time-varying delays are also supposed to be mode-dependent with lower and upper bounds known a priori. The unideal measurements model includes the phenomena of randomly occurring quantization and missing measurements in a unified form. The desired resilient filters are designed such that the filtering error system is stochastically stable with a guaranteed H∞ performance index. A monotonicity is disclosed in filtering performance index as the degree of asynchronous jumps changes. A numerical example is provided to demonstrate the potential and validity of the theoretical results.

Energy-to-peak state estimation for Markov jump RNNs with time-varying delays via nonsynchronous filter with nonstationary mode transitions

In this paper, the problem of energy-to-peak state estimation for a class of discrete-time Markov jump recurrent neural networks (RNNs) with randomly occurring nonlinearities (RONs) and time-varying delays is investigated. A practical phenomenon of nonsynchronous jumps between RNNs modes and desired mode-dependent filters is considered, and a nonstationary mode transition among the filters is used to model the nonsynchronous jumps to different degrees that are also mode dependent. The RONs are used to model a class of sector-like nonlinearities that occur in a probabilistic way according to a Bernoulli sequence. The time-varying delays are supposed to be mode dependent and unknown, but with known lower and upper bounds a priori. Sufficient conditions on the existence of the nonsynchronous filters are obtained such that the filtering error system is stochastically stable and achieves a prescribed energy-to-peak performance index. Further to the recent study on the class of nonsynchronous estimation problem, a monotonicity is observed in obtaining filtering performance index, while changing the degree of nonsynchronous jumps. A numerical example is presented to verify the theoretical findings.

Dynamically pre-trained deep recurrent neural networks using environmental monitoring data for predicting PM2.5

Fine particulate matter (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hbox {PM}_{2.5}$$\end{document}PM2.5) has a considerable impact on human health, the environment and climate change. It is estimated that with better predictions, US$9 billion can be saved over a 10-year period in the USA (State of the science fact sheet air quality. http://www.noaa.gov/factsheets/new, 2012). Therefore, it is crucial to keep developing models and systems that can accurately predict the concentration of major air pollutants. In this paper, our target is to predict \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hbox {PM}_{2.5}$$\end{document}PM2.5 concentration in Japan using environmental monitoring data obtained from physical sensors with improved accuracy over the currently employed prediction models. To do so, we propose a deep recurrent neural network (DRNN) that is enhanced with a novel pre-training method using auto-encoder especially designed for time series prediction. Additionally, sensors selection is performed within DRNN without harming the accuracy of the predictions by taking advantage of the sparsity found in the network. The numerical experiments show that DRNN with our proposed pre-training method is superior than when using a canonical and a state-of-the-art auto-encoder training method when applied to time series prediction. The experiments confirm that when compared against the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hbox {PM}_{2.5}$$\end{document}PM2.5 prediction system VENUS (National Institute for Environmental Studies. Visual Atmospheric Environment Utility System. http://envgis5.nies.go.jp/osenyosoku/, 2014), our technique improves the accuracy of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hbox {PM}_{2.5}$$\end{document}PM2.5 concentration level predictions that are being reported in Japan.

Non-fragile robust synchronization for Markovian jumping chaotic neural networks of neutral-type with randomly occurring uncertainties and mode-dependent time-varying delays

This paper deals with the problem of robust synchronization for uncertain chaotic neutral-type Markovian jumping neural networks with randomly occurring uncertainties and randomly occurring control gain fluctuations. Then, a sufficient condition is proposed for the existence of non-fragile output controller in terms of linear matrix inequalities (LMIs). Uncertainty terms are separately taken into consideration. This network involves both mode dependent discrete and mode dependent distributed time-varying delays. Based on the Lyapunov-Krasovskii functional (LKF) with new triple integral terms, convex combination technique and free-weighting matrices method, delay-dependent sufficient conditions for the solvability of these problems are established in terms of LMIs. Furthermore, the problem of non-fragile robust synchronization is reduced to the optimization problem involving LMIs, and the detailed algorithm for solving the restricted LMIs is given. Numerical examples are provided to show the effectiveness of the proposed theoretical results.

Sequence analysis by iterated maps, a review

Among alignment-free methods, Iterated Maps (IMs) are on a particular extreme: they are also scale free (order free). The use of IMs for sequence analysis is also distinct from other alignment-free methodologies in being rooted in statistical mechanics instead of computational linguistics. Both of these roots go back over two decades to the use of fractal geometry in the characterization of phase-space representations. The time series analysis origin of the field is betrayed by the title of the manuscript that started this alignment-free subdomain in 1990, 'Chaos Game Representation'. The clash between the analysis of sequences as continuous series and the better established use of Markovian approaches to discrete series was almost immediate, with a defining critique published in same journal 2 years later. The rest of that decade would go by before the scale-free nature of the IM space was uncovered. The ensuing decade saw this scalability generalized for non-genomic alphabets as well as an interest in its use for graphic representation of biological sequences. Finally, in the past couple of years, in step with the emergence of BigData and MapReduce as a new computational paradigm, there is a surprising third act in the IM story. Multiple reports have described gains in computational efficiency of multiple orders of magnitude over more conventional sequence analysis methodologies. The stage appears to be now set for a recasting of IMs with a central role in processing nextgen sequencing results.

Synchronization of Coupled Neutral-Type Neural Networks With Jumping-Mode-Dependent Discrete and Unbounded Distributed Delays

In this paper, the synchronization problem is studied for an array of N identical delayed neutral-type neural networks with Markovian jumping parameters. The coupled networks involve both the mode-dependent discrete-time delays and the mode-dependent unbounded distributed time delays. All the network parameters including the coupling matrix are also dependent on the Markovian jumping mode. By introducing novel Lyapunov-Krasovskii functionals and using some analytical techniques, sufficient conditions are derived to guarantee that the coupled networks are asymptotically synchronized in mean square. The derived sufficient conditions are closely related with the discrete-time delays, the distributed time delays, the mode transition probability, and the coupling structure of the networks. The obtained criteria are given in terms of matrix inequalities that can be efficiently solved by employing the semidefinite program method. Numerical simulations are presented to further demonstrate the effectiveness of the proposed approach.

SYNCHRONIZATION OF STOCHASTIC DELAYED NEURAL NETWORKS WITH MARKOVIAN SWITCHING AND ITS APPLICATION

In this paper, the problem of adaptive synchronization for a class of stochastic neural networks (SNNs) which involve both mixed delays and Markovian jumping parameters is investigated. The mixed delays comprise the time-varying delays and distributed delays, both of which are mode-dependent. The stochastic perturbations are described in terms of Browian motion. By the adaptive feedback technique, several sufficient criteria have been proposed to ensure the synchronization of SNNs in mean square. Moreover, the proposed adaptive feedback scheme is applied to the secure communication. Finally, the corresponding simulation results are given to demonstrate the usefulness of the main results obtained.

DETECTING AND SORTING TARGETING PEPTIDES WITH NEURAL NETWORKS AND SUPPORT VECTOR MACHINES

This paper presents a composite multi-layer classifier system for predicting the subcellular localization of proteins based on their amino acid sequence. The work is an extension of our previous predictor PProwler v1.1 which is itself built upon the series of predictors SignalP and TargetP. In this study we outline experiments conducted to improve the classifier design. The major improvement came from using Support Vector machines as a "smart gate" sorting the outputs of several different targeting peptide detection networks. Our final model (PProwler v1.2) gives MCC values of 0.873 for non-plant and 0.849 for plant proteins. The model improves upon the accuracy of our previous subcellular localization predictor (PProwler v1.1) by 2% for plant data (which represents 7.5% improvement upon TargetP).

DYNAMICS ANALYSIS OF IMPULSIVE STOCHASTIC HIGH-ORDER BAM NEURAL NETWORKS WITH MARKOVIAN JUMPING AND MIXED DELAYS

This paper deals with the problem of asymptotical stability in mean square for a class of impulsive stochastic high-order bi-directional associative memory (BAM) neural networks with mixed delays and Markovian jumping parameters. Based on Lyapunov stability theory, linear matrix inequality and mathematical induction, some sufficient conditions are derived for the asymptotical stability in mean square of the equilibrium point of the neural networks. The results obtained in this paper are new and complement previously known results.