Solving the inverse problem of noise-driven dynamic networks

Diverse phase transitions in optimized directed network models with distinct inward and outward node weights

We consider growing directed network models that aim at minimizing the weighted connection expenses while at the same time favoring other important network properties such as weighted local node degrees. We employed statistical mechanics methods to study the growth of directed networks under the principle of optimizing some objective function. By mapping the system to an Ising spin model, analytic results are derived for two such models, exhibiting diverse and interesting phase transition behaviors for general edge weight, inward and outward node weight distributions. In addition, the unexplored cases of negative node weights are also investigated. Analytic results for the phase diagrams are derived showing even richer phase transition behavior, such as first-order transition due to symmetry, second-order transitions with possible reentrance, and hybrid phase transitions. We further extend previously developed zero-temperature simulation algorithm for undirected networks to the present directed case and for negative node weights, and we can obtain the minimal cost connection configuration efficiently. All the theoretical results are explicitly verified by simulations. Possible applications and implications are also discussed.

Efficient reconstruction of directed networks from noisy dynamics using stochastic force inference

We consider coupled network dynamics under uncorrelated noises that fluctuate about the noise-free long-time asymptotic state. Our goal is to reconstruct the directed network only from the time-series data of the dynamics of the nodes. By using the stochastic force inference method with a simple natural choice of linear polynomial basis, we derive a reconstruction scheme of the connection weights and the noise strength of each node. Explicit simulations for directed and undirected random networks with various node dynamics are carried out to demonstrate the good accuracy and high efficiency of the reconstruction scheme. We further consider the case when only a subset of the network and its node dynamics can be observed, and it is demonstrated that the directed weighted connections among the observed nodes can be easily and faithfully reconstructed. In addition, we propose a scheme to infer the number of hidden nodes and their effects on each observed node. The accuracy of these results is illustrated by simulations.

Reconstructing distant interactions of multiple paths between perceptible nodes in dark networks

Quantitative research of interdisciplinary fields, including biological and social systems, has attracted great attention in recent years. Complex networks are popular and important tools for the investigations. Explosively increasing data are created by practical networks, from which useful information about dynamic networks can be extracted. From data to network structure, i.e., network reconstruction, is a crucial task. There are many difficulties in fulfilling network reconstruction, including data shortage (existence of hidden nodes) and time delay for signal propagation between adjacent nodes. In this paper a deep network reconstruction method is proposed, which can work in the conditions that even only two nodes (say A and B) are perceptible and all other network nodes are hidden. With a well-designed stochastic driving on node A, this method can reconstruct multiple interaction paths from A to B based on measured data. The distance, effective intensity, and transmission time delay of each path can be inferred accurately.

Optimized two-dimensional networks with edge-crossing cost: Frustrated antiferromagnetic spin system

We consider a quasi-two-dimensional network connection growth model that minimizes the wiring cost while maximizing the network connections, but at the same time edge crossings are penalized or forbidden. This model is mapped to a dilute antiferromagnetic Ising spin system with frustrations. We obtain analytic results for the order-parameter or mean degree of the optimized network using mean-field theories. The cost landscape is analyzed in detail showing complex structures due to frustration as the crossing penalty increases. For the case of strictly no edge crossing is allowed, the mean-field equations lead to a new algorithm that can effectively find the (near) optimal solution even for this strongly frustrated system. All these results are also verified by Monte Carlo simulations and numerical solution of the mean-field equations. Possible applications and relation to the planar triangulation problem is also discussed.

Effects of hidden nodes on noisy network dynamics

We consider coupled network dynamics under uncorrelated noises, but only a subset of the network and their node dynamics can be observed. The effects of hidden nodes on the dynamics of the observed nodes can be viewed as having an extra effective noise acting on the observed nodes. These effective noises possess spatial and temporal correlations whose properties are related to the hidden connections. The spatial and temporal correlations of these effective noises are analyzed analytically and the results are verified by simulations on undirected and directed weighted random networks and small-world networks. Furthermore, by exploiting the network reconstruction relation for the observed network noisy dynamics, we propose a scheme to infer information of the effects of the hidden nodes such as the total number of hidden nodes and the weighted total hidden connections on each observed node. The accuracy of these results are demonstrated by explicit simulations.

Exploring node interaction relationship in complex networks by using high-frequency signal injection

Many practical systems can be described by complex networks. These networks produce, day and night, rich data which can be used to extract information from the systems. Often, output data of some nodes in the networks can be successfully measured and collected while the structures of networks producing these data are unknown. Thus, revealing network structures by analyzing available data, referred to as network reconstruction, turns to be an important task in many realistic problems. Limitation of measurable data is a very common challenge in network reconstruction. Here we consider an extreme case, i.e., we can only measure and process the data of a pair of nodes in a large network, and the task is to explore the relationship between these two nodes while all other nodes in the network are hidden. A driving-response approach is proposed to do so. By loading a high-frequency signal to a node (defined as node A), we can measure data of the partner node (node B), and work out the connection structure, such as the distance from node A to node B and the effective intensity of interaction from A to B, with the data of node B only. A systematical smoothing technique is suggested for treating noise problem. The approach has practical significance.

Deriving pairwise transfer entropy from network structure and motifs

Transfer entropy (TE) is an established method for quantifying directed statistical dependencies in neuroimaging and complex systems datasets. The pairwise (or bivariate) TE from a source to a target node in a network does not depend solely on the local source-target link weight, but on the wider network structure that the link is embedded in. This relationship is studied using a discrete-time linearly coupled Gaussian model, which allows us to derive the TE for each link from the network topology. It is shown analytically that the dependence on the directed link weight is only a first approximation, valid for weak coupling. More generally, the TE increases with the in-degree of the source and decreases with the in-degree of the target, indicating an asymmetry of information transfer between hubs and low-degree nodes. In addition, the TE is directly proportional to weighted motif counts involving common parents or multiple walks from the source to the target, which are more abundant in networks with a high clustering coefficient than in random networks. Our findings also apply to Granger causality, which is equivalent to TE for Gaussian variables. Moreover, similar empirical results on random Boolean networks suggest that the dependence of the TE on the in-degree extends to nonlinear dynamics.

Identifying the pulsed neuron networks' structures by a nonlinear Granger causality method

Background

It is a crucial task of brain science researches to explore functional connective maps of Biological Neural Networks (BNN). The maps help to deeply study the dominant relationship between the structures of the BNNs and their network functions.

Results

In this study, the ideas of linear Granger causality modeling and causality identification are extended to those of nonlinear Granger causality modeling and network structure identification. We employed Radial Basis Functions to fit the nonlinear multivariate dynamical responses of BNNs with neuronal pulse firing. By introducing the contributions from presynaptic neurons and detecting whether the predictions for postsynaptic neurons’ pulse firing signals are improved or not, we can reveal the information flows distribution of BNNs. Thus, the functional connections from presynaptic neurons can be identified from the obtained network information flows. To verify the effectiveness of the proposed method, the Nonlinear Granger Causality Identification Method (NGCIM) is applied to the network structure discovery processes of Spiking Neural Networks (SNN). SNN is a simulation model based on an Integrate-and-Fire mechanism. By network simulations, the multi-channel neuronal pulse sequence data of the SNNs can be used to reversely identify the synaptic connections and strengths of the SNNs.

Conclusions

The identification results show: for 2–6 nodes small-scale neural networks, 20 nodes medium-scale neural networks, and 100 nodes large-scale neural networks, the identification accuracy of NGCIM with the Gaussian kernel function was 100%, 99.64%, 98.64%, 98.37%, 98.31%, 84.87% and 80.56%, respectively. The identification accuracies were significantly higher than those of a traditional Linear Granger Causality Identification Method with the same network sizes. Thus, with an accumulation of the data obtained by the existing measurement methods, such as Electroencephalography, functional Magnetic Resonance Imaging, and Multi-Electrode Array, the NGCIM can be a promising network modeling method to infer the functional connective maps of BNNs.

Detecting network structures from measurable data produced by dynamics with hidden variables

Depicting network structures from measurable data is of significance. In real-world situations, it is common that some variables of networks are unavailable or even unknown. These unavailable and unknown variables, i.e., hidden variables, will lead to much reconstruction error, even make reconstruction methods useless. In this paper, to solve hidden variable problems, we propose three reconstruction methods, respectively, based on the following conditions: statistical characteristics of hidden variables, linearizable hidden variables, and white noise injection. Among them, the method based on white noise injection is active and invasive. In our framework, theoretic analyses of these three methods are given at first, and, furthermore, the validity of theoretical derivations and the robustness of these methods are fully verified through numerical results. Our work may be, therefore, helpful for practical experiments.

Reconstruction of dynamic networks with time-delayed interactions in the presence of fast-varying noises

Most complex social, biological and technological systems can be described by dynamic networks. Reconstructing network structures from measurable data is a fundamental problem in almost all interdisciplinary fields. Network nodes interact with each other and those interactions often have diversely distributed time delays. Accurate reconstruction of any targeted interaction to a node requires measured data of all its neighboring nodes together with information on the time delays of interactions from these neighbors. When networks are large, these data are often not available and time-delay factors are deeply hidden. Here we show that fast-varying noise can be of great help in solving these challenging problems. By computing suitable correlations, we can infer the intensity and time delay of any targeted interaction with the data of two related nodes (driving and driven nodes) only while all other nodes in the network are hidden. This method is analytically derived and fully justified by extensive numerical simulations.

Fluctuation relations between hierarchical kinetically equivalent networks with Arrhenius-type transitions and their roles in systems and structural biology

The legality of using simple kinetic schemes to determine the stochastic properties of a complex system depends on whether the fluctuations generated from hierarchical equivalent schemes are consistent with one another. To analyze this consistency, we perform lumping processes on the stochastic differential equations and the generalized fluctuation-dissipation theorem and apply them to networks with the frequently encountered Arrhenius-type transition rates. The explicit Langevin force derived from those networks enables us to calculate the state fluctuations caused by the intrinsic and extrinsic noises on the free energy surface and deduce their relations between kinetically equivalent networks. In addition to its applicability to wide classes of network related systems, such as those in structural and systems biology, the result sheds light on the fluctuation relations for general physical variables in Keizer's canonical theory.

Reconstruction of noise-driven nonlinear networks from node outputs by using high-order correlations

Many practical systems can be described by dynamic networks, for which modern technique can measure their outputs, and accumulate extremely rich data. Nevertheless, the network structures producing these data are often deeply hidden in the data. The problem of inferring network structures by analyzing the available data, turns to be of great significance. On one hand, networks are often driven by various unknown facts, such as noises. On the other hand, network structures of practical systems are commonly nonlinear, and different nonlinearities can provide rich dynamic features and meaningful functions of realistic networks. Although many works have considered each fact in studying network reconstructions, much less papers have been found to systematically treat both difficulties together. Here we propose to use high-order correlation computations (HOCC) to treat nonlinear dynamics; use two-time correlations to decorrelate effects of network dynamics and noise driving; and use suitable basis and correlator vectors to unifiedly infer all dynamic nonlinearities, topological interaction links and noise statistical structures. All the above theoretical frameworks are constructed in a closed form and numerical simulations fully verify the validity of theoretical predictions.

Reconstructing network topology and coupling strengths in directed networks of discrete-time dynamics

Reconstructing network connection topology and interaction strengths solely from measurement of the dynamics of the nodes is a challenging inverse problem of broad applicability in various areas of science and engineering. For a discrete-time step network under noises whose noise-free dynamics is stationary, we derive general analytic results relating the weighted connection matrix of the network to the correlation functions obtained from time-series measurements of the nodes for networks with one-dimensional intrinsic node dynamics. Information about the intrinsic node dynamics and the noise strengths acting on the nodes can also be obtained. Based on these results, we develop a scheme that can reconstruct the above information of the network using only the time-series measurements of node dynamics as input. Reconstruction formulas for higher-dimensional node dynamics are also derived and illustrated with a two-dimensional node dynamics network system. Furthermore, we extend our results and obtain a reconstruction scheme even for the cases when the noise-free dynamics is periodic. We demonstrate that our method can give accurate reconstruction results for weighted directed networks with linear or nonlinear node dynamics of various connection topologies, and with linear or nonlinear couplings.

Reconstructing links in directed networks from noisy dynamics

We address the long-standing challenge of how to reconstruct links in directed networks from measurements, and present a general method that makes use of a noise-induced relation between network structure and both the time-lagged covariance of measurements taken at two different times and the covariance of measurements taken at the same time. When the coupling functions have certain additional properties, we can further reconstruct the weights of the links.

Adaptive input data transformation for improved network reconstruction with information theoretic algorithms

We propose a novel systematic procedure of non-linear data transformation for an adaptive algorithm in the context of network reverse-engineering using information theoretic methods. Our methodology is rooted in elucidating and correcting for the specific biases in the estimation techniques for mutual information (MI) given a finite sample of data. These are, in turn, tied to lack of well-defined bounds for numerical estimation of MI for continuous probability distributions from finite data. The nature and properties of the inevitable bias is described, complemented by several examples illustrating their form and variation. We propose an adaptive partitioning scheme for MI estimation that effectively transforms the sample data using parameters determined from its local and global distribution guaranteeing a more robust and reliable reconstruction algorithm. Together with a normalized measure (Shared Information Metric) we report considerably enhanced performance both for in silico and real-world biological networks. We also find that the recovery of true interactions is in particular better for intermediate range of false positive rates, suggesting that our algorithm is less vulnerable to spurious signals of association.

Recovery of couplings and parameters of elements in networks of time-delay systems from time series

We propose a method for the recovery of coupling architecture and the parameters of elements in networks consisting of coupled oscillators described by delay-differential equations. For each oscillator in the network, we introduce an objective function characterizing the distance between the points of the reconstructed nonlinear function. The proposed method is based on the minimization of this objective function and the separation of the recovered coupling coefficients into significant and insignificant coefficients. The efficiency of the method is shown for chaotic time series generated by model equations of diffusively coupled time-delay systems and for experimental chaotic time series gained from coupled electronic oscillators with time-delayed feedback.

Consistency between functional and structural networks of coupled nonlinear oscillators

In data-based reconstruction of complex networks, dynamical information can be measured and exploited to generate a functional network, but is it a true representation of the actual (structural) network? That is, when do the functional and structural networks match and is a perfect matching possible? To address these questions, we use coupled nonlinear oscillator networks and investigate the transition in the synchronization dynamics to identify the conditions under which the functional and structural networks are best matched. We find that, as the coupling strength is increased in the weak-coupling regime, the consistency between the two networks first increases and then decreases, reaching maximum in an optimal coupling regime. Moreover, by changing the network structure, we find that both the optimal regime and the maximum consistency will be affected. In particular, the consistency for heterogeneous networks is generally weaker than that for homogeneous networks. Based on the stability of the functional network, we propose further an efficient method to identify the optimal coupling regime in realistic situations where the detailed information about the network structure, such as the network size and the number of edges, is not available. Two real-world examples are given: corticocortical network of cat brain and the Nepal power grid. Our results provide new insights not only into the fundamental interplay between network structure and dynamics but also into the development of methodologies to reconstruct complex networks from data.