Detection of time delays and directional interactions based on time series from complex dynamical systems

The geometry of synchronization: quantifying the coupling direction of physiological signals of stress between individuals using inter-system recurrence networks

In the study of synchronization dynamics between interacting systems, several techniques are available to estimate coupling strength and coupling direction. Currently, there is no general 'best' method that will perform well in most contexts. Inter-system recurrence networks (IRN) combine auto-recurrence and cross-recurrence matrices to create a graph that represents interacting networks. The method is appealing because it is based on cross-recurrence quantification analysis, a well-developed method for studying synchronization between 2 systems, which can be expanded in the IRN framework to include N > 2 interacting networks. In this study we examine whether IRN can be used to analyze coupling dynamics between physiological variables (acceleration, blood volume pressure, electrodermal activity, heart rate and skin temperature) observed in a client in residential care with severe to profound intellectual disabilities (SPID) and their professional caregiver. Based on the cross-clustering coefficients of the IRN conclusions about the coupling direction (client or caregiver drives the interaction) can be drawn, however, deciding between bi-directional coupling or no coupling remains a challenge. Constructing the full IRN, based on the multivariate time series of five coupled processes, reveals the existence of potential feedback loops. Further study is needed to be able to determine dynamics of coupling between the different layers.

A general model-based causal inference method overcomes the curse of synchrony and indirect effect

To identify causation, model-free inference methods, such as Granger Causality, have been widely used due to their flexibility. However, they have difficulty distinguishing synchrony and indirect effects from direct causation, leading to false predictions. To overcome this, model-based inference methods that test the reproducibility of data with a specific mechanistic model to infer causality were developed. However, they can only be applied to systems described by a specific model, greatly limiting their applicability. Here, we address this limitation by deriving an easily testable condition for a general monotonic ODE model to reproduce time-series data. We built a user-friendly computational package, General ODE-Based Inference (GOBI), which is applicable to nearly any monotonic system with positive and negative regulations described by ODE. GOBI successfully inferred positive and negative regulations in various networks at both the molecular and population levels, unlike existing model-free methods. Thus, this accurate and broadly applicable inference method is a powerful tool for understanding complex dynamical systems.

Conditional cross-map-based technique: From pairwise dynamical causality to causal network reconstruction

Causality detection methods based on mutual cross mapping have been fruitfully developed and applied to data originating from nonlinear dynamical systems, where the causes and effects are non-separable. However, these pairwise methods still have shortcomings in discriminating typical network structures, including common drivers, indirect dependencies, and facing the curse of dimensionality, when they are stepping to causal network reconstruction. A few endeavors have been devoted to conquer these shortcomings. Here, we propose a novel method that could be regarded as one of these endeavors. Our method, named conditional cross-map-based technique, can eliminate third-party information and successfully detect direct dynamical causality, where the detection results can exactly be categorized into four standard normal forms by the designed criterion. To demonstrate the practical usefulness of our model-free, data-driven method, data generated from different representative models covering all kinds of network motifs and measured from real-world systems are investigated. Because correct identification of the direct causal links is essential to successful modeling, predicting, and controlling the underlying complex systems, our method does shed light on uncovering the inner working mechanisms of real-world systems only using the data experimentally obtained in a variety of disciplines.

Effects of climate change on vegetation dynamics of the Qinghai-Tibet Plateau, a causality analysis using empirical dynamic modeling

Given the vital role of the Qinghai-Tibet Plateau (QTP) as water tower in Asia and regulator for regional and even global climate, the relationship between climate change and vegetation dynamics on it has received considerable focused attention. Climate change may influence the vegetation growth on the plateau, but clear empirical evidence of such causal linkages is sparse. Herein, using datasets CRU-TS v4.04 and AVHHR NDVI from 1981 to 2019, we quantify causal effects of climate factors on vegetation dynamics with an empirical dynamical model (EDM) -- a nonlinear dynamical systems analysis approach based on state-space reconstruction rather than correlation. Results showed the following: (1) climate change promotes the growth of vegetation on the QTP, and specifically, this favorable influence of temperature is stronger than precipitation's; (2) the direction and strength of climate effects on vegetation varied over time, and the effects are seasonally different; (3) a significant increase in temperature and a slight increase in precipitation are beneficial to vegetation growth, specifically, NDVI will increase within 2% in the next 40 years with the climate trend of warming and humidity. Besides the above results, another interesting finding is that the two seasons in which precipitation strongly influence vegetation in the Three-River Source region (part of the QTP) are spring and winter. This study provides insights into the mechanisms by which climate change affects vegetation growth on the QTP, aiding in the modeling of vegetation dynamics in future scenarios.

Inferring structural and dynamical properties of gene networks from data with deep learning

The reconstruction of gene regulatory networks (GRNs) from data is vital in systems biology. Although different approaches have been proposed to infer causality from data, some challenges remain, such as how to accurately infer the direction and type of interactions, how to deal with complex network involving multiple feedbacks, as well as how to infer causality between variables from real-world data, especially single cell data. Here, we tackle these problems by deep neural networks (DNNs). The underlying regulatory network for different systems (gene regulations, ecology, diseases, development) can be successfully reconstructed from trained DNN models. We show that DNN is superior to existing approaches including Boolean network, Random Forest and partial cross mapping for network inference. Further, by interrogating the ensemble DNN model trained from single cell data from dynamical system perspective, we are able to unravel complex cell fate dynamics during preimplantation development. We also propose a data-driven approach to quantify the energy landscape for gene regulatory systems, by combining DNN with the partial self-consistent mean field approximation (PSCA) approach. We anticipate the proposed method can be applied to other fields to decipher the underlying dynamical mechanisms of systems from data.

Continuity Scaling: A Rigorous Framework for Detecting and Quantifying Causality Accurately

Data-based detection and quantification of causation in complex, nonlinear dynamical systems is of paramount importance to science, engineering, and beyond. Inspired by the widely used methodology in recent years, the cross-map-based techniques, we develop a general framework to advance towards a comprehensive understanding of dynamical causal mechanisms, which is consistent with the natural interpretation of causality. In particular, instead of measuring the smoothness of the cross-map as conventionally implemented, we define causation through measuring the scaling law for the continuity of the investigated dynamical system directly. The uncovered scaling law enables accurate, reliable, and efficient detection of causation and assessment of its strength in general complex dynamical systems, outperforming those existing representative methods. The continuity scaling-based framework is rigorously established and demonstrated using datasets from model complex systems and the real world.

Inferring causality in biological oscillators

MOTIVATION

Fundamental to biological study is identifying regulatory interactions. The recent surge in time-series data collection in biology provides a unique opportunity to infer regulations computationally. However, when components oscillate, model-free inference methods, while easily implemented, struggle to distinguish periodic synchrony and causality. Alternatively, model-based methods test the reproducibility of time series given a specific model but require inefficient simulations and have limited applicability.

RESULTS

We develop an inference method based on a general model of molecular, neuronal and ecological oscillatory systems that merges the advantages of both model-based and model-free methods, namely accuracy, broad applicability and usability. Our method successfully infers the positive and negative regulations within various oscillatory networks, e.g. the repressilator and a network of cofactors at the pS2 promoter, outperforming popular inference methods.

AVAILABILITY AND IMPLEMENTATION

We provide a computational package, ION (Inferring Oscillatory Networks), that users can easily apply to noisy, oscillatory time series to uncover the mechanisms by which diverse systems generate oscillations. Accompanying MATLAB code under a BSD-style license and examples are available at https://github.com/Mathbiomed/ION. Additionally, the code is available under a CC-BY 4.0 License at https://doi.org/10.6084/m9.figshare.16431408.v1.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Fast and effective pseudo transfer entropy for bivariate data-driven causal inference

Identifying, from time series analysis, reliable indicators of causal relationships is essential for many disciplines. Main challenges are distinguishing correlation from causality and discriminating between direct and indirect interactions. Over the years many methods for data-driven causal inference have been proposed; however, their success largely depends on the characteristics of the system under investigation. Often, their data requirements, computational cost or number of parameters limit their applicability. Here we propose a computationally efficient measure for causality testing, which we refer to as pseudo transfer entropy (pTE), that we derive from the standard definition of transfer entropy (TE) by using a Gaussian approximation. We demonstrate the power of the pTE measure on simulated and on real-world data. In all cases we find that pTE returns results that are very similar to those returned by Granger causality (GC). Importantly, for short time series, pTE combined with time-shifted (T-S) surrogates for significance testing strongly reduces the computational cost with respect to the widely used iterative amplitude adjusted Fourier transform (IAAFT) surrogate testing. For example, for time series of 100 data points, pTE and T-S reduce the computational time by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$82\%$$\end{document}82% with respect to GC and IAAFT. We also show that pTE is robust against observational noise. Therefore, we argue that the causal inference approach proposed here will be extremely valuable when causality networks need to be inferred from the analysis of a large number of short time series.

Exploring node interaction relationship in complex networks by using high-frequency signal injection

Many practical systems can be described by complex networks. These networks produce, day and night, rich data which can be used to extract information from the systems. Often, output data of some nodes in the networks can be successfully measured and collected while the structures of networks producing these data are unknown. Thus, revealing network structures by analyzing available data, referred to as network reconstruction, turns to be an important task in many realistic problems. Limitation of measurable data is a very common challenge in network reconstruction. Here we consider an extreme case, i.e., we can only measure and process the data of a pair of nodes in a large network, and the task is to explore the relationship between these two nodes while all other nodes in the network are hidden. A driving-response approach is proposed to do so. By loading a high-frequency signal to a node (defined as node A), we can measure data of the partner node (node B), and work out the connection structure, such as the distance from node A to node B and the effective intensity of interaction from A to B, with the data of node B only. A systematical smoothing technique is suggested for treating noise problem. The approach has practical significance.

The Human Organism as an Integrated Interaction Network: Recent Conceptual and Methodological Challenges

The field of Network Physiology aims to advance our understanding of how physiological systems and sub-systems interact to generate a variety of behaviors and distinct physiological states, to optimize the organism's functioning, and to maintain health. Within this framework, which considers the human organism as an integrated network, vertices are associated with organs while edges represent time-varying interactions between vertices. Likewise, vertices may represent networks on smaller spatial scales leading to a complex mixture of interacting homogeneous and inhomogeneous networks of networks. Lacking adequate analytic tools and a theoretical framework to probe interactions within and among diverse physiological systems, current approaches focus on inferring properties of time-varying interactions-namely strength, direction, and functional form-from time-locked recordings of physiological observables. To this end, a variety of bivariate or, in general, multivariate time-series-analysis techniques, which are derived from diverse mathematical and physical concepts, are employed and the resulting time-dependent networks can then be further characterized with methods from network theory. Despite the many promising new developments, there are still problems that evade from a satisfactory solution. Here we address several important challenges that could aid in finding new perspectives and inspire the development of theoretic and analytical concepts to deal with these challenges and in studying the complex interactions between physiological systems.

Partial cross mapping eliminates indirect causal influences

Causality detection likely misidentifies indirect causations as direct ones, due to the effect of causation transitivity. Although several methods in traditional frameworks have been proposed to avoid such misinterpretations, there still is a lack of feasible methods for identifying direct causations from indirect ones in the challenging situation where the variables of the underlying dynamical system are non-separable and weakly or moderately interacting. Here, we solve this problem by developing a data-based, model-independent method of partial cross mapping based on an articulated integration of three tools from nonlinear dynamics and statistics: phase-space reconstruction, mutual cross mapping, and partial correlation. We demonstrate our method by using data from different representative models and real-world systems. As direct causations are keys to the fundamental underpinnings of a variety of complex dynamics, we anticipate our method to be indispensable in unlocking and deciphering the inner mechanisms of real systems in diverse disciplines from data.

It is crucial yet challenging to identify cause-consequence relation in complex dynamical systems where direct causal links can mix with indirect ones. Leng et al. propose a data-driven model-independent method to distinguish direct from indirect causality and test its applicability to real-world data.

Control of dynamics via identical time-lagged stochastic inputs

We investigate the impact of a stochastic forcing, comprised of a sum of time-lagged copies of a single source of noise, on the system dynamics. This type of stochastic forcing could be made artificially, or it could be the result of shared upstream inputs to a system through different channel lengths. By means of a rigorous mathematical framework, we show that such a system is, in fact, equivalent to the classical case of a stochastically-driven dynamical system with time-delayed intrinsic dynamics but without a time lag in the input noise. We also observe a resonancelike effect between the intrinsic period of the oscillation and the time lag of the stochastic forcing, which may be used to determine the intrinsic period of oscillations or the inherent time delay in dynamical systems with oscillatory behavior or delays. As another useful application of imposing time-lagged stochastic forcing, we show that the dynamics of a system can be controlled by changing the time lag of this stochastic forcing, in a fashion similar to the classical case of Pyragas control via delayed feedback. To confirm these results experimentally, we set up a laser diode system with such stochastic inputs, which effectively behaves as a Langevin system. As in the theory, a peak emerged in the autocorrelation function of the output signal that could be tuned by the lag of the stochastic input. Our findings, thus, indicate a new approach for controlling useful instabilities in dynamical systems.

Reconstructing directional causal networks with random forest: Causality meeting machine learning

Inspired by the decision tree algorithm in machine learning, a novel causal network reconstruction framework is proposed with the name Importance Causal Analysis (ICA). The ICA framework is designed in a network level and fills the gap between traditional mutual causality detection methods and the reconstruction of causal networks. The potential of the method to identify the true causal relations in complex networks is validated by both benchmark systems and real-world data sets.

Reconstruction of dynamic networks with time-delayed interactions in the presence of fast-varying noises

Most complex social, biological and technological systems can be described by dynamic networks. Reconstructing network structures from measurable data is a fundamental problem in almost all interdisciplinary fields. Network nodes interact with each other and those interactions often have diversely distributed time delays. Accurate reconstruction of any targeted interaction to a node requires measured data of all its neighboring nodes together with information on the time delays of interactions from these neighbors. When networks are large, these data are often not available and time-delay factors are deeply hidden. Here we show that fast-varying noise can be of great help in solving these challenging problems. By computing suitable correlations, we can infer the intensity and time delay of any targeted interaction with the data of two related nodes (driving and driven nodes) only while all other nodes in the network are hidden. This method is analytically derived and fully justified by extensive numerical simulations.

Detecting directional couplings from multivariate flows by the joint distance distribution

The identification of directional couplings (or drive-response relationships) in the analysis of interacting nonlinear systems is an important piece of information to understand their dynamics. This task is especially challenging when the analyst's knowledge of the systems reduces virtually to time series of observations. Spurred by the success of Granger causality in econometrics, the study of cause-effect relationships (not to be confounded with statistical correlations) was extended to other fields, thus favoring the introduction of further tools such as transfer entropy. Currently, the research on old and new causality tools along with their pitfalls and applications in ever more general situations is going through a time of much activity. In this paper, we re-examine the method of the joint distance distribution to detect directional couplings between two multivariate flows. This method is based on the forced Takens theorem, and, more specifically, it exploits the existence of a continuous mapping from the reconstructed attractor of the response system to the reconstructed attractor of the driving system, an approach that is increasingly drawing the attention of the data analysts. The numerical results with Lorenz and Rössler oscillators in three different interaction networks (including hidden common drivers) are quite satisfactory, except when phase synchronization sets in. They also show that the method of the joint distance distribution outperforms the lowest dimensional transfer entropy in the cases considered. The robustness of the results to the sampling interval, time series length, observational noise, and metric is analyzed too.

Delay models for the early embryonic cell cycle oscillator

Time delays are known to play a crucial role in generating biological oscillations. The early embryonic cell cycle in the frog Xenopus laevis is one such example. Although various mathematical models of this oscillating system exist, it is not clear how to best model the required time delay. Here, we study a simple cell cycle model that produces oscillations due to the presence of an ultrasensitive, time-delayed negative feedback loop. We implement the time delay in three qualitatively different ways, using a fixed time delay, a distribution of time delays, and a delay that is state-dependent. We analyze the dynamics in all cases, and we use experimental observations to interpret our results and put constraints on unknown parameters. In doing so, we find that different implementations of the time delay can have a large impact on the resulting oscillations.